WO2022253027A1

WO2022253027A1 - Underwater sonar image matching method based on gaussian distribution clustering

Info

Publication number: WO2022253027A1
Application number: PCT/CN2022/094444
Authority: WO
Inventors: 王慧; 邱海洋; 智鹏飞; 董苗
Original assignee: 江苏科技大学
Priority date: 2021-05-31
Filing date: 2022-05-23
Publication date: 2022-12-08
Also published as: CN113313172A; CN113313172B

Abstract

An underwater sonar image matching method based on Gaussian distribution clustering. Accurate three-dimensional reconstruction is performed on a two-dimensional sonar image by means of performing image registration and optimizing sonar three-dimensional motion parameters. The method comprises the following steps: step A: extracting features, establishing a matching relationship between the features, and obtaining elevation information of a map; and step B: performing position and posture estimation, and updating a feature map, so as to generate a three-dimensional space map. By means of the present invention, feature extraction to reconstruction of an environment map covering elevation information are realized for a sonar image, and certain motion posture estimation information is also provided, which can be used in the field of underwater robot sonar image processing and mapping.

Description

Underwater sonar image matching method based on Gaussian distribution clustering

technical field

The invention relates to the fields of ship and marine technology and sonar image processing, in particular to methods for underwater detection and perception and three-dimensional reconstruction of targets, and specifically proposes an underwater sonar image matching method based on Gaussian distribution clustering.

Background technique

In the past decade, various feature-based, template-based, region-based, and Fourier-based methods for two-dimensional FS sonar image registration have been explored, and high-resolution two-dimensional (2D) Forward-scanning beam (FS) sonar video systems are commercially available for operation and imaging in turbid waters. But previous studies are generally based on single feature points, and then image registration is formulated as an optimization problem, using a normal distribution transformation. But the choice of grid size involves a trade-off between resolution and computation time and is not necessarily automatic. The distribution of points does not obey the univariate Gaussian distribution. The previous calculation of Gaussian distribution will lead to the wrong representation of the real distribution, and the optimization time of complex grid structure calculation is too long. When the elevation map is generated, the elevation angle will be lost when the 3D is projected onto the 2D sonar image, which will cause a large error in the reconstruction target, and the imperfection of the optimization method will affect image registration and accurate 3D reconstruction.

Contents of the invention

The purpose of the present invention is to develop an underwater sonar image matching method based on Gaussian distribution clustering, through image registration and optimization of sonar three-dimensional motion parameters, to carry out three-dimensional underwater environment reconstruction through two-dimensional sonar images.

A method for matching underwater sonar images based on Gaussian distribution clustering, comprising the following steps:

Step A, extract features, establish the matching relationship between features, and obtain the height information of the map; specifically, it includes the following three steps:

Step A1, collecting sonar data and performing feature detection;

Step A2, image registration, generating a Gaussian map;

Step A3, calculating the scene elevation map;

Step B, perform pose estimation, and update the feature map to generate a three-dimensional space map; specifically, the following steps are included:

Step B1, pose estimation;

Step B2, parameter optimization;

Step B3, generating a three-dimensional map.

Further, collecting sonar data in step A1, performing feature extraction includes the following 4 steps:

Step A1-1, collecting sonar data, selecting an area with obvious pixels from the sonar data image as the target processing area;

Step A1-2, use the gray level distribution information of the target feature area and analyze the scene to divide the target area into three categories, distinguish the target pixel area and other non-target areas, and eliminate the pixels caused by noise according to the gray level distribution information Region, to realize the description of feature points.

Further, it is characterized in that step A-1-2 includes the following two steps:

Step A1-2a, using the gray distribution information of the target feature area and through scene analysis, the relatively obvious gray value in the sonar image is divided into three categories: bright small spots that form 3D targets or structures, and adjacent to 3D targets The shaded area cast by the surface of , a flat surface between the first two grayscale values;

Step A1-2b, through scene analysis, take the bright small spots that make up the 3D target or structure as the target area, distinguish the target pixels from other non-target areas, remove the pixel area due to noise, and reduce the image error.

Further, image registration in step A2, generating a Gaussian map includes the following steps:

Step A2-1, clustering target pixels with strong pixel intensity by K-means clustering method;

Step A2-2, using low-pass filtering to remove noise and reduce errors in feature extraction.

Further, step A2-1 includes the following steps:

In step A2-1a, the target feature points obtained in step A1-2b are gridded, and based on the points in the grid, each gridded target pixel is clustered in the grid through the K-means clustering method;

Step A2-1b, use Gaussian distribution to represent the target feature points clustered by k-means, calculate the eigenvalues and eigenvectors of the covariance matrix to reflect the direction and smoothness of the surface, and use the cluster mean and covariance to represent any Clustering, generating a Gaussian map. The mean vector and covariance matrix correspond to the location, size and orientation of each 2D image region.

Further, step A2-1b includes the following two steps:

Step A2-1b-1, select 1%-2% of the pixels with high brightness, remove the pixels less than 8, select the k value so that any sub-region is not larger than 32 pixels, and divide the region larger than 32 pixels into smaller regions, Get a suitable feature area;

Step A2-1b-2, calculating the mean value vector and covariance matrix corresponding to the position, size and direction of each two-dimensional image region, and generating a Gaussian map.

Further, calculating the scene elevation map in step A3 includes the following steps:

Step A3-1, according to the planarity assumption method, calculate the measurement value of the elevation angle of the projected shadow point of the scanning azimuth;

Step A3-2, use the image points of the minimum and maximum distance values to fix the plane, that is, the three-dimensional coordinates of the front point and the trailing edge point are fixed, similar point pairs are scanned in different directions, and the occlusion contour points are determined by projecting shadows to establish The corresponding relationship between the elevation angle of the target object and the shadow, the elevation value of each 3D object from the front end to the back end is filled by linear interpolation;

Step A3-3, process the smoothed data through low-pass filtering to reduce the influence of image noise on local peaks; through k-means segmentation, establish thresholds for background, objects and shadow areas, that is, after clustering, according to gray The degree value is divided into three categories, dividing the image category to locate the conversion of objects to shadows and shadows to the ground;

In step A3-4, the size of the 3D object is estimated by the elevation angle of the trailing edge point and the angle of the occlusion contour, and the scene elevation map is calculated using the following formula:

In the formula, Hs is the distance from the sonar to the bottom, Rs is the slant distance, Ls is the shadow length, Ht is the height of the target, and Lt is the length of the target.

Further, step B1 pose estimation includes the following steps:

In step B1-1, in the scene elevation map obtained in step A3-4, select the first frame of sonar image as the reference image, and the second frame of image as the image to be matched, and each image point is moved to the new sonar image after rigid sonar motion. The position of the two sonar images is registered by calculating the frame-to-frame motion parameters;

Step B1-2, calculate the sonar motion parameters, seek the most suitable transformation characteristics from the reference image to the image to be matched, use the space transformation function to estimate the pose, and compare the transformation of the two images, such as rotation and translation, to obtain the best Transform parameters; then each time the previous registration image is used as a reference to register the next frame of sonar images until the registration of all sonar images to be matched in the entire image sequence is completed, and the registration of all adjacent frames Uniform into adjacent frames of reference to reduce accumulated errors caused by pairwise registrations.

Further, step B2 parameter optimization includes the following steps:

In step B2-1, each image point of the elevation map obtained in step A3-4 is projected to the corresponding space point, and after sonar rigid motion, the new position 3D scene point is calculated to obtain two sonar views;

Step B2-2, optimize the parameters of the two sonar views, transform the 3D points of the second view into the coordinate system of the first view, and evaluate the characteristic Gaussian distribution of all transformations for the best registration.

Further, generating the three-dimensional map in step B3 includes the following steps:

After the parameters are optimized according to step B2, the calculation speed is accelerated to obtain the optimal solution, and the sonar trajectory relative to the initial position is calculated to complete accurate three-dimensional reconstruction.

The beneficial effects achieved by the present invention are: realizing feature extraction of sonar images to reconstruction of environmental maps covering elevation information, and providing certain motion attitude estimation information, which can be used in the field of sonar image processing and mapping of underwater robots.

Description of drawings

Fig. 1 is a flow chart of the whole sonar image processing in the embodiment of the present invention.

Fig. 2 is a target processing area with obvious pixels in the sonar data image in the embodiment of the present invention.

FIG. 3 is a three-level pixel area obtained through scene analysis in an embodiment of the present invention.

Fig. 4 is an image processed by seabed sonar image and k-means clustering in the embodiment of the present invention.

FIG. 5 shows the Gaussian distribution used to represent the target pixels of k-means clustering processing in the embodiment of the present invention.

Fig. 6 is an elevation map calculated by using elevation angle information in an embodiment of the present invention.

Fig. 7 is a sonar view 1 and a view 2 obtained by rigid motion in an embodiment of the present invention.

Detailed ways

The technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings.

The purpose of the present invention is to develop an underwater sonar image matching method based on Gaussian distribution clustering, through image registration and optimization of sonar three-dimensional motion parameters, three-dimensional reconstruction of underwater environment through two-dimensional sonar images, the overall steps are shown in the figure 1, including the following steps:

Step A: Extract features, establish matching relationships between features, and obtain map height information;

Step B: Perform pose estimation and update the feature map to generate a three-dimensional space map.

The feature detection in step A includes the following steps: step A1 collects sonar data and performs feature detection; step A2 image registration to generate a Gaussian map; step A3 calculates the scene elevation map.

Collecting sonar data in step A1, performing feature extraction includes the following two steps:

Step A1-1, collect sonar data, and select an area with obvious pixels from the sonar data image as the target processing area, as shown in FIG. 2 .

Step A1-2 includes the following 2 steps:

Step A1-2a, using the gray distribution information of the target feature area and analyzing the scene, the relatively obvious gray values in the sonar image are divided into three categories, as shown in Figure 3: Small spots; 2. Shadow areas cast by surfaces adjacent to the 3D object; 3. Flat surfaces between the first two grayscale values.

Step A1-2b, through scene analysis and pixel intensity, distinguish target pixels from other non-target areas, remove pixel areas due to noise, and reduce image errors.

Image registration in step A2, generating a Gaussian map includes the following steps:

In step A2-1, the target pixels with strong pixel intensity are clustered by the K-MEANS clustering method, as shown in FIG. 4 .

As a further definition of the present invention, step A2-1 includes the following steps:

In step A2-1a, the target feature points obtained in step A1-2b are gridded, and based on the points in the grid, each gridded target pixel is clustered in the grid through the K-means clustering method, such as Figure 5, select 1%-2% of the pixels with high brightness, eliminate the pixels less than 8, select the k value so that any sub-region is not greater than 32 pixels, divide the region greater than 32 pixels into smaller regions, and obtain suitable features area.

Step A2-1b, use Gaussian distribution to represent the target feature points clustered by k-means, calculate the eigenvalues and eigenvectors of the covariance matrix by formula 1 and formula 2 to reflect information such as the direction and smoothness of the surface, and use the cluster mean and covariance represent any clustering, yielding a Gaussian map. The mean vector and covariance matrix correspond to the location, size and orientation of each 2D image region.

Among them, all scanning points in a grid are shown, and the i-th point in the set of Nj points in area j (a total of R areas) is indicated by subscripts, superscripts, and value ranges. Feature region j is represented by Gaussian mean value μj and variance Σj as shown in Figure 3. Different from the normal distribution, the cluster-based Gaussian map (statistical mean, covariance) representation eliminates the need to maintain a cumbersome grid structure, while Reduce computation time during optimization without losing precision.

Calculating the scene elevation map in step A3 includes the following steps:

Step A3-1, calculate according to the method of planarity assumption, estimate the elevation angle of the scene relative to the flat part and the point on the three-dimensional object. The planarity assumption basically estimates the elevation angles of points on relatively flat parts of the scene and on 3-D objects (based on the shadows they cast).

Step A3-2, use the image points of the minimum and maximum distance values to fix the plane, that is, the three-dimensional coordinates of the front point and the trailing edge point are fixed, as shown in Figure 6, similar point pairs are scanned in different orientations, and are projected The shadow determines the occlusion contour point to establish the corresponding relationship between the elevation angle of the target object and the shadow, and the elevation value of each 3D object from the front end to the back end is filled by linear interpolation.

Step A3-3, process the smoothed data by low-pass filtering to reduce the influence of image noise on local peaks; through k-means segmentation, establish thresholds for background, objects and shadow regions (in step A1-2 Create) positioning object to shadow and shadow to ground transformations.

Step A3-4, using Formula 3 to estimate the size of the three-dimensional object through the elevation angle of the trailing edge point and the angle of the occlusion contour, and calculate the scene elevation map. Small objects may not have obvious shadows, for these objects, set the object height to zero.

In the formula, Hs: distance detected by sonar to the bottom, Rs: slant distance, Ls: shadow length, Rh: horizontal distance, Ht: target height, Lt: target length.

Step B includes the following steps: step B1 pose estimation; step B2 parameter optimization; step B3 generate a three-dimensional map.

Step B1 pose estimation includes the following steps:

Step B1-1, in the scene elevation map obtained in step A3-4, select the first frame of sonar image as the reference image, and the second frame of image as the image to be matched, and each image point can be moved to The new position, as shown in Figure 7, is used to register the two sonar images by calculating frame-to-frame motion parameters.

The elevation angle is lost when the 3D world is projected onto the 2D sonar image, which can be expressed as the mapping h(P _s ) on the zero-elevation (X _s , Y _s ) plane of the 3D scene point P _s along the distance and azimuth of the 3D point, and calculate The formula is:

In the formula, R is the distance from the sonar beam to the reflected target, and Φ is the azimuth angle.

The three-dimensional coordinate system P _s = (X _s Y _s Z _s ) ^T of the point P in the sonar coordinate system can be expressed by (R,θ,Φ) ^T , and the conversion formula of Cartesian and spherical sonar coordinates is:

Use 6 components to describe the three-dimensional motion of the sonar, T=[t _x , _ty ,t _z ] ^T and W=[w _x ,w _y ,w _z ] ^T , the two-dimensional image does not include the rotation component [w _x ,w _y ,], register the two images by calculating the frame-to-frame motion parameters [t _x ,t _y ,t _z ,w _z ].

Step B1-2, calculate the sonar motion parameters, seek the most suitable transformation characteristics from the reference image to the image to be matched, use the spatial transformation function to estimate the pose, and use the formula 7 to project the first image to the second image S′ , the transformation of the two images, that is, the comparison of rotation and translation. The movement of the sonar position produces two scene images (in the same range R, at different elevation angles). Accept the motion parameters [t _x , t _y , t _z , w _z ], find the minimum S and S' (formula 9), and get the best transformation parameters; then use the previous registration image as a reference each time , to register the next frame of sonar images until the registration of all sonar images to be matched in the entire image sequence is completed, and the registration of all adjacent frames is unified into the adjacent reference frame to reduce the error caused by pairwise registration accumulate errors.

S'＝HS (9)

where H is a transformation matrix including translation and in-plane rotation.

Step B2 parameter optimization comprises the following steps:

Step B2-1, each image point of the elevation map obtained by step A3-4 can be projected to the corresponding space point P _'s by using formula 5. After the component m=(T, W) sonar rigid motion, the new position The three-dimensional scene point P' is determined by Equation 10 and Equation 11.

In the formula, R is the rotation matrix, and W is the rotation velocity vector.

S=h(P _s ) and S'=h(P' _s ) corresponding to the sonar image are determined by Formula 9 and Formula 12.

After the sonar rigid motion, the new position of the 3D scene point is calculated, and two sonar views are obtained.

Step B2-2 includes the following steps:

The parameters of the two sonar views obtained in step B2-1 are optimized by Equation 13, and the 3D points of the second view are transformed into the coordinate system of the first view. The best registration evaluates the characteristic Gaussian distribution of all transformations produces the maximum function value.

In the formula, G ^j (s) represents the Gaussian distribution in the first view (corresponding to the jth feature), G′ ^j (s) represents the Gaussian distribution in the second view (corresponding to the jth feature), P _si and P′ _si represents the 3D scene corresponding to the feature point sets of the two views, and M ^-1 transforms the 3D points of the second view into the coordinate system of the first view.

Step B3 generating a three-dimensional map includes the following steps:

After optimizing the parameters according to step B2, directly map from the first image to the second view of S′, and use the transformation matrix H=(M, R, Φ) in the formula 8 to calculate the speed and optimize as the key situation , cosΦ≈1, S=h(MP _s ) and S′=h(M ⁻¹ P _s ) are calculated by Equation 14.

The homogeneous transformation M is recursively calculated according to formula 15 to calculate the sonar trajectory relative to the initial position, and complete accurate three-dimensional reconstruction.

^k M ₀ = ^k M _k-1 ^k-1 M ₀ (15)

In the formula, k represents the number of frames (time index).

The above descriptions are only preferred embodiments of the present invention, and the scope of protection of the present invention is not limited to the above embodiments, but all equivalent modifications or changes made by those of ordinary skill in the art according to the disclosure of the present invention should be included within the scope of protection described in the claims.

Claims

A kind of underwater sonar image matching method based on Gaussian distribution clustering, it is characterized in that: described method comprises the following steps:

Step A, extract features, establish the matching relationship between features, and obtain the elevation information of the map; specifically, it includes the following three steps:

Step A1, collecting sonar data and performing feature extraction;

Step A2, image registration, generating a Gaussian map;

Step A3, calculating the scene elevation map;

Calculating the scene elevation map in step A3 includes the following steps:

Step A3-1, according to the method of planarity assumption, estimating the elevation angle of the point on the relatively flat part of the scene and the three-dimensional object; estimating the elevation angle of the point on the relatively flat part and the 3-D object in the scene based on the cast shadow;

Step A3-2, use the image points of the minimum and maximum distance values to fix the plane, that is, the three-dimensional coordinates of the front point and the trailing edge point are fixed, similar point pairs are scanned in different directions, and the occlusion contour points are determined by projecting shadows to establish The corresponding relationship between the elevation angle of the target object and the shadow, the elevation value of each 3D object from the front end to the back end is filled by linear interpolation;

Step A3-3, process the smoothed data through low-pass filtering to reduce the influence of image noise on local peaks; through k-means segmentation, establish thresholds for background, objects and shadow areas, and locate objects to shadows and shadows conversion to ground;

In step A3-4, the size of the 3D object is estimated by the elevation angle of the trailing edge point and the angle of the occlusion contour, and the scene elevation map is calculated using the following formula:

In the formula, Hs is the distance from the sonar to the seabed, Rs is the slant distance, Ls is the shadow length, Ht is the height of the target, and Lt is the length of the target;

Step B, perform pose estimation, and update the feature map to generate a three-dimensional space map; specifically, the following steps are included:

Step B1, pose estimation;

Step B2, parameter optimization;

Step B3, generating a three-dimensional map, includes the following steps:

After the parameters are optimized according to step B2, the calculation speed is accelerated to obtain the optimal solution, and the sonar trajectory relative to the initial position is calculated to complete accurate three-dimensional reconstruction.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 1, characterized in that: collecting sonar data in step A1, performing feature extraction comprises the following 2 steps:

Step A1-1, collecting sonar data, selecting an area with obvious pixels from the sonar data image as the target processing area;

Step A1-2, use the gray level distribution information of the target feature area and analyze the scene to divide the target area into three categories, distinguish the target pixel area and other non-target areas, and eliminate the pixels caused by noise according to the gray level distribution information Region, to realize the description of feature points.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 2, characterized in that: step A1-2 comprises the following two steps:

Step A1-2a, using the gray distribution information of the target feature area and through scene analysis, the relatively obvious gray value in the sonar image is divided into three categories: bright small spots that form 3D targets or structures, and adjacent to 3D targets The shaded area cast by the surface of , a flat surface between the first two grayscale values;

Step A1-2b, through scene analysis and pixel intensity, distinguish the target pixel area from other non-target areas, remove the pixel area due to noise, and reduce the image error.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 1, it is characterized in that: image registration in step A2, generating Gaussian map comprises the following steps:

Step A2-1, clustering target pixels with strong pixel intensity by K-means clustering method;

Step A2-2, using low-pass filtering to remove noise and reduce errors in feature extraction.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 4, it is characterized in that: step A2-1 comprises the following steps:

Step A2-1a, grid the obtained target feature points, based on the points in the grid, use the K-means clustering method to cluster each grid target pixel in the grid;

Step A2-1b, use Gaussian distribution to represent the target feature points clustered by k-means, calculate the eigenvalues and eigenvectors of the covariance matrix to reflect the direction and smoothness information of the surface, and use the cluster mean and covariance to represent any cluster Class that generates a Gaussian map; the mean vector and covariance matrix correspond to the location, size, and orientation of each 2D image region.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 5, it is characterized in that: step A2-1a comprises the following steps:

Select 1%-2% of the pixels with high brightness, eliminate the pixels smaller than 8, make any sub-region not larger than 32 pixels, divide the region larger than 32 pixels into smaller regions, and obtain suitable feature regions.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 1, it is characterized in that: step B1 pose estimation comprises the following steps:

Step B1-1, in the obtained scene elevation map, select the first frame of sonar image as the reference image, and the second frame of image as the image to be matched. Compute frame-to-frame motion parameters to register two sonar images;

Step B1-2, calculate the sonar motion parameters, find the most suitable transformation characteristics from the reference image to the image to be matched, use the space transformation function to estimate the pose, and compare the transformation of the two images, that is, the rotation and translation, to obtain the best transformation parameters; then each time the previous registration image is used as a reference to register the next frame of sonar images until the registration of all sonar images to be matched in the entire image sequence is completed, and the registration of all adjacent frames is unified into adjacent frames of reference to reduce the cumulative error caused by pairwise registration.
A kind of underwater sonar image matching method based on Gaussian distribution clustering according to claim 1, is characterized in that: step B2 parameter optimization comprises the following steps:

In step B2-1, each image point of the elevation map obtained in step A3-4 is projected to the corresponding space point, and after sonar rigid motion, the new position 3D scene point is calculated to obtain two sonar views;

Step B2-2, optimize the parameters of the two sonar views, transform the 3D points of the second view into the coordinate system of the first view, and evaluate the characteristic Gaussian distribution of all transformations for the best registration.