CN116597164A

CN116597164A - Port ship waterline detection method based on computer vision

Info

Publication number: CN116597164A
Application number: CN202310571299.5A
Authority: CN
Inventors: 余建平; 王志立; 陆成刚; 陈志斌
Original assignee: Hangzhou Shuju Chain Technology Co ltd
Current assignee: Hangzhou Shuju Chain Technology Co ltd
Priority date: 2023-05-18
Filing date: 2023-05-18
Publication date: 2023-08-15

Abstract

The application discloses a port ship waterline detection method based on computer vision, which comprises the following steps of: inputting a photographed waterline image; carrying out image preprocessing on the waterline image to obtain a preprocessed image; performing image feature separation operation on the preprocessed image to obtain a waterline feature map; and obtaining the waterline position through edge detection and marking based on the waterline characteristic diagram, and marking the waterline. According to the method, the influence of noise can be reduced to the greatest extent through image preprocessing, other useless features can be removed through image feature separation operation, and then the position of the water outlet line can be accurately found and marked through edge detection and marking, so that a foundation is laid for further load calculation.

Description

Port ship waterline detection method based on computer vision

Technical Field

The application belongs to the technical field of ship detection, and particularly relates to a port ship waterline detection method based on computer vision.

Background

In the ship port entering process, customs needs to detect the draft of the ship, and calculates the load of the port entering ship by combining other data provided by the ship. The existing solution is to let port staff walk around the ship by a circle by taking a boat, observe the scale of the water gauge in front of, in the middle of and behind the ship, and calculate the load of the ship by combining other data provided by the ship. Later along with the development of four-axis unmanned aerial vehicle, carry cloud platform camera through four-axis unmanned aerial vehicle and replace the water gauge mark of boats and ships and shoot, harbour staff utilize the picture that unmanned aerial vehicle sent to the water gauge scale of boats and ships to research and judge. Both modes have respective limitations, no matter the boat or the unmanned aerial vehicle is, the boat or the unmanned aerial vehicle is always influenced by subjective factors of port staff when detecting the water gauge reading, meanwhile, the objectivity of the reading can be influenced by the environment of the water gauge reading, such as weak light, fog, stormy waves and the like, and particularly, under the environment of large stormy waves, the water gauge reading acquired by the boat or the unmanned aerial vehicle has large errors.

Disclosure of Invention

In order to solve the technical problems, the application provides a port ship waterline detection method based on computer vision, which aims to solve the problem that in the prior art, the water gauge reading has larger error.

In order to achieve the above purpose, the application provides a port ship waterline detection method based on computer vision, which comprises the following steps:

inputting a photographed waterline image;

carrying out image preprocessing on the waterline image to obtain a preprocessed image;

performing image feature separation operation on the preprocessed image to obtain a waterline feature map;

and obtaining the waterline position through edge detection and marking based on the waterline characteristic diagram, and marking the waterline.

Preferably, the image preprocessing process for the waterline image includes: converting the water line image into an RGB format;

filtering the RGB format waterline image to remove image noise;

and carrying out brightness compensation on the filtered waterline image to obtain a preprocessed image.

Preferably, the method of removing image noise includes:

removing Gaussian noise of RGB format waterline images through Gaussian filtering;

and removing the salt and pepper noise of the RGB format waterline image through median filtering.

Preferably, the method of brightness compensation includes: setting a brightness threshold value, and converting the filtered RGB format waterline image into an HSV format;

extracting a brightness value matrix of the HSV format waterline image, traversing brightness values in the brightness value matrix, reserving brightness values higher than a brightness threshold value, and setting the brightness values lower than the brightness threshold value as the brightness threshold value;

and converting the HSV format waterline image into an RGB format waterline image.

Preferably, the method for removing gaussian noise comprises: a weighted average of pixel values in the gaussian seed image is created.

Preferably, the process of obtaining the waterline characteristic map by performing the image characteristic separation operation includes: based on the pixel matrix of the preprocessed image, an objective function is obtained through a Potts model, wherein the objective function is expressed as follows:

where f represents the input image pixel matrix and the fidelity of the data is represented by the norm L ² Deciding that u represents a piecewise constant function that can skip or discontinuously encode boundaries of corresponding image features;as image feature boundaries; gamma is an empirical constant, the larger the constant, the fewer image features are obtained, and conversely the more image constants are obtained;

decomposing the objective function into two secondary optimization problems by adopting an ADMM algorithm;

optimizing the two secondary optimization problems through dynamic iteration to obtain a global minimum value and a weight coefficient array;

the global minimum value and the weight coefficient array are brought into an ADMM algorithm, and a waterline image subjected to feature separation is obtained;

and carrying out feature unification and image corrosion treatment on the waterline image subjected to feature separation to obtain a waterline feature image.

Preferably, the method for performing feature unified processing on the waterline image subjected to feature separation comprises the following steps: the waterline image features include hull, sea water, rust, waterline and spray;

classifying and counting pixels in the waterline image subjected to feature separation, and screening out two pixels with the largest proportion as hull and sea water features;

and taking the maximum two pixels as a reference, calculating the difference between other pixels and the maximum two pixels, and converting into the pixels with the same completion characteristics when the difference between the other pixels and the maximum two pixels is small from the pixels of the two pixels.

Preferably, the process of edge detection includes: calculating pixel gradients using a Sobel operator;

eliminating stray effect by adopting a non-maximum pixel gradient suppression method;

eliminating small gradient modulus points through threshold hysteresis processing;

and screening out edge points through isolated weak edge inhibition to obtain a waterline point set.

Preferably, the waterline detection method further comprises the step of constructing a linear equation according to the marked waterline, and the method for constructing the linear equation comprises the following steps:

converting the marked waterline into a linear equation by adopting Hough conversion when the sea waves calm;

when the sea wave is not calm, a centering method is adopted, the highest point and the lowest point in the point set are taken, a straight line is drawn as a straight line waterline after an average value is made, and a straight line equation is obtained according to the straight line waterline.

Compared with the prior art, the application has the following advantages and technical effects:

according to the port ship waterline detection method based on computer vision, noise influence can be reduced to the greatest extent through image preprocessing, other useless features can be removed through image feature separation operation, and the position of a water outlet line is accurately found and marked through edge detection and marking, so that a foundation is laid for further load calculation.

Drawings

The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:

fig. 1 is a flowchart of a method for detecting a waterline of a port vessel according to an embodiment of the present application.

Detailed Description

It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.

As shown in fig. 1, the application provides a port ship waterline detection method based on computer vision, which comprises the following steps:

inputting a photographed waterline image;

In a further preferred embodiment, the method for preprocessing the image comprises the following steps:

the image is first imported into matlab and converted to RGB format, the order of preprocessing of the image is as follows:

1. gaussian filtering

The objective of gaussian filtering is mainly to filter out gaussian noise in the image, the principle being to use the isotropy and separability of the gaussian kernel to create a suitable gaussian kernel for weighted averaging of the pixel values in the image. Since the image is two-dimensional and therefore belongs to a two-dimensional gaussian distribution, the following equation can be listed:

considering that the image pixels are present in discrete form in the matrix, a discrete approximation is required. The Gaussian kernel set here is [2,4,5,4,2;4,9,12,9,4;5,12,15,12,5;4,9,12,9,4;2,4,5,4,2 in practice, each pixel in the waterline image is scanned with the convolution matrix, and the value of the convolution center pixel point is replaced with a weighted average of pixels in the neighborhood determined by the convolution.

2. Median filtering

After the Gaussian filtering is finished, median filtering is carried out below, and the purpose is to remove salt and pepper noise in the waterline image, which is caused by image cutting. The formula for median filtering an image of size M x N using a template of M x N is as follows:

where m=2a+1, n=2b+1.

In order to prevent the differences between the hull and sea features from being blurred and to make the internal pixels of the hull and sea features as uniform as possible, the template size is selected to be 1*3, i.e., [1, 1], in consideration of the particularity of the waterline image, so that the blurring of the longitudinal pixel differences can be avoided and the internal pixel differences between the hull and sea features in the transverse direction can be reduced.

3. Brightness compensation

It is considered that even if the median filtering is performed, there is still a large difference in the internal pixels of the hull feature and the sea water feature due to the uneven brightness, and thus brightness compensation is required. Setting a brightness threshold H, converting the processed RGB image into an HSV format, extracting a brightness value matrix, traversing brightness values in the RGB image, reserving brightness values higher than the brightness threshold H, setting brightness values lower than the brightness threshold H as H, converting the brightness values into the RGB format, and entering the next link.

In a further preferred embodiment, the method for separating the image features comprises the following steps:

image feature separation is primarily considered herein as a minimum problem for a particular penalty function that is set to minimize the total boundary length of the feature image being separated, and thus the problem can ultimately be generalized to a classical Potts model, as shown in the following equation:

where f represents the input image pixel matrix and the fidelity of the data is represented by the norm L ² It is decided that u represents a piecewise constant function that can skip or discontinuously encode the boundaries of the corresponding image feature.May be considered as image feature boundaries. Gamma is an empirical constant, generally the larger the constant, the fewer image features are obtained, and conversely the more image constants are obtained.

In order to be able to adapt to the separation of image features, the model can be written as follows:

where f is the pixel matrix from the waterline RGB image, u _i,j Is regarded as a vector function of the pixel, otherwise inside [ · ]]Is considered as an iferson bracket, and satisfies the condition 1, but does not satisfy 0. Adjacency system 8 and non-negative weighting omega decisionThe length of the discrete boundaries of the image features is determined. Based on the above model, continuing to supplement and perfect, the jump penalty function can be written as:

and applying a lagrangian, the model can evolve into the following equation:

where λ is the multivariate Lagrangian multiplier matrix. To this end, an optimized objective function has been determined, which is decomposed into two secondary optimization problems using the ADMM strategy in order to solve the objective function, and iterated until the difference between u and v is below the threshold stop. The first of two secondary optimization problems is an iteration of solving u:

the second is the iteration of solving v:

where μ and λ are also iterated, the iterative formula μ=τμ and λ=λ+μ (u-v). The inputs to the above algorithm are set as: u is an image pixel matrix with an initial value of 0, v and f are both the image pixel matrix, and μ has an initial value of μ ₀ The initial value of λ is λ ₀

Solving the two secondary optimization problems would be extended to L ² Dynamic planning of norms aims at dividing a problem into a plurality of stages, each stage corresponds to a decision, a state array is used for recording the state of each stage, and a later state array is deduced according to the current state array. To a certain extentThese two secondary optimization problems can be considered as classical univariate Potts model, and the formula can be given:

according to the state theory of dynamic programming, the equation can be written as:

P _γ (h ^r )＝minP _γ (h ^r-1 )+γ+d[l,r]l＝1…r

where gamma is a constant and d [ l, r ] is the squared mean deviation.

The mean value can be written as:

wherein, the liquid crystal display device comprises a liquid crystal display device,

to this end, the theoretical derivation of the two secondary optimization problems is completed, and the previously entered values can be categorized as: 1. an image pixel matrix f;2. model constant γ;3. a weight matrix omega. During the operation, the following common values are also included:

left and right interval boundaries l, r, potts value P, two accumulation matrices M, S, accumulation matrix W, candidate Potts value P, square mean deviation d, and rightmost jumping array position J.

Wherein M, S, W, J will be set to 0 with r starting from 1 to n ending in the iterative sequence:

M _r ＝M _r-1 +ω _r f _r

S _r ＝S _r-1 +ω _r f _r ²

W _r ＝W _r-1 +ω _r

at this time, l starts from r until 2 ends, and the iteration sequence is as follows:

p＝P _l-1 +γ+d

if P _r <d+γ, the iteration is ended, and during the iteration, if p<P _r Then P is stored into P _r In J _r ＝l-1。

After completion of the above iteration, r=n, l=j _r Re-shaping the minimum value h from the optimal jump position, at r>In the case of 0, a valuation cycle is made, in which,

h _i ＝μ[l+1,r]；

wherein i ε [ l+1, r ];

after the operation is completed, the global minimum h and the weight coefficient array can be output and returned to the previous ADMM algorithm, and the end condition of the ADMM algorithm is thatAnd finally, u is output, wherein u is the waterline image after feature separation. Through multiple experiments, a relatively suitable empirical value is obtained, gamma is set to 2, TOL is set to 10 ^-10 。

The waterline image is typically separated into hull, sea water, rust, waterline, etc. features. Although image preprocessing has been performed previously, there are still pixel imbalance problems in hull and sea water characteristics for a variety of reasons, and thus further fusion is required. Before fusion, the pixels in the image need to be classified and counted, and two types of pixels with the largest proportion are screened out to be taken as hull and sea water characteristics. And taking the two pixels as reference, the other pixels need to calculate the difference between the two pixels, and the pixel with the small difference is replaced by the pixel. In this way, the waterline image will retain only two pixels. In view of the complexity of the waterline image, it is necessary to use a corrosion operation in computer morphology, in order to remove some small impurities in the waterline image, for example, pixels of the waterline feature may be replaced by pixels of the sea water feature in some cases, and this is required to be removed, where the corrosion operation is to use a m×n template to traverse each pixel, and pick the smallest pixel in the template to assign a value to the traversed pixel, so that the salient points on the periphery of the feature can be corroded. After the steps are finished, the characteristic separation of the waterline image is basically finished.

The specific method for detecting and marking the edges is as follows:

the waterline image features are separated, and edge labeling is needed to be carried out by adopting an edge detection algorithm. The edge detection and marking adopts a Canny edge detection algorithm, and the algorithm can divide the image based on the abrupt change of gray scale, so that the characteristics of discontinuous parts in the image can be extracted, and the waterline well accords with the target of the algorithm. The conventional Canny edge detection algorithm mainly comprises five parts, namely Gaussian filtering, pixel gradient calculation, non-maximum suppression, hysteresis thresholding and isolated weak edge suppression. Since the gaussian filtering has been used in the previous image preprocessing, it is not repeated here, and the second step is directly performed.

1. The pixel gradient was calculated using the Sobel operator, which is two 3*3 matrices, used to calculate pixel gradient matrices for two directions of the image, respectively, in the form shown below:

where I is the matrix of image pixels of the image after gray scale processing and the inner x is considered as the cross correlation operation.

Due to

Thus, a gradient intensity matrix G can be obtained _xy 。

2. To eliminate spurious effects from edge detection, non-maximum pixel gradient suppression has been developed. Since the gradient edge calculated previously is a plurality of pixels wide, it needs to be optimized to a precise dot width. It is therefore necessary to compare the gradient intensity of the current image pixel with the gradient intensity of the adjacent image pixels in the positive and negative gradient directions, and if the gradient intensity of the current image pixel is the largest compared with the gradient intensity of the other pixels in the same direction, the value is retained, otherwise, the suppression is performed, and the value is set to zero. In view of the exact calculation, linear interpolation can be used between two adjacent image pixels to obtain the corresponding image pixel gradients.

3. After shortening the edge width, a large number of small gradient modulus points remain, requiring a threshold hysteresis process. Among these, a high threshold and a low threshold need to be set to distinguish edge pixels. Three cases can occur after the edge pixels are screened: 1. the gradient value of the edge pixel point is larger than the high threshold value, and the edge pixel point is regarded as a strong edge point; 2. an edge pixel point gradient value between a low threshold value and a high threshold value is considered as a weak edge point; 3. the edge pixel gradient value is less than the low threshold and is set directly to zero deletion.

4. The main purpose of the isolated weak edge suppression is to screen out the edge points meeting the conditions, so that the boundary is more reliable, on the basis of the prior art, the strong edge points can be considered as true edges, and part of the weak edge points can be caused by noise change and need to be directly deleted. The weak edge points caused by the true edge are generally considered to be identical to the strong edge points, but the weak edge points caused by noise are not, so that 8-connected field pixels of each weak edge point can be checked by means of a depth-first algorithm, if a strong edge point exists beside the weak edge point, the weak edge point is reserved, and otherwise, the weak edge point is deleted directly.

After the steps are finished, a corresponding waterline point set can be obtained on the waterline image, and after the processing of the steps, pixels of the waterline image except the waterline point set are marked as black, and the waterline point set is marked as white. The coordinates of the waterline point set can be found by traversing the white image pixels and returned to the original image, and the point set pixels are changed to green, so that green wave lines/lines can be seen to appear between the ship hull and the sea waves.

Further optimizing the scheme, under the condition of completing the waterline marking, the method can go deeper into one step, namely, the method is converted into a linear equation. The two situations can be considered in the moment, the first situation is that the sea wave is calm, the edge point set can be approximately seen as a straight line, the Hough conversion can be adopted in the moment to convert the straight line into a straight line equation, and other calculations (such as water gauge reading by matching with water gauge scales) can be conveniently performed. And secondly, under the condition of larger sea waves, the edge point set is presented as an irregular wave curve, a centering method is adopted at the moment, the highest point and the lowest point in the point set are taken, an average value is made, a straight line is drawn as an actual waterline, and then the straight line is converted into a straight line equation.

If the Hough conversion is needed, a coordinate system is set first, the upper left corner of the image is taken as an image origin, the left-to-right x axis is the y axis, and the coordinate set of the waterline pixels can be obtained. On the basis, hough conversion is introduced for solving a linear formula of the waterline. The principle of hough conversion is that a linear formula y=kx+b is reversely utilized, k and b are regarded as points, the points are mapped into k-b space, the intersection point parameter of a straight line in k-b space is a target straight line, a straight line coordinate system can be converted into a polar coordinate system in consideration of the slope of the target straight line and possibly a plurality of straight lines with similar parameters, the polar coordinate system is set to be rho=xcos theta+ysin theta and projected to a corresponding rho-theta space, the parameter space is set to be discrete, a two-dimensional accumulation array A (rho, theta) is set, a corresponding value range is set, the point is initialized to be 0, the discrete value of each rho in the parameter space is brought into each point in the image coordinate space, and the two-dimensional accumulation array is correspondingly added by one. After the calculation is finished, the maximum peak value of the two-dimensional accumulation array is taken, and the corresponding parameter A (rho, theta) is the solved linear equation (polar coordinate form).

The algorithm specifically corresponding to generalized Hough transform comprises the following steps:

1. on the basis of the previously processed image, the diagonal length rmax is calculated;

2. the previously obtained image matrix is a binary image matrix (only black and white), the edge points are marked as white, θ is taken to be-90 to 90 for the marked edge points (x, y), calculation is performed according to ρ=xcos θ+ysinθ, a table with the size of 180 x rmax is obtained, each edge point is traversed, and weighting statistics are performed on +1 in the table obtained by the calculation in the above formula.

3. After counting, the distance and angle under the polar coordinate system of the first 3 maximum values in the table are obtained, and the corresponding xy coordinates are reversely deduced, so that the linear equation of the linear equation is deduced.

4. And screening the obtained linear equation, wherein the angle is smaller than-60 or larger than 60 and belongs to a normal waterline.

Thus, the waterline linear equation of the waterline image can be obtained.

The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. The port ship waterline detection method based on computer vision is characterized by comprising the following steps of:

inputting a photographed waterline image;

2. The method for detecting a waterline of a port vessel based on computer vision according to claim 1, wherein,

the image preprocessing process for the waterline image comprises the following steps: converting the water line image into an RGB format;

filtering the RGB format waterline image to remove image noise;

3. The method for detecting a waterline of a port vessel based on computer vision according to claim 2, wherein,

the method for removing the image noise comprises the following steps:

4. The method for detecting a waterline of a port vessel based on computer vision according to claim 2, wherein,

the brightness compensation method comprises the following steps: setting a brightness threshold value, and converting the filtered RGB format waterline image into an HSV format;

5. The port vessel waterline detection method based on computer vision according to claim 3, wherein,

the method for removing Gaussian noise comprises the following steps: a weighted average of pixel values in the gaussian seed image is created.

6. The method for detecting a waterline of a port vessel based on computer vision according to claim 1, wherein,

the process for obtaining the waterline characteristic map by performing the image characteristic separation operation comprises the following steps: based on the pixel matrix of the preprocessed image, an objective function is obtained through a Potts model, wherein the objective function is expressed as follows:

7. The method for detecting a waterline of a port vessel based on computer vision as claimed in claim 6, wherein,

the method for carrying out the unified processing of the characteristics on the waterline image subjected to the characteristic separation comprises the following steps: the waterline image features include hull, sea water, rust, waterline and spray;

8. The method for detecting a waterline of a port vessel based on computer vision according to claim 1, wherein,

the edge detection process comprises the following steps: calculating pixel gradients using a Sobel operator;

9. The method for detecting a waterline of a port vessel based on computer vision according to claim 1, wherein,

the waterline detection method further comprises the steps of constructing a linear equation according to the marked waterline, and the method for constructing the linear equation comprises the following steps: