CN114140412A - Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar - Google Patents
Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar Download PDFInfo
- Publication number
- CN114140412A CN114140412A CN202111406948.3A CN202111406948A CN114140412A CN 114140412 A CN114140412 A CN 114140412A CN 202111406948 A CN202111406948 A CN 202111406948A CN 114140412 A CN114140412 A CN 114140412A
- Authority
- CN
- China
- Prior art keywords
- point
- image
- water surface
- point cloud
- river bank
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 title claims abstract description 52
- 238000000034 method Methods 0.000 title claims abstract description 38
- 239000002245 particle Substances 0.000 claims abstract description 57
- 238000001914 filtration Methods 0.000 claims abstract description 37
- 238000009499 grossing Methods 0.000 claims abstract description 13
- 230000007797 corrosion Effects 0.000 claims abstract description 12
- 238000005260 corrosion Methods 0.000 claims abstract description 12
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 238000012163 sequencing technique Methods 0.000 claims abstract description 6
- 238000007670 refining Methods 0.000 claims abstract description 5
- 238000013527 convolutional neural network Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 9
- 238000013507 mapping Methods 0.000 claims description 9
- 230000003628 erosive effect Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 238000004364 calculation method Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 claims description 4
- 238000012952 Resampling Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000006073 displacement reaction Methods 0.000 claims description 3
- 238000012546 transfer Methods 0.000 claims description 3
- 239000011159 matrix material Substances 0.000 claims description 2
- 230000008859 change Effects 0.000 abstract description 7
- 238000013528 artificial neural network Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 5
- 230000004048 modification Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000003062 neural network model Methods 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 208000034699 Vitreous floaters Diseases 0.000 description 1
- 230000004075 alteration Effects 0.000 description 1
- 230000004888 barrier function Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000012459 cleaning agent Substances 0.000 description 1
- 238000013480 data collection Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 238000005286 illumination Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000001788 irregular Effects 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000000306 recurrent effect Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000002054 transplantation Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/70—Denoising; Smoothing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20021—Dividing image into blocks, subimages or windows
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Quality & Reliability (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Processing (AREA)
Abstract
The invention relates to the technical field of laser radar sensors and unmanned ship automatic driving, in particular to a method for extracting a passable area on a water surface by a high-robustness unmanned ship based on laser radar. The method comprises the steps of obtaining experimental environment data by using a laser radar sensor, performing semantic segmentation on point cloud data through a neural network, fusing continuous multiframe river bank point cloud data, refining river bank point cloud by adopting an image corrosion method through smooth filtering, extracting feature points, and sequencing to serve as control points. And then smoothing the control points through unscented kalman filtering in order to relieve the situation of the river bank abrupt change. And fitting the smoothed control points by using a B-spline curve, and finally filtering the noise of the extracted water surface area based on particle filtering. The robustness and the precision of extracting the passable area of the water surface are effectively improved.
Description
Technical Field
The invention relates to the technical field of laser radar sensors and unmanned ship automatic driving, in particular to a method for extracting a passable area on a water surface by a high-robustness unmanned ship based on laser radar.
Background
Chinese patent CN108303988A discloses a target identification tracking system of unmanned ship and a working method thereof, 3D laser radar and vector hydrophones are adopted to scan obstacles on water surface and underwater, an industrial personal computer analyzes the position of the obstacle on the water surface, a hydrophone processor determines the position of the obstacle under water, and according to the acquired position of the obstacle, the industrial personal computer adopts an obstacle avoidance algorithm to avoid the obstacle, namely, the unmanned ship avoids the obstacle on the water surface and underwater, and the purpose of operating in a water area with complex underwater conditions is achieved. The invention mainly aims at avoiding barriers of obstacles underwater on the water surface, and can generate random noise influences of river banks in various shapes, water surface ripples and the like in narrow river scenes, and the invention cannot solve the limitations. The method is characterized in that a water surface and underwater target is identified and tracked based on a laser radar, only fixed obstacles can be identified and analyzed, and small floating objects such as ripples and leaves existing on the water surface can be mistaken for the obstacles, so that the extraction precision is influenced; based on the regional area that the surface of water can pass is drawed to laser radar, use laser radar data collection on the ship can lead to appearing the more obvious condition of river bank change of gathering because the hull jolts, if unmanned ship is gone at the surface of water, receive influence such as wind, wave, can jolt for the radar can not scan the river bank at a certain moment great probability, causes the condition that the river bank point disappears, and this can produce serious influence to the algorithm of follow-up extraction passable region.
Disclosure of Invention
In order to overcome at least one defect in the prior art, the invention provides the method for extracting the water surface passable area by the high-robustness unmanned ship based on the laser radar, and the robustness and the precision for extracting the water surface passable area are effectively improved.
In order to solve the technical problems, the invention adopts the technical scheme that: a method for extracting a passable area on a water surface by a high-robustness unmanned ship based on a laser radar comprises the following steps:
s1, semantic segmentation is carried out on point cloud data collected by a laser radar by using a deep learning network of SqueezeSeg, and the point cloud data is divided into three categories of river banks, vegetation and bridges;
s2, performing smooth filtering operation on the river bank point extracted in the step S1, and then obtaining a control point in an image processing mode; firstly, fusing continuous multiframe river bank point cloud data, then performing smooth filtering, and then refining river bank points by an image corrosion method to obtain characteristic points of a river bank;
s3, sequencing the characteristic points of the river bank to serve as control points for fitting of a B spline;
s4, performing unscented Kalman filtering UKF smoothing on the sequenced control points;
s5, fitting the control points by adopting a B spline to draw a curve of a river bank shape;
and S6, obtaining a river surface area through the step 5, identifying the area occupied by the obstacles in the river surface area, and carrying out particle filtering.
The invention relates to a method for extracting a passable area on a water surface by a high-robustness unmanned ship based on a laser radar, which comprises the steps of acquiring experimental environment data by using a laser radar sensor, performing semantic segmentation on point cloud data through a neural network, fusing continuous multiframe river bank point cloud data, refining river bank point cloud by adopting an image corrosion method through smooth filtering, extracting characteristic points, and then sequencing to serve as control points. And then smoothing the control points through unscented kalman filtering in order to relieve the situation of the river bank abrupt change. And fitting the smoothed control points by using a B-spline curve, and finally filtering the noise of the extracted water surface area based on particle filtering.
Aiming at complex and variable water surface conditions, the invention provides a method for extracting a passable water surface area by a high-robustness unmanned ship based on a laser radar, which can be applied to the water surface extraction of wide sea levels and narrow rivers. And performing semantic segmentation on the point cloud data acquired by the laser radar based on a deep learning network, and dividing the point cloud data into three types, namely a river bank, vegetation, a bridge and the like. The unmanned ship runs on the water surface, is influenced by wind, waves and the like, can jolt, and can cause the problem that a river bank point disappears, and the problem can be solved by fusing river bank point clouds of continuous time frames. And then, by smooth filtering, the river bank point can keep stable change under the condition that the unmanned ship jolts, and then the river bank point is refined by an image corrosion method to obtain the characteristic point of the river bank. And sequencing the characteristic points of the river bank according to cosine for subsequent B-spline fitting to serve as control points of the B-spline. In order to prevent the river bank curve from generating a sharp change, the ordered control points are smoothed through unscented Kalman filtering UKF. And then, describing the river bank curve by adopting a B-spline value-added method for the control points. Through the operation, a river surface area can be obtained, and then the area occupied by the obstacle in the river surface area is identified. Because the extracted water surface area has random noises such as fixed obstacles, ripples, small floaters and the like, the water surface noise can be filtered by adopting particle filtering. The traditional particle filtering is commonly used for tracking multiple target objects, due to the characteristic that noise is discontinuous in time and space, point clouds can be tracked through the particle filtering, and if the point clouds cannot be continuously tracked, the point clouds are described as noise point clouds, so that the robustness and the accuracy of extracting a passable area of the water surface are improved.
Further, the step S1 specifically includes: firstly, the direct output of laser radar point cloud is required to be changed to be used as input, then semantic segmentation is carried out on an end-to-end pipeline of a conditional random field CRF based on a convolutional neural network and a reconstructed conditional random field, and point cloud data are divided into three categories of river banks, vegetation and bridges.
Further, the step S2 specifically includes: combining the point cloud of the t frame and the point clouds of the first two frames to obtain Q, mapping the Q to an Image coordinate system to obtain a Gray level Image Gray _ Image, then carrying out corrosion operation on the Gray _ Image, removing the misclassified river bank point cloud according to the geometric characteristics of a river bank, then smoothing by using mean value filtering, assigning the pixel value of which is less than the threshold value M to be 0, reducing the combination error, and finally mapping the pixel value of which is more than 0 in the Image to the point cloud coordinate system to obtain the characteristic point of the river bank point cloud.
Further, the image corrosion is to convolute the image by using an inner core, wherein one point in the inner core is defined as an anchor point, and then the pixel minimum value of the inner core coverage area is extracted to replace the pixel value of the anchor point; the corrosion has the functions of eliminating small objects in the image and smoothing the boundary of a large object; the erosion of the structural element B to the (x, y) position in the image A is defined as the minimum value of the region in the image A coinciding with B, with the origin of B placed at (x, y), and is expressed as follows:
and (4) accessing each pixel in the image A by the original point of the structural element B, and obtaining the corroded image C by the formula.
Further, in step S3, the feature points of the river bank extracted in step 2 are divided into four quadrants according to coordinates, and are sorted according to the size of the cosine.
Further, the step S4 specifically includes:
s41, forecasting, namely constructing a sigma point set through the optimal value x and the covariance p of the previous frame, and mapping a new sigma point set through a state transfer function; predicting a state estimation value x and covariance P;
s42, observing, namely constructing a sigma point set, mapping the sigma point set to a new sigma point set through an observation function, and predicting an observed estimated value z and covariance Pz;
s43, updating, calculating Kalman gain K according to the covariance matrix measured in the same state, and obtaining an updated state by using Kalman gain weighted average.
Further, in step S5: and (4) connecting the series of control points, the series of nodes and the series of coefficients obtained in the step (S4), wherein each coefficient corresponds to one control point, the calculation of the coefficients must ensure a certain continuous condition, all curve segments are connected together to meet the certain continuous condition, and finally, a plurality of Bezier curves are connected to obtain the B spline.
Further, the step S6 specifically includes the following steps:
s61, in an initialization stage, initializing a particle set;
s62, carrying out particle propagation, wherein the propagation position is obtained by calculating the displacement difference of the previous two frames to obtain the estimated position of the current frame and adding Gaussian white noise;
s63, calculating the weight w of the particle, assuming that the observation value of the current particle is y, and calculating the number of point clouds observed by the particle in the observation range of the current frame
And the number of observation point clouds R (x) of the object tracked by the particlet-1) Calculating the weight of the particle according to the following formula; if the noise is generated, the number of point clouds observed by the particles is small, and the calculated weight correspondence is small; the expression of the formula is:
s64, resampling, namely sorting the particles according to the weight of the particles, and copying the attribute of the particle with the large weight to the particle with the small weight;
s65, forecasting, namely carrying out weighted summation on the weight of the particles, solving a state estimation value and forecasting the current position of the tracking object; if the object is not detected in several continuous frames, the object point cloud tracked by the particle is noise;
and S66, updating the particles, and updating the state of the particles by using the current latest measured value.
The present invention also provides an electronic device comprising:
a memory for storing a computer program;
a processor for implementing the steps of the above method when executing the computer program.
The invention also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method described above.
Compared with the prior art, the beneficial effects are:
1. compared with other methods for realizing water surface extraction by using sensors such as cameras and the like, the method disclosed by the invention has less environmental requirements by using the laser radar sensor. The camera is not only sensitive to the illumination condition, but also has more strict requirements on the environment;
2. the method provided by the invention expands the range of the use scene. Compared with other water surface applications, most of the water surface environment-friendly water surface cleaning agent can only be applied to relatively ideal water surface environments. The experimental scene of the invention is an irregular narrow river, thus greatly expanding the application scene range;
3. the accuracy and robustness of the passable water surface area extracted by the method are high. There are some limitations to using lidar to collect data on water, such as ripples and leaves on water surface that reflect lidar. According to the invention, water surface noise points such as ripples and the like are filtered through particle filtering, so that the accuracy of extracting a passable area of the water surface is greatly improved;
4. the invention solves the problem that the change is relatively rapid in continuous frames when the laser radar acquires data due to the jolt of the unmanned ship. When the unmanned ship runs on the water surface, jolts of different degrees are generated, so that the radar installed on the unmanned ship cannot scan a river bank at a certain moment with high probability, or the scanned river bank is compared with the previous moment, the number of point clouds on the river bank is obviously different, passable areas extracted by using the river bank points can be caused, the area change is severe in a continuous time frame, and the stability and the accuracy of an algorithm are seriously influenced.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
FIG. 2 is a diagram of the present invention's SqueezeSeg-based neural network model.
FIG. 3 is a diagram of mean filtering according to the present invention.
FIG. 4 is a schematic view of the inventive image erosion.
FIG. 5 is a block diagram of the unscented Kalman filtering framework of the present invention.
FIG. 6 is a schematic diagram of a B-spline curve of the present invention.
Fig. 7 is pseudo code of the riparian point feature point extraction algorithm of the present invention.
Detailed Description
The drawings are for illustration purposes only and are not to be construed as limiting the invention; for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the invention.
The method comprises the steps of extracting the water surface passable area based on the laser radar, namely performing semantic segmentation, extracting the water surface area, and filtering noise of the water surface area. The overall frame diagram is shown in fig. 1.
Step 1: and (3) introducing a neural network based on the SqueezeSeg to segment the point cloud data acquired by the laser radar. The network structure of the SqueezeSeg is a typical encoder-skip connection-decoder structure, and is a lightweight FCN, and a neural network model is shown in FIG. 2. Firstly, the direct output of laser radar point cloud is required to be changed to be used as input, and then semantic segmentation is carried out on an end-to-end pipeline based on a convolutional neural network and a reconstructed conditional random field CRF. In order to be applied to three-dimensional laser radar point cloud, the CNN is designed to receive the transformed laser radar point cloud, classification is carried out, then a CRF model reconstructed into a Recurrent Neural Network (RNN) is further refined, the parameter size and the calculation complexity are reduced, and end-to-end training is carried out on the CNN model.
(1) Point cloud data
And (3) labeling point cloud data, and dividing the data into three types of objects: river banks, vegetation, and bridges. The training data consists of 450 frames of point cloud data, and the total number of the point clouds is 1350 ten thousand. In our dataset, the point clouds for the bank, vegetation and bridge account for 25.2%, 58.3% and 16.5%, respectively.
(2) Point cloud conversion
Due to the sparsity and irregularity of the point cloud, a general 2D CNN cannot be directly processed, and 3D point cloud data needs to be converted into a CNN-friendly data structure, and the point cloud is converted into a front view by using spherical projection.
(3) Neural network
The cited neural network SqueezeSeg is derived from SqueezeNet, which is a lightweight CNN, as shown in fig. 2, with a 50-fold reduction in parameters. The semantics of the point cloud are encoded by referencing the SqueezeNet transplantation layers conv1a to fire9 for feature extraction, followed by down-sampling the intermediate feature map with max-firing, the fire9 outputting the down-sampled feature map. The SqueezeSeg network uses a deconvolution module to upsample the feature map in the width dimension to obtain a full resolution label prediction for each point. fireModules and firedeconvvs replace the convolution and deconvolution layers to reduce the number of model parameters and computations.
(4) Conditional random field CRF layer
Accurate point-by-point tag prediction requires knowledge not only of the high level semantics of the objects and scenes, but also of the low level details. Low level details are lost in downsampling operations such as Max boosting, and the label graph for CNN prediction usually has fuzzy boundaries. In order to improve the precision of semantic segmentation, CRF is applied as the final RNN layer to refine the label graph. If two points in the cloud are adjacent to each other and have similar intensity measurements, they may belong to the same object and therefore have the same label, and the conditional random field CRF may be used to refine the label map generated by the CNN.
Step 2: in the part, continuous multiframe river bank point cloud data are fused, then smoothing operation is carried out through mean value filtering, and then river bank points are refined through an image corrosion method to obtain characteristic points of the river banks.
The mean filtering is the most commonly used means in image processing, and is a low-pass filter from the viewpoint of frequency domain, and high-frequency signals are removed, so that sharp noise of an image can be eliminated, and functions of smoothing and blurring the image can be realized. Ideally, the mean filtering is performed by replacing each pixel in the image with the mean value calculated for each pixel and its surrounding pixels. As shown in fig. 3, dark gray is the central pixel, the average of nine pixels around it and itself is calculated, the average is taken as the central blue pixel value, and if the pixel value of the point does not exceed the set threshold, the point is deleted.
And corroding the image and refining the point cloud of the river bank. The erosion of an image is described as a pixel value, and the principle of erosion is to perform a specific logical operation on a region corresponding to a binary image at each pixel position. The operation structure is a corresponding pixel of the output image. The operational effect depends on the structural element size content and the logical operational nature. That is, the image is convolved (scanned) with an kernel, one point in the kernel is defined as an anchor point, and then the minimum value (black direction) of the pixels in the kernel coverage area is extracted to replace the pixel value at the anchor point. The erosion has the functions of eliminating small objects in the image and smoothing larger object boundaries, and the erosion of a structural element (B) to an (x, y) position in an image a is defined as that the origin of the B is placed at the (x, y), and the minimum value of a region in the image a, which coincides with the B, is expressed as follows:
and (4) accessing each pixel in the image A by the original point of the structural element B, and obtaining the corroded image C by the formula. As shown in fig. 4, the structural element is a pixel block of 3 × 3, image a is image x, and the corrosion result of B on a is image v.
The whole process of the part is illustrated as follows: as shown in fig. 7, in the algorithm, the point cloud of the t frame and the point clouds of the two previous frames are merged to obtain Q (line 12), the Q is mapped into an Image coordinate system to obtain a Gray level Image Gray _ Image, then the Gray _ Image is corroded, the mistakenly classified river bank point cloud is removed according to the geometric features of the river bank, then, the average filtering is used for smoothing, the pixel with the pixel value smaller than the threshold value M is assigned to be 0, the merging error is reduced, and finally the pixel with the pixel value larger than 0 in the Image is mapped into the point cloud coordinate system to obtain the feature point of the river bank point cloud.
And step 3: dividing the points of the river bank characteristics extracted in the step 2 into four quadrants according to coordinates, and sequencing the four quadrants according to the sizes of the cosines respectively to be used as subsequent control points.
And 4, step 4: as shown in fig. 5, unscented kalman filtering UKF smoothing is performed on the ordered control points.
The UKF unscented Kalman filtering is developed on the basis of Kalman filtering and transformation, and the unscented Kalman filtering under linear assumption is applied to a nonlinear system by utilizing lossless transformation. Firstly, a conversion model is established, model parameters are fitted according to the model and training data, and then a Kalman prediction and updating stage is started. And predicting and sampling the optimal value x of the frame in the prediction stage, mapping the optimal value x to a new point set through a transfer function, predicting an estimated value and covariance according to the point set, processing the current observation point set by using the same method, calculating Kalman gain according to the observed values and the variances of the two, and updating the optimal value of the current frame according to the gain.
And 5: and fitting the control points taken out in the steps by adopting a B spline.
A B-spline curve is a special representation in mathematical sub-disciplinary numerical analysis, which is a linear combination of B-spline basis curves. And connecting a series of control points, a series of nodes and a series of coefficients obtained in the steps, wherein each coefficient corresponds to one control point, the calculation of the coefficients must ensure a certain continuous condition, all curve segments are connected together to meet the certain continuous condition, and finally, a plurality of Bezier curves are connected to obtain the B spline.
As shown in fig. 6, there are a total of 8 control points connected in sequence by line segments, and the B-spline curve is formed by connecting a series of 5 bezier curves 3 times. Generally, the lower the degree, the easier it is for the B-spline to approximate the control polyline.
Step 6: and 5, filtering the unmanned ship water surface area extracted in the step 5 by adopting particle filtering.
S61, in an initialization stage, initializing a particle set;
s62, carrying out particle propagation, wherein the propagation position is obtained by calculating the displacement difference of the previous two frames to obtain the estimated position of the current frame and adding Gaussian white noise;
s63, calculating the weight w of the particles, falseSetting the observation value of the current particle as y, and calculating the number R(s) of point clouds observed by the particle in the observation range of the current framet (i)) And the number of observation point clouds R (x) of the object tracked by the particlet-1) Calculating the weight of the particle according to the following formula; if the noise is generated, the number of point clouds observed by the particles is small, and the calculated weight correspondence is small; the expression of the formula is:
s64, resampling, namely sorting the particles according to the weight of the particles, and copying the attribute of the particle with the large weight to the particle with the small weight;
s65, forecasting, namely carrying out weighted summation on the weight of the particles, solving a state estimation value and forecasting the current position of the tracking object; if the object is not detected in several continuous frames, the object point cloud tracked by the particle is noise;
and S66, updating the particles, and updating the state of the particles by using the current latest measured value.
Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention, and that variations, modifications, substitutions and alterations can be made to the above embodiments by those of ordinary skill in the art within the scope of the present invention.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (10)
1. A method for extracting a passable area on a water surface by a high-robustness unmanned ship based on a laser radar is characterized by comprising the following steps:
s1, semantic segmentation is carried out on point cloud data collected by a laser radar by using a deep learning network of SqueezeSeg, and the point cloud data is divided into three categories of river banks, vegetation and bridges;
s2, performing smooth filtering operation on the river bank point extracted in the step S1, and then obtaining a control point in an image processing mode; firstly, fusing continuous multiframe river bank point cloud data, then performing smooth filtering, and then refining river bank points by an image corrosion method to obtain characteristic points of a river bank;
s3, sequencing the characteristic points of the river bank to serve as control points for fitting of a B spline;
s4, performing unscented Kalman filtering UKF smoothing on the sequenced control points;
s5, fitting the control points by adopting a B spline to draw a curve of a river bank shape;
and S6, obtaining a river surface area through the step 5, identifying the area occupied by the obstacles in the river surface area, and carrying out particle filtering.
2. The method for extracting the passable area on the water surface of the unmanned ship based on the lidar according to claim 1, wherein the step S1 specifically comprises: firstly, the direct output of laser radar point cloud is required to be changed to be used as input, then semantic segmentation is carried out on an end-to-end pipeline of a conditional random field CRF based on a convolutional neural network and a reconstructed conditional random field, and point cloud data are divided into three categories of river banks, vegetation and bridges.
3. The method for extracting the passable area on the water surface of the unmanned ship based on the lidar according to claim 2, wherein the step S2 specifically comprises: combining the point cloud of the t frame and the point clouds of the first two frames to obtain Q, mapping the Q to an Image coordinate system to obtain a Gray level Image Gray _ Image, then carrying out corrosion operation on the Gray _ Image, removing the misclassified river bank point cloud according to the geometric characteristics of a river bank, then smoothing by using mean value filtering, assigning the pixel value of which is less than the threshold value M to be 0, reducing the combination error, and finally mapping the pixel value of which is more than 0 in the Image to the point cloud coordinate system to obtain the characteristic point of the river bank point cloud.
4. The method for extracting the passable area on the water surface of the unmanned ship based on the laser radar as claimed in claim 3, wherein the image erosion is to convolute the image by using an inner core, a point in the inner core is defined as an anchor point, and then the minimum value of the pixel of the inner core coverage area is extracted to replace the pixel value of the anchor point; the corrosion has the functions of eliminating small objects in the image and smoothing the boundary of a large object; the erosion of the structural element B to the (x, y) position in the image A is defined as the minimum value of the region in the image A coinciding with B, with the origin of B placed at (x, y), and is expressed as follows:
and (4) accessing each pixel in the image A by the original point of the structural element B, and obtaining the corroded image C by the formula.
5. The method for extracting the passable area on the water surface by the unmanned ship based on the lidar according to claim 3, wherein in the step S3, the characteristic points of the river bank extracted in the step 2 are divided into four quadrants according to coordinates, and the quadrants are sorted according to cosine.
6. The method for extracting the passable area on the water surface of the unmanned ship based on the lidar according to claim 5, wherein the step S4 specifically comprises:
s41, forecasting, namely constructing a sigma point set through the optimal value x and the covariance p of the previous frame, and mapping a new sigma point set through a state transfer function; predicting a state estimation value x and covariance P;
s42, observing, namely constructing a sigma point set, mapping the sigma point set to a new sigma point set through an observation function, and predicting an observed estimated value z and covariance Pz;
s43, updating, calculating Kalman gain K according to the covariance matrix measured in the same state, and obtaining an updated state by using Kalman gain weighted average.
7. The method for extracting the passable area on the water surface of the unmanned ship based on the lidar according to claim 6, wherein the step S5 comprises: and (4) connecting the series of control points, the series of nodes and the series of coefficients obtained in the step (S4), wherein each coefficient corresponds to one control point, the calculation of the coefficients must ensure a certain continuous condition, all curve segments are connected together to meet the certain continuous condition, and finally, a plurality of Bezier curves are connected to obtain the B spline.
8. The method for extracting the passable area on the water surface of the unmanned ship based on the lidar according to claim 7, wherein the step S6 comprises the following steps:
s61, in an initialization stage, initializing a particle set;
s62, carrying out particle propagation, wherein the propagation position is obtained by calculating the displacement difference of the previous two frames to obtain the estimated position of the current frame and adding Gaussian white noise;
s63, calculating the weight w of the particle, assuming that the observation value of the current particle is y, and calculating the number of point clouds observed by the particle in the observation range of the current frame
And the number of observation point clouds R (x) of the object tracked by the particlet-1) Calculating the weight of the particle according to the following formula; if the noise is generated, the number of point clouds observed by the particles is small, and the calculated weight correspondence is small; the expression of the formula is:
s64, resampling, namely sorting the particles according to the weight of the particles, and copying the attribute of the particle with the large weight to the particle with the small weight;
s65, forecasting, namely carrying out weighted summation on the weight of the particles, solving a state estimation value and forecasting the current position of the tracking object; if the object is not detected in several continuous frames, the object point cloud tracked by the particle is noise;
and S66, updating the particles, and updating the state of the particles by using the current latest measured value.
9. An electronic device, comprising:
a memory for storing a computer program;
a processor for implementing the steps of the method according to any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, which computer program, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111406948.3A CN114140412A (en) | 2021-11-24 | 2021-11-24 | Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111406948.3A CN114140412A (en) | 2021-11-24 | 2021-11-24 | Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114140412A true CN114140412A (en) | 2022-03-04 |
Family
ID=80391571
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111406948.3A Pending CN114140412A (en) | 2021-11-24 | 2021-11-24 | Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114140412A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114879685A (en) * | 2022-05-25 | 2022-08-09 | 合肥工业大学 | River bank line detection and autonomous cruising method for unmanned ship |
| CN115097442A (en) * | 2022-08-24 | 2022-09-23 | 陕西欧卡电子智能科技有限公司 | Water surface environment map construction method based on millimeter wave radar |
-
2021
- 2021-11-24 CN CN202111406948.3A patent/CN114140412A/en active Pending
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114879685A (en) * | 2022-05-25 | 2022-08-09 | 合肥工业大学 | River bank line detection and autonomous cruising method for unmanned ship |
| CN115097442A (en) * | 2022-08-24 | 2022-09-23 | 陕西欧卡电子智能科技有限公司 | Water surface environment map construction method based on millimeter wave radar |
| CN115097442B (en) * | 2022-08-24 | 2022-11-22 | 陕西欧卡电子智能科技有限公司 | Water surface environment map construction method based on millimeter wave radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111201451B (en) | Method and device for detecting object in scene based on laser data and radar data of scene | |
| Prasad et al. | Video processing from electro-optical sensors for object detection and tracking in a maritime environment: A survey | |
| CN110232350B (en) | Real-time water surface multi-moving-object detection and tracking method based on online learning | |
| Chen et al. | Ship detection from coastal surveillance videos via an ensemble Canny-Gaussian-morphology framework | |
| CN114140412A (en) | Method for extracting water surface passable area by high-robustness unmanned ship based on laser radar | |
| CN116503803A (en) | Obstacle detection method, obstacle detection device, electronic device and storage medium | |
| Venugopal | Sample selection based change detection with dilated network learning in remote sensing images | |
| CN115861359A (en) | Self-adaptive segmentation and extraction method for water surface floating garbage image | |
| Petković et al. | An overview on horizon detection methods in maritime video surveillance | |
| CN109993744B (en) | Infrared target detection method under offshore backlight environment | |
| Patil et al. | Road extraction techniques from remote sensing images: a review | |
| Cheng et al. | Sonar image garbage detection via global despeckling and dynamic attention graph optimization | |
| CN117079117B (en) | Underwater image processing and target identification method and device, storage medium and electronic equipment | |
| Günsel et al. | Reconstruction and boundary detection of range and intensity images using multiscale MRF representations | |
| Alsharay et al. | Improved sea-ice identification using semantic segmentation with raindrop removal | |
| Patil et al. | Road segmentation in high-resolution images using deep residual networks | |
| CN109872358B (en) | Marine oil film identification method for shipborne radar image based on active contour model | |
| CN116664851A (en) | Automatic driving data extraction method based on artificial intelligence | |
| Huiying et al. | The water coastline detection approaches based on USV vision | |
| CN116310832A (en) | Remote sensing image processing method, device, equipment, medium and product | |
| CN114155425B (en) | Weak and small target detection method based on Gaussian Markov random field motion direction estimation | |
| Wang et al. | Sonar image detection based on multi-scale multi-column convolution neural networks | |
| CN114879180A (en) | Seamless situation perception method for real-time fusion of unmanned ship-borne multi-element multi-scale radar | |
| Jiang et al. | Ship contour extraction from SAR images based on faster R-CNN and chan–vese model | |
| Yang et al. | A review of intelligent ship marine object detection based on RGB camera |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |