CN116958265A - Ship pose measurement method and system based on binocular vision - Google Patents
Ship pose measurement method and system based on binocular vision Download PDFInfo
- Publication number
- CN116958265A CN116958265A CN202311204406.7A CN202311204406A CN116958265A CN 116958265 A CN116958265 A CN 116958265A CN 202311204406 A CN202311204406 A CN 202311204406A CN 116958265 A CN116958265 A CN 116958265A
- Authority
- CN
- China
- Prior art keywords
- ship
- point cloud
- binocular camera
- detected
- acquiring
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000691 measurement method Methods 0.000 title claims abstract description 19
- 238000005259 measurement Methods 0.000 claims abstract description 24
- 238000012549 training Methods 0.000 claims abstract description 10
- 238000013528 artificial neural network Methods 0.000 claims abstract description 7
- 238000000034 method Methods 0.000 claims description 20
- 239000011159 matrix material Substances 0.000 claims description 15
- 238000013519 translation Methods 0.000 claims description 9
- 238000006073 displacement reaction Methods 0.000 claims description 4
- 230000009466 transformation Effects 0.000 claims description 4
- 238000007781 pre-processing Methods 0.000 claims description 3
- 238000006243 chemical reaction Methods 0.000 claims 1
- 238000001514 detection method Methods 0.000 abstract description 10
- 230000001133 acceleration Effects 0.000 description 5
- 239000003086 colorant Substances 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000009434 installation Methods 0.000 description 2
- 238000003754 machining Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/044—Recurrent networks, e.g. Hopfield networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
- G06T7/337—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
- G06V10/765—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects using rules for classification or partitioning the feature space
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20228—Disparity calculation for image-based rendering
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Databases & Information Systems (AREA)
- Medical Informatics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Data Mining & Analysis (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- General Engineering & Computer Science (AREA)
- Mathematical Physics (AREA)
- Biophysics (AREA)
- Biomedical Technology (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The application discloses a binocular vision-based ship pose measurement method and a binocular vision-based ship pose measurement system, which comprise the following steps: calibrating internal and external parameters of a binocular camera, acquiring ship images in real time through the calibrated binocular camera, and training a mask RCNN neural network by utilizing the ship images to acquire a trained ship identification model and a ship mask map; acquiring a parallax map based on the internal and external parameters of the binocular camera and the ship image, acquiring a ship parallax map to be detected based on the parallax map in combination with the ship mask map, and acquiring a current-frame ship point cloud map to be detected based on the ship parallax map to be detected and the internal and external parameters of the binocular camera; and carrying out point cloud registration according to the point cloud images of the ship to be detected of the front frame and the rear frame, and obtaining the pose of the ship to be detected. The application adopts non-contact pose measurement, so that the influence of the detection freedom degree detection instrument on the ship under the action of wind wave current is minimized.
Description
Technical Field
The application belongs to the technical field of ship pose measurement, and particularly relates to a ship pose measurement method and system based on binocular vision.
Background
Modern ships are a large system of many subsystems, the hull of one of the ship's subsystems being both a carrier supporting the entire weight of the ship and an important factor affecting the main properties of the ship (such as stability, quickness, resistance to sinking, resistance to waves, etc.). In the wave water tank, the ship generates the movement amount of the pose under the influence of the wind and wave currents, and the measurement of the pose of the ship can provide good technical support for the design of the ship body. The current measuring method for the pose and the motion of the ship is generally divided into the following steps: acceleration sensor measurement, mechanical device measurement, visual measurement. The acceleration sensor is mainly arranged on a ship, three corners and three motion accelerations of the ship are measured by the aid of the sensor, and then the speed and the displacement are calculated through integration. Because the mode of the sensor is adopted, besides the accumulated errors during measurement of the acceleration sensor, the installation of the acceleration sensor on the ship can also influence the mass distribution of the ship, and the subsequent calculation is influenced. The mechanical device measurement is a method for measuring by driving a set of complex and precise machining mechanical system through the ship to be measured, and has the defects of complex machining and manufacturing of equipment and unavoidable influence on the movement of the ship. The visual measurement is to acquire a ship picture through a camera, and to acquire the pose of the ship through computer vision. In the prior art, waterproof color blocks with three colors of red, green and blue are adhered to different positions on a ship, and then three-dimensional coordinates of the color blocks with the three colors can be obtained according to the imaging principle of a camera. From the three-dimensional coordinates of measuring a large number of color blocks, the motion trail of the ship can be reconstructed, or the contact measurement is adopted, the paper color blocks on the ship can influence the motion of the ship, so that the measurement is inaccurate, and therefore, the ship pose measurement method and system based on binocular vision are needed to be provided.
Disclosure of Invention
In order to solve the technical problems, the application provides a binocular vision-based ship pose measurement method and a binocular vision-based ship pose measurement system, which adopt binocular stereo cameras, do not need to arrange complicated acquisition instruments and processors, and are simple and convenient in data acquisition, low in maintenance cost and small in occupied area. The training time of the neural network may be short for the identification of a single target. And non-contact pose measurement is adopted, so that the influence of the detection freedom degree detection instrument on the ship under the action of wind wave current is minimized.
In order to achieve the above purpose, the application provides a binocular vision-based ship pose measurement method, which comprises the following steps:
calibrating internal and external parameters of a binocular camera, acquiring ship images in real time through the calibrated binocular camera, and training a mask RCNN neural network by utilizing the ship images to acquire a trained ship identification model and a ship mask map;
acquiring a parallax map based on the internal and external parameters of the binocular camera and the ship image, acquiring a ship parallax map to be detected based on the parallax map in combination with the ship mask map, and acquiring a current-frame ship point cloud map to be detected based on the ship parallax map to be detected and the internal and external parameters of the binocular camera;
and carrying out point cloud registration according to the point cloud images of the ship to be detected of the front frame and the rear frame, and obtaining the pose of the ship to be detected.
Optionally, the method further comprises installing the binocular camera and adjusting the baseline and the focal length of the binocular camera before calibrating the internal and external parameters of the binocular camera.
Optionally, installing the binocular camera and adjusting a baseline and a focal length of the binocular camera specifically includes:
and adjusting the base line and the focal length of the binocular camera according to the length and width data of the ship to be tested, wherein the distance range between the ship to be tested and the binocular camera is 10-30 times of the binocular base line, and the binocular camera is arranged on the front side surface of the ship to be tested and irradiates the whole movable range of the ship to be tested.
Optionally, the method for acquiring the parallax map based on the internal and external parameters of the binocular camera and the ship image comprises the following steps: and acquiring the parallax map through an SGBM algorithm based on the internal and external parameters of the binocular camera and the ship image.
Optionally, the method for performing point cloud registration according to the point cloud images of the ship to be detected, which are acquired by the front frame and the rear frame, comprises the following steps:
the point cloud coordinate transformation matrix M is obtained by ICP registration and is:
wherein the rotation vector R is:
the displacement vector is:
the rotation matrix of the Euler angle can be obtained according to the rotation sequence of X-Y-Z:
and (3) solving to obtain:
wherein R corresponds to a 3*3 digital matrix, M corresponds to a 4*4 digital matrix, and T corresponds to a 1*3 digital matrix; alpha corresponds to the rotation angle around the X axis, beta corresponds to the rotation angle of the Y axis, gamma corresponds to the rotation angle of the Z axis, T 1 Corresponding to the translation in the X-axis direction, T 2 Corresponding to the translation in the Y-axis direction, T 3 Corresponding to translation in the Z-axis direction.
Optionally, the method further comprises obtaining an initial point cloud before the point cloud registration is performed according to the point cloud images of the ship to be detected obtained in the front and the rear frames, when the ship to be detected is driven to the right front of the binocular camera, coordinate system transformation is performed based on the point cloud images of the ship to be detected in the current frame, and the follow-up coordinate system of the ship to be detected is transferred to a coordinate system taking the left-eye camera as an origin according to the point cloud images of the ship to be detected in the current frame and the combined ship mass center of the length and the width of the ship to be detected, so that the initial point cloud is obtained.
Optionally, the method for obtaining the ship mask map includes:
preprocessing the ship image, inputting the preprocessed ship image into the trained ship identification model, and obtaining a corresponding feature map;
setting a plurality of interested areas for each point in the feature map, and obtaining a plurality of candidate interested areas;
and inputting a plurality of candidate regions of interest into the trained ship identification model to perform binarization classification and frame regression, deleting the invalid regions of interest, and obtaining the ship mask map.
Optionally, the method for acquiring the current frame of the ship point cloud image to be detected based on the ship parallax image to be detected and the internal and external parameters of the binocular camera comprises the following steps:
and acquiring a current frame of ship point cloud image to be detected through an SGBM algorithm based on the binocular camera, the trained ship identification model and the internal and external parameters of the binocular camera.
In order to achieve the above object, the present application further provides a binocular vision-based ship pose measurement system, comprising: the ship point cloud system comprises a camera calibration module, a ship identification module, a ship point cloud module and a ship point cloud registration module, wherein the camera calibration module, the ship identification module, the ship point cloud module and the ship point cloud registration module are connected;
the camera calibration module is used for calibrating the binocular camera to acquire the internal and external parameters of the binocular camera;
the ship identification module is used for training and identifying the ship to be tested by adopting the calibrated binocular camera, and respectively acquiring a ship identification model and a ship mask map of the ship to be tested;
the ship point cloud module is used for acquiring a current frame of ship point cloud image to be tested through the internal and external parameters of the binocular camera, the ship identification model of the ship to be tested and the ship mask image;
the ship point cloud registration module is used for carrying out point cloud registration by acquiring front and rear ship point cloud images to be detected and acquiring the pose of the ship to be detected.
Optionally, the system further comprises a device mounting module for mounting the binocular camera and adjusting the shooting position of the binocular camera.
The application has the technical effects that: the application discloses a binocular vision-based ship pose measurement method and a binocular vision-based ship pose measurement system. The training time of the neural network may be short for the identification of a single target. And non-contact pose measurement is adopted, so that the influence of the detection freedom degree detection instrument on the ship under the action of wind wave current is minimized.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the application. In the drawings:
fig. 1 is a schematic flow chart of a ship pose measurement method based on binocular vision according to an embodiment of the application;
fig. 2 is a schematic structural diagram of a ship pose measurement system based on binocular vision according to an embodiment of the present application.
Detailed Description
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer executable instructions, and that although a logical order is illustrated in the flowcharts, in some cases the steps illustrated or described may be performed in an order other than that illustrated herein.
As shown in fig. 1, the present embodiment provides a binocular vision-based ship pose measurement method, which includes the following steps: calibrating internal and external parameters of the binocular camera, acquiring ship images in real time through the calibrated binocular camera, and training a mask NN by utilizing the ship images to acquire a trained ship identification model and a ship mask map; acquiring a parallax map based on the internal and external parameters of the binocular camera and the ship image, acquiring a ship parallax map to be detected based on the parallax map combined with the ship mask map, and acquiring a current frame of ship point cloud map to be detected based on the ship parallax map to be detected and the internal and external parameters of the binocular camera; and carrying out point cloud registration according to the point cloud images of the ship to be detected of the front frame and the rear frame, and obtaining the pose of the ship to be detected.
And calibrating internal and external parameters of the binocular camera, marking the ship picture obtained by the binocular camera, and training the mask RCNN neural network to obtain a ship identification model. And acquiring a ship image in real time through the calibrated binocular camera, and acquiring a ship mask based on the ship image and the ship identification model. The rest is to acquire a parallax map based on the internal and external parameters of the binocular camera and the ship image, acquire a ship parallax map to be detected based on the parallax map and the ship mask map, and acquire a current frame of ship point cloud map to be detected based on the ship parallax map to be detected and the internal and external parameters of the binocular camera; and carrying out point cloud registration according to the point cloud images of the ship to be detected of the front frame and the rear frame, and obtaining the pose of the ship to be detected. The application adopts non-contact pose measurement, so that the influence of the detection freedom degree detection instrument on the ship under the action of wind wave current is minimized.
The method comprises the steps of calibrating internal and external parameters of the binocular camera, installing the binocular camera, and adjusting the base line and the focal length of the binocular camera.
Before the binocular camera is adopted to acquire the ship image to be measured, the binocular camera is required to be installed, and the baseline and the focal length of the binocular camera are adjusted, and the method specifically comprises the following steps: and adjusting the base line and the focal length of the binocular camera according to the length and width data of the ship to be tested, wherein the distance range between the ship to be tested and the binocular camera is 10-30 times of the binocular base line, and the binocular camera is arranged on the front side surface of the ship to be tested and irradiates the whole moving range of the ship to be tested.
The method for acquiring the parallax map based on the internal and external parameters of the binocular camera and the ship image comprises the following steps: based on the internal and external parameters of the binocular camera and the ship image, a parallax image is obtained through an SGBM algorithm. After obtaining the internal and external parameters of the camera by adopting a Zhang calibration method, inputting the internal and external parameters of the camera into an SGBM algorithm for parameter adjustment, and adjusting the parameters of the SGBM algorithm to ensure that a parallax map obtained by stereo matching has a good effect; and inputting the images into the binocular image of the monitoring ship again to obtain a parallax image.
The method for carrying out point cloud registration according to the point cloud images of the ship to be detected, which are acquired by the front frame and the rear frame, comprises the following steps:
the point cloud coordinate transformation matrix M is obtained by ICP registration and is:
wherein the rotation vector R is:
the displacement vector is:
the rotation matrix of the Euler angle can be obtained according to the rotation sequence of X-Y-Z:
and (3) solving to obtain:
wherein R corresponds to a 3*3 digital matrix, M corresponds to a 4*4 digital matrix, and T corresponds to a 1*3 digital matrix; alpha corresponds to the rotation angle around the X axis, beta corresponds to the rotation angle of the Y axis, gamma corresponds to the rotation angle of the Z axis, T 1 Corresponding to the translation in the X-axis direction, T 2 Corresponding to the translation in the Y-axis direction, T 3 Corresponding to translation in the Z-axis direction.
The method comprises the steps of obtaining an initial point cloud according to the point cloud images of the ship to be detected, which are obtained according to the front frame and the rear frame, before the point cloud registration, when the ship to be detected is driven to the right front of the binocular camera, converting a coordinate system based on the point cloud images of the ship to be detected of the current frame, and transferring a follow-up coordinate system of the ship to be detected to a coordinate system taking the left-eye camera as an origin according to the point cloud images of the ship to be detected of the current frame and the length and the width of the ship to be detected, so as to obtain the initial point cloud.
The method for acquiring the ship mask map comprises the following steps: preprocessing a ship image, inputting the preprocessed ship image into a trained ship identification model, and obtaining a corresponding feature map; setting a plurality of interested areas for each point in the feature map, and obtaining a plurality of candidate interested areas; and inputting the multiple candidate regions of interest into a trained ship identification model to perform binarization classification and frame regression, deleting invalid regions of interest, and obtaining a ship mask map.
The method for acquiring the point cloud image of the ship to be tested in the current frame based on the parallax image of the ship to be tested and the internal and external parameters of the binocular camera comprises the following steps: based on the binocular camera, the trained ship recognition model and the internal and external parameters of the binocular camera, acquiring a point cloud picture of a coordinate system taking the left-eye camera optical center of the binocular camera as an origin, a far camera as a z axis and a right-hand coordinate as a label through an SGBM algorithm.
As shown in fig. 2, in this embodiment, a ship pose measurement system based on binocular vision is provided, including: the device comprises a device mounting module, a camera calibration module, a ship identification module, a ship point cloud module and a ship point cloud registration module, wherein the device mounting module, the camera calibration module, the ship identification module, the ship point cloud module and the ship point cloud registration module are connected; the equipment installation module is used for installing the binocular camera and adjusting the shooting position of the binocular camera; the camera calibration module is used for calibrating the binocular camera to obtain the internal and external parameters of the binocular camera; the ship identification module is used for identifying the ship to be detected by adopting the calibrated binocular camera, and acquiring a ship identification model and a ship mask map of the ship to be detected; the ship point cloud module is used for acquiring a point cloud image of the ship to be tested in the current frame through the internal and external parameters of the binocular camera, the ship identification model of the ship to be tested and the ship mask image; and the ship point cloud registration module is used for carrying out point cloud registration by acquiring the point cloud images of the ship to be detected of the front frame and the rear frame, and acquiring the pose of the ship to be detected.
The application discloses a binocular vision-based ship pose measurement method and a binocular vision-based ship pose measurement system. The training time of the neural network may be short for the identification of a single target. And non-contact pose measurement is adopted, so that the influence of the detection freedom degree detection instrument on the ship under the action of wind wave current is minimized.
The present application is not limited to the above-mentioned embodiments, and any changes or substitutions that can be easily understood by those skilled in the art within the technical scope of the present application are intended to be included in the scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.
Claims (10)
1. The ship pose measurement method based on binocular vision is characterized by comprising the following steps of:
calibrating internal and external parameters of a binocular camera, acquiring ship images in real time through the calibrated binocular camera, and training a mask RCNN neural network by utilizing the ship images to acquire a trained ship identification model and a ship mask map;
acquiring a parallax map based on the internal and external parameters of the binocular camera and the ship image, acquiring a ship parallax map to be detected based on the parallax map in combination with the ship mask map, and acquiring a current-frame ship point cloud map to be detected based on the ship parallax map to be detected and the internal and external parameters of the binocular camera;
and carrying out point cloud registration according to the point cloud images of the ship to be detected of the front frame and the rear frame, and obtaining the pose of the ship to be detected.
2. The binocular vision based marine vessel pose measurement method of claim 1, further comprising installing the binocular camera and adjusting a baseline and a focal length of the binocular camera before calibrating the internal and external parameters of the binocular camera.
3. The binocular vision based ship pose measurement method of claim 2, wherein,
installing the binocular camera and adjusting the baseline and focal length of the binocular camera, specifically comprising:
and adjusting the base line and the focal length of the binocular camera according to the length and width data of the ship to be tested, wherein the distance range between the ship to be tested and the binocular camera is 10-30 times of the binocular base line, and the binocular camera is arranged on the front side surface of the ship to be tested and irradiates the whole movable range of the ship to be tested.
4. The binocular vision based ship pose measurement method of claim 1, wherein,
the method for acquiring the parallax map based on the internal and external parameters of the binocular camera and the ship image comprises the following steps: and acquiring the parallax map through an SGBM algorithm based on the internal and external parameters of the binocular camera and the ship image.
5. The binocular vision based ship pose measurement method of claim 1, wherein,
the method for carrying out point cloud registration according to the point cloud images of the ship to be detected, which are acquired by the front frame and the rear frame, comprises the following steps:
the point cloud coordinate transformation matrix M is obtained by ICP registration and is:
wherein the rotation vector R is:
the displacement vector is: />The rotation matrix of the Euler angle can be obtained according to the rotation sequence of X-Y-Z:
and (3) solving to obtain:
wherein R corresponds to a 3*3 digital matrix, M corresponds to a 4*4 digital matrix, and T corresponds to a 1*3 digital matrix; alpha corresponds to the rotation angle around the X axis, beta corresponds to the rotation angle of the Y axis, gamma corresponds to the rotation angle of the Z axis, T 1 Corresponding to the translation in the X-axis direction, T 2 Corresponding to the translation in the Y-axis direction, T 3 Corresponding to translation in the Z-axis direction.
6. The binocular vision based ship pose measurement method of claim 1, wherein,
and acquiring an initial point cloud before carrying out point cloud registration according to the point cloud images of the ship to be detected acquired in the front and the rear frames, when the ship to be detected is driven to the right front of the binocular camera, carrying out coordinate system conversion based on the point cloud images of the ship to be detected in the current frame, and transferring a follow-up coordinate system of the ship to be detected to a coordinate system taking the left-eye camera as an origin according to the point cloud images of the ship to be detected in the current frame and the length and width combined ship centroid of the ship to be detected, so as to acquire the initial point cloud.
7. The binocular vision based ship pose measurement method of claim 1, wherein,
the method for acquiring the ship mask map comprises the following steps:
preprocessing the ship image, inputting the preprocessed ship image into the trained ship identification model, and obtaining a corresponding feature map;
setting a plurality of interested areas for each point in the feature map, and obtaining a plurality of candidate interested areas;
and inputting a plurality of candidate regions of interest into the trained ship identification model to perform binarization classification and frame regression, deleting the invalid regions of interest, and obtaining the ship mask map.
8. The binocular vision based ship pose measurement method of claim 1, wherein,
the method for acquiring the current frame of ship point cloud image to be detected based on the ship parallax image to be detected and the internal and external parameters of the binocular camera comprises the following steps:
and acquiring a current frame of ship point cloud image to be detected through an SGBM algorithm based on the binocular camera, the trained ship identification model and the internal and external parameters of the binocular camera.
9. The utility model provides a ship position appearance measurement system based on binocular vision which characterized in that includes: the ship point cloud system comprises a camera calibration module, a ship identification module, a ship point cloud module and a ship point cloud registration module, wherein the camera calibration module, the ship identification module, the ship point cloud module and the ship point cloud registration module are connected;
the camera calibration module is used for calibrating the binocular camera to acquire the internal and external parameters of the binocular camera;
the ship identification module is used for training and identifying the ship to be tested by adopting the calibrated binocular camera, and respectively acquiring a ship identification model and a ship mask map of the ship to be tested;
the ship point cloud module is used for acquiring a current frame of ship point cloud image to be tested through the internal and external parameters of the binocular camera, the ship identification model of the ship to be tested and the ship mask image;
the ship point cloud registration module is used for carrying out point cloud registration by acquiring front and rear ship point cloud images to be detected and acquiring the pose of the ship to be detected.
10. The binocular vision based marine vessel pose measurement system of claim 9, further comprising a device mounting module for mounting the binocular camera and adjusting a photographing position of the binocular camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311204406.7A CN116958265A (en) | 2023-09-19 | 2023-09-19 | Ship pose measurement method and system based on binocular vision |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311204406.7A CN116958265A (en) | 2023-09-19 | 2023-09-19 | Ship pose measurement method and system based on binocular vision |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116958265A true CN116958265A (en) | 2023-10-27 |
Family
ID=88442778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311204406.7A Pending CN116958265A (en) | 2023-09-19 | 2023-09-19 | Ship pose measurement method and system based on binocular vision |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116958265A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117173650A (en) * | 2023-11-02 | 2023-12-05 | 浙江华是科技股份有限公司 | Ship measurement and identification method and system based on laser radar |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109101957A (en) * | 2018-10-29 | 2018-12-28 | 长沙智能驾驶研究院有限公司 | Binocular solid data processing method, device, intelligent driving equipment and storage medium |
| CN109522837A (en) * | 2018-11-14 | 2019-03-26 | 海信集团有限公司 | A kind of pavement detection method and device |
| CN110853133A (en) * | 2019-10-25 | 2020-02-28 | 深圳奥比中光科技有限公司 | Method, device, system and readable storage medium for reconstructing three-dimensional model of human body |
| CN110920824A (en) * | 2019-11-21 | 2020-03-27 | 深圳市若雅方舟科技有限公司 | Ship model attitude measurement system and method based on binocular stereo vision |
| CN111462236A (en) * | 2020-04-02 | 2020-07-28 | 集美大学 | Method and system for detecting relative pose between ships |
| CN111915677A (en) * | 2020-07-08 | 2020-11-10 | 哈尔滨工程大学 | Ship pose estimation method based on three-dimensional point cloud characteristics |
| CN112381870A (en) * | 2020-11-25 | 2021-02-19 | 河海大学常州校区 | Ship identification and navigational speed measurement system and method based on binocular vision |
| CN112529946A (en) * | 2020-12-04 | 2021-03-19 | 中南大学 | High discrete body model optimization method and system based on elevation data, electronic equipment and readable storage medium |
| CN113077476A (en) * | 2021-03-17 | 2021-07-06 | 浙江大华技术股份有限公司 | Height measurement method, terminal device and computer storage medium |
| WO2021196941A1 (en) * | 2020-04-03 | 2021-10-07 | 浙江商汤科技开发有限公司 | Method and apparatus for detecting three-dimensional target |
| CN113840127A (en) * | 2021-08-12 | 2021-12-24 | 长光卫星技术有限公司 | Method for acquiring water area automatic mask processing DSM by satellite video image |
| CN114140527A (en) * | 2021-11-19 | 2022-03-04 | 苏州科技大学 | Dynamic environment binocular vision SLAM method based on semantic segmentation |
| CN114245097A (en) * | 2021-11-05 | 2022-03-25 | 丁俊飞 | Three-dimensional imaging method and device based on three-color mask single-color camera |
| CN114627175A (en) * | 2020-12-11 | 2022-06-14 | 武汉Tcl集团工业研究院有限公司 | Image processing method, intelligent terminal and computer readable storage medium |
| CN114758093A (en) * | 2022-03-31 | 2022-07-15 | 深圳万兴软件有限公司 | Three-dimensional model generation method, device, equipment and medium based on image sequence |
| CN115451964A (en) * | 2022-08-18 | 2022-12-09 | 武汉理工大学 | Ship scene simultaneous mapping and positioning method based on multi-mode mixed features |
| CN115619790A (en) * | 2022-12-20 | 2023-01-17 | 北京维卓致远医疗科技发展有限责任公司 | Mixed perspective method, system and equipment based on binocular positioning |
| CN116007623A (en) * | 2022-12-02 | 2023-04-25 | 纯米科技(上海)股份有限公司 | Robot navigation method, apparatus and computer readable storage medium |
| WO2023083273A1 (en) * | 2021-11-10 | 2023-05-19 | 梅卡曼德（北京）机器人科技有限公司 | Grip point information acquisition method and apparatus, electronic device, and storage medium |
| CN116597086A (en) * | 2023-05-19 | 2023-08-15 | 京东科技控股股份有限公司 | Image processing method and device |
-
2023
- 2023-09-19 CN CN202311204406.7A patent/CN116958265A/en active Pending
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109101957A (en) * | 2018-10-29 | 2018-12-28 | 长沙智能驾驶研究院有限公司 | Binocular solid data processing method, device, intelligent driving equipment and storage medium |
| CN109522837A (en) * | 2018-11-14 | 2019-03-26 | 海信集团有限公司 | A kind of pavement detection method and device |
| CN110853133A (en) * | 2019-10-25 | 2020-02-28 | 深圳奥比中光科技有限公司 | Method, device, system and readable storage medium for reconstructing three-dimensional model of human body |
| CN110920824A (en) * | 2019-11-21 | 2020-03-27 | 深圳市若雅方舟科技有限公司 | Ship model attitude measurement system and method based on binocular stereo vision |
| CN111462236A (en) * | 2020-04-02 | 2020-07-28 | 集美大学 | Method and system for detecting relative pose between ships |
| WO2021196941A1 (en) * | 2020-04-03 | 2021-10-07 | 浙江商汤科技开发有限公司 | Method and apparatus for detecting three-dimensional target |
| CN111915677A (en) * | 2020-07-08 | 2020-11-10 | 哈尔滨工程大学 | Ship pose estimation method based on three-dimensional point cloud characteristics |
| CN112381870A (en) * | 2020-11-25 | 2021-02-19 | 河海大学常州校区 | Ship identification and navigational speed measurement system and method based on binocular vision |
| CN112529946A (en) * | 2020-12-04 | 2021-03-19 | 中南大学 | High discrete body model optimization method and system based on elevation data, electronic equipment and readable storage medium |
| CN114627175A (en) * | 2020-12-11 | 2022-06-14 | 武汉Tcl集团工业研究院有限公司 | Image processing method, intelligent terminal and computer readable storage medium |
| CN113077476A (en) * | 2021-03-17 | 2021-07-06 | 浙江大华技术股份有限公司 | Height measurement method, terminal device and computer storage medium |
| CN113840127A (en) * | 2021-08-12 | 2021-12-24 | 长光卫星技术有限公司 | Method for acquiring water area automatic mask processing DSM by satellite video image |
| CN114245097A (en) * | 2021-11-05 | 2022-03-25 | 丁俊飞 | Three-dimensional imaging method and device based on three-color mask single-color camera |
| WO2023083273A1 (en) * | 2021-11-10 | 2023-05-19 | 梅卡曼德（北京）机器人科技有限公司 | Grip point information acquisition method and apparatus, electronic device, and storage medium |
| CN114140527A (en) * | 2021-11-19 | 2022-03-04 | 苏州科技大学 | Dynamic environment binocular vision SLAM method based on semantic segmentation |
| CN114758093A (en) * | 2022-03-31 | 2022-07-15 | 深圳万兴软件有限公司 | Three-dimensional model generation method, device, equipment and medium based on image sequence |
| CN115451964A (en) * | 2022-08-18 | 2022-12-09 | 武汉理工大学 | Ship scene simultaneous mapping and positioning method based on multi-mode mixed features |
| CN116007623A (en) * | 2022-12-02 | 2023-04-25 | 纯米科技(上海)股份有限公司 | Robot navigation method, apparatus and computer readable storage medium |
| CN115619790A (en) * | 2022-12-20 | 2023-01-17 | 北京维卓致远医疗科技发展有限责任公司 | Mixed perspective method, system and equipment based on binocular positioning |
| CN116597086A (en) * | 2023-05-19 | 2023-08-15 | 京东科技控股股份有限公司 | Image processing method and device |
Non-Patent Citations (3)
| Title |
|---|
| JIAMING SUN ETC.: "Disp r-cnn: Stereo 3d object detection via shape prior guided instance disparity estimation", PROCEEDINGS OF THE IEEE/CVF CONFERENCE ON COMPUTER VISION ANDPATTERN RECOGNITION (CVPR) * |
| 柳碧辉等: "基于改进MASK_R-CNN的舰船目标三维识别", 无线电通信技术, vol. 48, no. 1, pages 1 - 2 * |
| 邵平等: "掩膜边缘图及带掩膜图像窗口的统计特征快速算法", 肇庆学院学报, vol. 36, no. 5 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117173650A (en) * | 2023-11-02 | 2023-12-05 | 浙江华是科技股份有限公司 | Ship measurement and identification method and system based on laser radar |
| CN117173650B (en) * | 2023-11-02 | 2024-01-26 | 浙江华是科技股份有限公司 | Ship measurement and identification method and system based on laser radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2016112714A1 (en) | Assistant docking method and system for vessel | |
| CN110517325B (en) | Coordinate transformation and method and system for positioning objects around vehicle body through coordinate transformation | |
| CN109297413B (en) | Visual measurement method for large-scale cylinder structure | |
| CN105809689B (en) | Hull six degree of freedom measurement method based on machine vision | |
| CN105716542B (en) | A kind of three-dimensional data joining method based on flexible characteristic point | |
| CN110174088A (en) | A kind of target ranging method based on monocular vision | |
| CN104880176A (en) | Moving object posture measurement method based on prior knowledge model optimization | |
| CN103852060B (en) | A kind of based on single visual visible images distance-finding method felt | |
| CN109035309A (en) | Pose method for registering between binocular camera and laser radar based on stereoscopic vision | |
| CN111435081B (en) | Sea surface measuring system, sea surface measuring method and storage medium | |
| CN111462236A (en) | Method and system for detecting relative pose between ships | |
| CN106871787A (en) | Large space line scanning imagery method for three-dimensional measurement | |
| CN116958265A (en) | Ship pose measurement method and system based on binocular vision | |
| CN111220126A (en) | Space object pose measurement method based on point features and monocular camera | |
| CN110047111A (en) | A kind of airplane parking area shelter bridge butting error measurement method based on stereoscopic vision | |
| CN111274959B (en) | Oil filling taper sleeve pose accurate measurement method based on variable field angle | |
| CN110470226A (en) | A kind of bridge structure displacement measurement method based on UAV system | |
| CN110966956A (en) | Binocular vision-based three-dimensional detection device and method | |
| CN114993608B (en) | Wind tunnel model three-dimensional attitude angle measuring method | |
| CN105806318A (en) | Visual measurement method for space three-dimensional information based on motion time quantity | |
| CN106296703A (en) | Scaling board, camera calibration method and device | |
| CN113554697A (en) | Cabin section profile accurate measurement method based on line laser | |
| CN112595236A (en) | Measuring device for underwater laser three-dimensional scanning and real-time distance measurement | |
| CN113739720A (en) | Underwater high-precision measurement and defect detection method integrating acoustic method and optical method | |
| CN112461213B (en) | Multi-mode wave monitoring device and monitoring method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |