CN106097235A - A kind of based on machine vision from alongside job parameter computational methods - Google Patents

Abstract

The present invention relates to a kind of based on machine vision from alongside job parameter computational methods, including: pre-build hull coordinate system, image coordinate system, camera coordinate system and mark source coordinate system, measure the distance of stem reference point and ship stern reference point, and the distance of camera installation site and stem reference point, obtain camera course angle and the angle of pitch identifying imaging moment on camera opposite bank；Calculate mark source coordinate system to the translation vector of camera coordinate system；Calculate hull coordinate system and the Eulerian angles of mark source coordinate system；Calculate from alongside job parameter according to above-mentioned parameter, it may be judged whether terminate from alongside operation, if it does not, again calculate from alongside job parameter, until from the alongside end of job；If it is, terminate to calculate.The present invention ensure on the bank mark can imaging in limited camera field range, relax camera visual field, the constraint of ship bank position orientation relation, improve from alongside parameter calculation precision；Calculating process is simple, and operand is little.

Description

Off-berthing operation parameter calculation method based on machine vision

Technical Field

The invention relates to the technical field of navigation, in particular to a method for calculating an off-berthing operation parameter based on machine vision.

Background

With the development of shipping industry, shipbuilding technology and marine technology have been advancing in a crossing manner, and the upsizing of ships has been turned into reality through a trend. Because of the characteristics of large-scale ships, such as large mass and volume, difficult operation, the ship needs to be assisted by a tugboat to finish the berthing. The berthing angle, the ship-shore distance and the ship-shore speed are important parameters in the berthing-off operation process, accurate auxiliary decision information is provided for a pilot by calculating the parameters in real time through information equipment, and the method is an effective way for efficiently and safely realizing berthing-off of a large ship. In recent years, instrumentation for assisting large ships in berthing off is rapidly developed, so that driving personnel can be promoted to utilize advanced equipment for digital quantitative berthing from the past purely experience-based berthing with feeling.

From the division of working principle, the off-berthing operation equipment can be divided into laser off-berthing operation equipment, differential guide off-berthing operation equipment and optical off-berthing operation equipment. The laser off-berthing operation equipment and the differential guide off-berthing operation equipment are developed more mature and are widely applied to ports and ships. The optical off-berthing operation equipment can be used under the weather conditions with poor visibility such as rain, fog and the like by utilizing the particularity of the solar blind ultraviolet band, and is gradually developed in recent years.

Currently, major problems with optical off-dock work equipment include: firstly, the coupling relation between the off-berthing operation parameters and the heading angle and the pitch angle of the camera is not considered at the same time, when the onshore identification is positioned at the edge of the camera field of view, the camera angle is difficult to adjust through a plurality of dimensions, the imaging quality of the onshore identification in the camera field of view is influenced, and the off-berthing operation parameter resolving precision is reduced; secondly, by adopting the different-surface identification, the off-berthing operation parameter calculation method is complex and the calculation amount is large.

Disclosure of Invention

In view of the above analysis, the present invention aims to provide a method for calculating parameters of an off-berthing operation based on machine vision, so as to solve the problems of insufficient accuracy of solving the parameters of the existing off-berthing operation and complex calculation.

The purpose of the invention is mainly realized by the following technical scheme:

the invention provides a method for calculating an off-berthing operation parameter based on machine vision, which comprises the following steps:

a ship body coordinate system O is established in advance_b-X_bY_bZ_bImage coordinate system I-XY, camera coordinate system O_c-X_cY_cZ_cAnd identifying source coordinatesIs O_u-X_uY_uZ_u；

Measuring the distance L between the fore reference point and the stern reference point, and the distance L between the camera mounting position and the fore reference point_c；

Acquiring a camera heading angle rho and a pitch angle of a camera at the imaging moment of the identification of the camera on the opposite bank;

calculating identification source coordinate system O_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^T；

Calculate hull coordinate system O_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle of

Calculating an off-berthing operation parameter according to the parameters, judging whether the off-berthing operation is finished, if not, re-acquiring a camera course angle rho and a pitch angle at the imaging moment of the camera on the shore, and calculating the off-berthing operation parameter again until the off-berthing operation is finished; if so, the calculation is ended.

Further, the off-berthing operation parameters include:

mooring angle β, distance L between bow and shoreline_BStern and shoreline distance L_SThe transverse speed V of the bow relative to the shoreline_BTransverse velocity V of stern relative to shoreline_S。

Further, the off-berthing operation parameters are calculated according to the following formula:

L_B＝T_xcosρ+T_ysinρsin+T_zsinρcos-L_csinβ

L_S＝L_B+Lsinβ

$V_B={\stackrel{\·}{L}}_B$

$V_S={\stackrel{\·}{L}}_S.$

further, calculating an identifier source coordinate system O_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^TThe process specifically comprises the following steps:

according to the image coordinate system I-XY and the camera coordinate system O_c-X_cY_cZ_cDefinition of (C), obtaining principal points (C) of the image by camera calibration_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_y；

Calculating pose intermediate variable e₁、e₃、e₄、e₆、e₇；

According to image principal point (C)_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_yAnd calculating a marker source coordinate system O by using pose intermediate variables_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^T。

Further, a hull coordinate system O is calculated_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle ofThe process specifically comprises the following steps:

θ＝sin^-1[(r₆sin+r₂cos)cosρ-r₄sinρ]

wherein r is₁、r₂、r₃、r₄、r₅、r₆Determined by the following equation:

r₁＝e₇T_z

r₂＝e₈T_z

the invention has the following beneficial effects:

the method ensures that the onshore marker can be imaged in a limited camera view field range, relaxes the constraint on the camera view field and ship-shore pose relationship, and improves the off-berthing parameter resolving precision; the calculation process is simple and the calculation amount is small.

Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims hereof as well as the appended drawings.

Drawings

The drawings are only for purposes of illustrating particular embodiments and are not to be construed as limiting the invention, wherein like reference numerals are used to designate like parts throughout.

FIG. 1 is a schematic flow chart of a method according to an embodiment of the present invention;

fig. 2 is a schematic diagram of an off-berthing operation scenario in the method according to the embodiment of the present invention.

Detailed Description

The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form a part hereof, and which together with the embodiments of the invention serve to explain the principles of the invention.

As shown in fig. 1, fig. 1 is a schematic flow chart of a method for calculating an off-berthing operation parameter based on machine vision according to an embodiment of the present invention, which may specifically include the following steps:

step 101: establishing ship coordinatesIs O_b-X_bY_bZ_bImage coordinate system I-XY, camera coordinate system O_c-X_cY_cZ_cAnd mark the source coordinate system O_u-X_uY_uZ_u。

The definition of each coordinate system is as follows:

hull coordinate system O_b-X_bY_bZ_b: origin O_bAt center of mass, X, of the hull_bThe axis pointing to the starboard of the vessel, Y_bWith axis pointing towards bow, Z_bThe axis being perpendicular to the deck plane and directed upwards, with X_bAxis and Y_bThe axes constitute a right-hand coordinate system.

Image coordinate system I-XY: the origin I is located at the top left corner of the image, the X-axis points horizontally to the right, and the Y-axis points vertically downward.

Camera coordinate system O_c-X_cY_cZ_c: origin O_cAt the optical center of the camera, X_cAxis and Y_cThe axes being parallel to the X-axis and Y-axis of the image coordinate system, Z_cThe axis is along the camera optical axis and perpendicular to the planar image.

Identification source coordinate system O_u-X_uY_uZ_u: origin O_uAt a reference point of berth, X_uThe axis is perpendicular to the line in front of the wharf and is consistent with the mooring direction, Z_uPerpendicular to the dock floor and upwards, Y_uDetermined by the right hand coordinate system.

Step 102: with respect to the image coordinate system I-XY and the camera coordinate system O according to step 101_c-X_cY_cZ_cDefinition of (C), obtaining principal points (C) of the image by camera calibration_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_y。

Wherein, C_x、C_y、F_xAnd F_yAre all positive real numbers.

Step 103: with respect to identifying a source according to step 101System of symbols O_u-X_uY_uZ_uBy definition of (A), obtaining O_u-Y_uZ_uRespective marks P in plane_i(i ═ 1,2, 3.., n) coordinates (0 y)_iz_i)^T。

Wherein n is the number of identifiers, is a positive integer and is not less than 4; y is_i、z_i(i ═ 1,2, 3.., n) is a positive real number.

Step 104: measuring the distance L between the fore reference point and the stern reference point, and the distance L between the camera mounting position and the fore reference point_c。

Wherein L and L_cAre all positive real numbers.

Step 105: and acquiring a camera heading angle rho and a pitch angle of the camera at the imaging moment marked on the opposite shore of the camera.

Wherein rho represents the included angle between the optical axis of the camera and the north direction, the value is that rho is more than or equal to 0 and less than 2 pi, when the optical axis is coincident with the north direction, the value is 0, and the clockwise rotation is gradually increased; representing the angle between the optical axis of the camera and the horizontal plane, taking the value ofUpward is positive.

Step 106: for each mark P_i(i ═ 1,2, 3.., n) the coordinates of the image points in the image are extracted and recorded as

Wherein,andare all positive real numbers.

Step 107: pose intermediate variable e₁、e₂、e₃、e₄、e₅、e₆、e₇、e₈The calculation is carried out in the following two cases:

(I) when the identification source number is 4, that is, n in step 103 is 4, e₁、e₂、e₃、e₄、e₅、e₆、e₇、e₈The calculation is performed using equation (1):

wherein, as determined by steps 103 and 106.

(II) when the number of the identification sources is more than 4, namely n in the step three is more than 4, e₁、e₂、e₃、e₄、e₅、e₆、e₇、e₈The calculation is performed using equation (2):

in the formula, it is determined by steps 103 and 106 that in the subsequent calculations, only e is used₁、e₃、e₄、e₆、e₇These five quantities.

Step 108: calculating identification source coordinate system O_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^TThe calculation formula is as follows:

wherein e is₁、e₃、e₄、e₆、e₇Determined by step 106, C_x、C_y、F_x、F_yAs determined by step 102.

Step 109: calculate hull coordinate system O_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle ofThe calculation formula is as follows:

θ＝sin^-1[(r₆sin+r₂cos)cosρ-r₄sinρ](6)

wherein r is₁、r₂、r₃、r₄、r₅、r₆Determined by the following equation:

r₁＝e₇T_z(9)

r₂＝e₈T_z(10)

110, calculating the parameters of the off-berthing operation including the berthing angle β and the distance L between the bow and the shoreline_BStern and shoreline distance L_SThe transverse speed V of the bow relative to the shoreline_BTransverse velocity V of stern relative to shoreline_SThe calculation formula is as follows:

L_B＝T_xcosρ+T_ysinρsin+T_zsinρcos-L_csinβ (16)

L_S＝L_B+L sinβ (17)

wherein L and L_cDetermined by step four, ρ and by step five, T_x、T_y、T_zAs determined in the step eight, it is determined that,as determined by step 109.

Step 111: after the first off-berthing operation parameter calculation is completed through the steps 101 to 110, whether the off-berthing operation is finished or not is judged (when the distances between the bow and the shoreline and between the stern and the shoreline are zero, the off-berthing operation is finished), and if not, the step 105 is returned to carry out the subsequent time off-berthing operation parameter calculation; if so, the calculation is ended.

To facilitate an understanding of the methods described in the embodiments of the present invention, a specific example will be described below.

As shown in fig. 2, fig. 2 is a schematic diagram of a berthing-away operation scene in this example, a camera 1 is installed on the starboard of a ship, the shooting angle of the camera is adjusted by a rotating cradle head, the camera 1 images a shore mark every 1 second, the updating frequency of berthing-away operation parameters is 1Hz, the ship performs starboard berthing from a static state, the distance between a bow reference point 2 and a shoreline is 16.2 meters at the initial moment, and the distance between a stern reference point 3 and the shoreline is 29.3 meters.

Based on the scenario shown in fig. 2, the process of performing the undocking operation may specifically include:

step one, establishing a ship body coordinate system O_b-X_bY_bZ_bImage coordinate system I-XY, camera coordinate system O_c-X_cY_cZ_cAnd mark the source coordinate system O_u-X_uY_uZ_u. The definition of each coordinate system is as follows:

hull coordinate system O_b-X_bY_bZ_b: origin O_bIs located atCenter of mass of hull, X_bThe axis pointing to the starboard of the vessel, Y_bWith axis pointing towards bow, Z_bThe axis being perpendicular to the deck plane and directed upwards, with X_bAxis and Y_bThe axes constitute a right-hand coordinate system.

Image coordinate system I-XY: the origin I is located at the top left corner of the image, the X-axis points horizontally to the right, and the Y-axis points vertically downward.

Camera coordinate system O_c-X_cY_cZ_c: origin O_cAt the optical center of the camera, X_cAxis and Y_cThe axes being parallel to the X-axis and Y-axis of the image coordinate system, Z_cThe axis is along the camera optical axis and perpendicular to the planar image.

Identification source coordinate system O_u-X_uY_uZ_u: origin O_uAt a reference point of berth, X_uThe axis is perpendicular to the line in front of the wharf and is consistent with the mooring direction, Z_uPerpendicular to the dock floor and upwards, Y_uDetermined by the right hand coordinate system.

Step two, according to the step one, the image coordinate system I-XY and the camera coordinate system O_c-X_cY_cZ_cDefinition of (C), obtaining principal points (C) of the image by camera calibration_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_y。

Wherein, C_x、C_y、F_xAnd F_yAre all positive real numbers.

In this embodiment, C_xAnd C_yAre all 300 pixels, F_xAnd F_yAre all 200 pixels.

Thirdly, identifying a source coordinate system O according to the step one_u-X_uY_uZ_uBy definition of (A), obtaining O_u-Y_uZ_uRespective marks P in plane_i(i ═ 1,2, 3.., n) coordinates (0 y)_iz_i)^T。

Wherein n is a labelIdentifying the number, which is a positive integer and not less than 4; y is_i、z_i(i ═ 1,2, 3.., n) is a positive real number.

In the present embodiment, n is 4; y is₁2 m, z₁1 m; y is₂3 m, z₂6 m; y is₃6 m, z₃5 m; y is₄7 m, z₄2 m.

Step four, measuring the distance L between the bow reference point 2 and the stern reference point 3 and the distance L between the camera mounting position and the bow reference point 1_c。

Wherein L and L_cAre all positive real numbers.

In this embodiment, L is 150 m, L_c50 meters.

And fifthly, acquiring a heading angle rho and a pitch angle of the camera at the imaging moment of the identification of the camera on the opposite bank.

Wherein rho represents the included angle between the optical axis of the camera and the north direction, the value is that rho is more than or equal to 0 and less than 2 pi, when the optical axis is coincident with the north direction, the value is 0, and the clockwise rotation is gradually increased; representing the angle between the optical axis of the camera and the horizontal plane, taking the value ofUpward is positive.

In this embodiment, ρ is 85 ° and 0.

Step six, marking each mark P_i(i ═ 1,2, 3.., n) the coordinates of the image points in the image are extracted and recorded as

Wherein,andare all positive real numbers。

In the present embodiment, it is preferred that,

seventhly, aligning the intermediate variable e of the attitude₁、e₂、e₃、e₄、e₅、e₆、e₇、e₈The calculation is carried out in the following two cases:

(I) when the number of the identification sources is 4, that is, n in step three is 4:

wherein, the determination is carried out through the third step and the sixth step.

(II) when the number of the identification sources is more than 4, namely n in the step three is more than 4:

in the formula, determined by step three S6 and step six S9.

In this example, the process of (I) was followedLine calculation, e₁＝-20、e₂＝0、e₃＝2700、e₄＝0、e₅＝-20、e₆＝1500、e₇＝0、e₈＝0。

Eighthly, calculating an identifier source coordinate system O_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^TThe calculation formula is as follows:

wherein e is₁、e₃、e₄、e₆、e₇Determined by step six, C_x、C_y、F_x、F_yDetermined by step two.

In the present example, T is calculated by the formula (22) to the formula (24)_x120 m, T_y60 m, T_z10 meters.

Step nine, calculating a ship body coordinate system O_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle ofThe calculation formula is as follows:

θ＝sin^-1[(r₆sin+r₂cos)cosρ-r₄sinρ](25)

wherein r is₁、r₂、r₃、r₄、r₅、r₆Determined by the following equation:

r₁＝e₇T_z(28)

r₂＝e₈T_z(29)

in this example, θ is 0, γ is 0, calculated by formula (25) to formula (33),

step ten, calculating the off-berthing operation parameters including a berthing angle β and the distance L between the bow and the shoreline_BStern and shoreline distance L_SThe transverse speed V of the bow relative to the shoreline_BBoat and shipTransverse velocity V of stern relative to shore line_SThe calculation formula is as follows:

L_B＝T_xcosρ+T_ysinρsin+T_zsinρcos-L_csinβ (35)

L_S＝L_B+Lsinβ (36)

wherein L and L_cDetermined by step four, ρ and by step five, T_x、T_y、T_zAs determined in the step eight, it is determined that,determined by step nine.

In this example, β is calculated to be 5 ° and L by equation (34) to equation (38)_B16.06 m, L_S29.14 m, V_B0.14 m/s, V_S0.16 m/s.

Step eleven, after the first off-berthing operation parameter calculation is completed through the steps one to step eleven, judging whether the off-berthing operation is finished or not, if not, returning to the step five to calculate the off-berthing operation parameter at the subsequent moment; if so, the calculation is ended.

In summary, the embodiments of the present invention provide a method for calculating an off-berthing operation parameter based on machine vision, which considers the coupling relationship between an off-berthing operation parameter and a heading angle and a pitch angle of a camera, allows the camera angle to be manually or automatically adjusted during the off-berthing operation, ensures that an on-shore identifier can image in a limited camera field range, relaxes the constraint on the camera field and ship shore pose relationship, and improves the off-berthing parameter calculation accuracy; and secondly, a method for calculating the off-berthing operation parameters based on four or more coplanar identification sources is provided, the calculation process is simple, the calculation amount is small, and the requirement of real-time calculation on computer resources is reduced.

Those skilled in the art will appreciate that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program, which is stored in a computer readable storage medium, to instruct related hardware. The computer readable storage medium is a magnetic disk, an optical disk, a read-only memory or a random access memory.

Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention. Accordingly, the appended claims are intended to include within their scope such processes, devices, means, methods, or steps.

The above description is only for the preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention.

Claims (5)

1. A method for calculating an off-berthing operation parameter based on machine vision is characterized by comprising the following steps:

a ship body coordinate system O is established in advance_b-X_bY_bZ_bImage coordinate system I-XY, camera coordinate system O_c-X_cY_cZ_cAnd mark the source coordinate system O_u-X_uY_uZ_u；

Measuring the distance L between the fore reference point and the stern reference point, and the distance L between the camera mounting position and the fore reference point_c；

Acquiring a camera heading angle rho and a pitch angle of a camera at the imaging moment of the identification of the camera on the opposite bank;

calculating identification source coordinate system O_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^T；

Calculate hull coordinate system O_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle of

Calculating an off-berthing operation parameter according to the parameters, judging whether the off-berthing operation is finished, if not, re-acquiring a camera course angle rho and a pitch angle at the imaging moment of the camera on the shore identification, and calculating the off-berthing operation parameter again until the off-berthing operation is finished; if so, the calculation is ended.

2. The method of claim 1, wherein the off-berthing operation parameters comprise:

mooring angle β, distance L between bow and shoreline_BStern and shoreline distance L_SThe transverse speed V of the bow relative to the shoreline_BTransverse velocity V of stern relative to shoreline_S。

3. The method of claim 2, wherein the off-berthing operation parameters are calculated according to the following formula:

L_B＝T_xcosρ+T_ysinρsin+T_zsinρcos-L_csinβ

L_S＝L_B+Lsinβ

$V_B={\stackrel{\·}{L}}_B$

$V_S={\stackrel{\·}{L}}_S.$

4. the method of claim 1, wherein the computed marker source coordinate system O is a source coordinate system_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^TThe process specifically comprises the following steps:

according to the image coordinate system I-XY and the camera coordinate system O_c-X_cY_cZ_cDefinition of (C), obtaining principal points (C) of the image by camera calibration_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_y；

Calculating pose intermediate variable e₁、e₃、e₄、e₆、e₇；

According to image principal point (C)_x,C_y) Transverse equivalent focal length F_xLongitudinal equivalent focal length F_yAnd calculating a marker source coordinate system O by using pose intermediate variables_u-X_uY_uZ_uCoordinate system of imaging machine O_c-X_cY_cZ_cTranslation vector [ T ]_xT_yT_z]^T。

5. Method according to claim 1, characterized in that the hull coordinate system O is calculated_b-X_bY_bZ_bAnd mark the source coordinate system O_u-X_uY_uZ_uEuler angle ofThe process specifically comprises the following steps:

$\mathrm{\θ}={\sin }^{-1}\[(r_{6}sin\mathrm{\δ}+r_{2}cos\mathrm{\δ})cos\mathrm{\ρ}-r_{4}sin\mathrm{\ρ}\]$

$\mathrm{\γ}={\cos }^{-1}\left(\frac{r_{2}sin\mathrm{\δ}-r_{6}cos\mathrm{\δ}}{cos\mathrm{\θ}}\right)$

wherein r is₁、r₂、r₃、r₄、r₅、r₆Determined by the following equation:

r₁＝e₇T_z

r₂＝e₈T_z

$r_3=\frac{e_1{T}_z-C_x{r}_1}{F_x}$

$r_4=\frac{e_2{T}_z-C_x{r}_2}{F_x}$

$r_5=\frac{e_4{T}_z-C_y{r}_1}{F_y}$

$r_6=\frac{e_5{T}_z-C_y{r}_2}{F_y}.$