CN113689492B - Sea surface distance measurement method and system based on monocular camera - Google Patents
Sea surface distance measurement method and system based on monocular camera Download PDFInfo
- Publication number
- CN113689492B CN113689492B CN202111235602.1A CN202111235602A CN113689492B CN 113689492 B CN113689492 B CN 113689492B CN 202111235602 A CN202111235602 A CN 202111235602A CN 113689492 B CN113689492 B CN 113689492B
- Authority
- CN
- China
- Prior art keywords
- camera
- ship
- rotation matrix
- angle information
- calculating
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000691 measurement method Methods 0.000 title description 7
- 239000011159 matrix material Substances 0.000 claims abstract description 151
- 238000005259 measurement Methods 0.000 claims abstract description 46
- 238000000034 method Methods 0.000 claims abstract description 42
- 238000013519 translation Methods 0.000 claims abstract description 35
- 239000013598 vector Substances 0.000 claims description 11
- 238000004422 calculation algorithm Methods 0.000 claims description 8
- 238000012545 processing Methods 0.000 claims description 5
- 238000004441 surface measurement Methods 0.000 abstract description 4
- 238000004590 computer program Methods 0.000 description 14
- 238000010586 diagram Methods 0.000 description 9
- 238000004364 calculation method Methods 0.000 description 4
- 238000006243 chemical reaction Methods 0.000 description 3
- 238000001514 detection method Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
Abstract
The application relates to a sea surface distance measuring method and system based on a monocular camera, wherein the method comprises the following steps: measuring the camera pose angle information of the monocular camera and the ship pose angle information of the ship, calculating to obtain a camera rotation matrix and a ship rotation matrix, calculating to obtain a system rotation matrix and a system translation matrix according to the altitude of the camera, the camera rotation matrix and the ship rotation matrix, acquiring the pixel coordinates of the image shot by the camera, further according to the internal parameters and distortion parameters of the monocular camera, and a system rotation matrix and a system translation matrix, converting the pixel coordinates into sea surface physical coordinates to complete data measurement between the target point and the ship, through the method and the device, the problems of high distance measurement difficulty and large distance measurement error in a sea surface distance measurement scene are solved, the measurement of data such as the distance between a target ship and a target point, the azimuth angle of the target ship, the width of a ship body and the like in sea surface measurement is realized, and the accuracy of data measurement in sea surface distance measurement is improved.
Description
Technical Field
The application relates to the technical field of sea surface distance measurement, in particular to a sea surface distance measurement method and system based on a monocular camera.
Background
The existing distance measuring method mainly comprises laser distance measuring and camera vision distance measuring. The laser ranging method has the characteristics of good monochromaticity, strong directivity and the like, is wide in ranging range and high in precision, and is widely applied to professional ranging scenes; the camera vision distance measurement can be rapidly adapted to a computer vision algorithm, the matching of the functions of target distance measurement, target detection, target tracking and the like is realized, and the method is widely applied to intelligent scenes.
However, in a sea surface ranging scene, ranging is often performed together with ship identification, so that laser ranging cannot be well adopted; sea ranging has the characteristics of large ranging range and unstable viewing angle, and the traditional binocular camera ranging or monocular camera ranging cannot be well applied to the scene. Meanwhile, the camera vision distance measurement is limited by factors such as camera distortion and distance algorithm, and the distance measurement error of a long-distance target is large.
At present, no effective solution is provided for the problems of high distance measurement difficulty and large distance measurement error in the sea surface distance measurement scene in the related technology.
Disclosure of Invention
The embodiment of the application provides a monocular camera-based sea surface distance measurement method and a monocular camera-based sea surface distance measurement system, which are used for at least solving the problems of high distance measurement difficulty and large distance measurement error in a sea surface distance measurement scene in the related technology.
In a first aspect, an embodiment of the present application provides a monocular camera-based sea-surface distance measuring method, where the method includes:
acquiring an image containing a ship and a target point through a monocular camera, and processing to obtain pixel coordinates of pixel points in the image;
calculating to obtain camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
measuring the camera pose angle information and the altitude of the monocular camera, and calculating to obtain a camera rotation matrix according to the camera pose angle information;
measuring and acquiring ship pose angle information of a ship in real time, and calculating according to the ship pose angle information to obtain a ship rotation matrix;
calculating to obtain a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
calculating to obtain sea surface physical coordinates of the pixel points in a sea surface coordinate system according to the camera physical coordinates of the pixel points, the system rotation matrix and the system translation matrix;
and finishing data measurement between the target point and the ship according to the sea surface physical coordinate.
In some embodiments, measuring camera pose angle information of a monocular camera, and calculating a camera rotation matrix from the camera pose angle information comprises:
measuring sheetThe camera pose angle information of the camera comprises a deviation angle mu and a pitch angle
And roll angle
By the formula
Calculating to obtain a camera rotation matrix R2。
In some embodiments, the obtaining of the ship pose angle information of the ship through real-time measurement, and the obtaining of the ship rotation matrix through calculation according to the ship pose angle information includes:
the method comprises the steps of measuring and acquiring ship pose angle information of a ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
By the formula
Calculating to obtain a ship rotation matrix R3。
In some embodiments, calculating a system rotation matrix and a system translation matrix from the altitude, the camera rotation matrix, and the vessel rotation matrix comprises:
according to the camera rotation matrix R2And the ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
according to the altitude h and the system rotation matrix R, passing through a formula
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
In some embodiments, before the camera physical coordinates of the pixel point in the camera coordinate system are calculated according to the pixel coordinates and the intrinsic parameters and distortion parameters of the monocular camera, the method further includes:
and acquiring an image containing a calibration plate through the monocular camera, and calculating internal parameters and distortion parameters of the monocular camera by using a Zhang Zhen you camera calibration algorithm.
In a second aspect, an embodiment of the present application provides a monocular camera-based sea surface distance measuring system, where the system includes a monocular camera module, a gyroscope module, and a host module, where the host module is configured to process data information provided by the monocular camera module and the gyroscope module, and perform an operation to generate a sea surface distance measuring result;
the monocular camera module acquires an image containing a ship and a target point, and the host module processes the image to obtain pixel coordinates of pixel points in the image;
the host module calculates camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
the monocular camera module measures the camera pose angle information and the altitude of the monocular camera, and the host computer module calculates to obtain a camera rotation matrix according to the camera pose angle information;
the gyroscope module measures and acquires ship pose angle information of a ship in real time, and the host module calculates according to the ship pose angle information to obtain a ship rotation matrix;
the host module calculates a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
the host module calculates to obtain sea surface physical coordinates of the pixel points in a sea surface coordinate system according to the camera physical coordinates of the pixel points, the system rotation matrix and the system translation matrix;
and the host module completes data measurement between the target point and the ship according to the sea surface physical coordinates.
In some embodiments, the monocular camera module measures camera pose angle information of the monocular camera, and the host module calculates the camera rotation matrix according to the camera pose angle information includes:
the monocular camera module measures camera pose angle information of the monocular camera, and the camera pose angle information comprises a deviation angle mu and a pitch angle
And roll angle
Said host module passing formula
Calculating to obtain a camera rotation matrix R2。
In some embodiments, the gyroscope module measures and acquires ship pose angle information of a ship in real time, and the host module calculates a ship rotation matrix according to the ship pose angle information, including:
the gyroscope module measures and acquires ship pose angle information of a ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
Said host module passing formula
Calculating to obtain a ship rotation matrix R3。
In some embodiments, the calculating, by the host module, a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix includes:
the host module rotates the matrix R according to the camera2And the ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
the host module passes a formula according to the altitude h and the system rotation matrix R
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
In some embodiments, before the calculating, by the host module, the camera physical coordinates of the pixel point in the camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera, the method further includes:
the monocular camera module acquires an image containing a calibration plate, and the host module calculates internal parameters and distortion parameters of the monocular camera through a Zhang Yongyou camera calibration algorithm.
Compared with the related technology, the monocular camera-based sea surface distance measurement method and the monocular camera-based sea surface distance measurement system provided by the embodiment of the application measure the camera pose angle information and the altitude of the monocular camera and further calculate to obtain the camera rotation matrix, measure and obtain the ship pose angle information of the ship in real time and further calculate to obtain the ship rotation matrix, calculate to obtain the system rotation matrix and the system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix, obtain the image containing the ship and the target point through the monocular camera, process to obtain the pixel coordinates of the pixel points in the image, further convert and calculate the pixel coordinates of the pixel points to obtain the sea surface physical coordinates in the sea surface coordinate system according to the internal parameters and distortion parameters of the monocular camera, the system rotation matrix and the system translation matrix, complete the data measurement between the target point and the ship, and solve the problems of high distance measurement difficulty and large distance measurement error in the sea surface distance measurement scene, the distance between the target ship and a target point, the azimuth angle of the target ship, the width of the ship body and other data in sea surface measurement are measured, and the accuracy of data measurement in sea surface distance measurement is improved.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
FIG. 1 is a flow chart of steps of a monocular camera-based sea-surface ranging method according to an embodiment of the present application;
FIG. 2 is a block diagram of a monocular camera-based sea-surface ranging system according to an embodiment of the present application;
FIG. 3 is a flow chart illustrating a processor executing a computer program in a memory according to the present embodiment;
fig. 4 is an internal structural diagram of an electronic device according to an embodiment of the present application.
Description of the drawings: 21. a monocular camera module; 22. a gyroscope module; 23. a host module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be described and illustrated below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments provided in the present application without any inventive step are within the scope of protection of the present application.
It is obvious that the drawings in the following description are only examples or embodiments of the present application, and that it is also possible for a person skilled in the art to apply the present application to other similar contexts on the basis of these drawings without inventive effort. Moreover, it should be appreciated that in the development of any such actual implementation, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developers' specific goals, such as compliance with system-related and business-related constraints, which may vary from one implementation to another.
Reference in the specification to "an embodiment" means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the specification. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Those of ordinary skill in the art will explicitly and implicitly appreciate that the embodiments described herein may be combined with other embodiments without conflict.
Unless defined otherwise, technical or scientific terms referred to herein shall have the ordinary meaning as understood by those of ordinary skill in the art to which this application belongs. Reference to "a," "an," "the," and similar words throughout this application are not to be construed as limiting in number, and may refer to the singular or the plural. The present application is directed to the use of the terms "including," "comprising," "having," and any variations thereof, which are intended to cover non-exclusive inclusions; for example, a process, method, system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or elements, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus. Reference to "connected," "coupled," and the like in this application is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The term "plurality" as referred to herein means two or more. "and/or" describes an association relationship of associated objects, meaning that three relationships may exist, for example, "A and/or B" may mean: a exists alone, A and B exist simultaneously, and B exists alone. The character "/" generally indicates that the former and latter associated objects are in an "or" relationship. Reference herein to the terms "first," "second," "third," and the like, are merely to distinguish similar objects and do not denote a particular ordering for the objects.
Example 1
The embodiment of the application provides a monocular camera-based sea surface distance measurement method, fig. 1 is a flow chart of steps of the monocular camera-based sea surface distance measurement method according to the embodiment of the application, and as shown in fig. 1, the method includes the following steps:
step S102, measuring the camera pose angle information and the altitude of the monocular camera, and calculating to obtain a camera rotation matrix according to the camera pose angle information;
specifically, the angle of the monocular camera is adjusted so that the shooting view of the monocular camera covers part of the deck of the ship, and at least 4 points on the deck of the ship are taken within the view range of the monocular camera, so that the 4 points are not collinear and are not over concentrated. Drawing a space rectangular coordinate system which is a ship coordinate system, enabling an xOy plane of the space rectangular coordinate system to be located on a deck to obtain ship physical coordinates of 4 points, obtaining camera physical coordinates of 4 points on a monocular camera picture, calculating according to the ship physical coordinates and the camera physical coordinates of 4 points and an n-point perspective principle to obtain camera pose angle information of the monocular camera, and calculating according to the camera pose angle information through an Euler angle to obtain a camera rotation matrix;
and calculating the altitude of the monocular camera according to the ship parameters and the installation position of the monocular camera.
Step S104, measuring and acquiring ship pose angle information of a ship in real time, and calculating according to the ship pose angle information to obtain a ship rotation matrix;
specifically, a gyroscope is arranged on the ground close to a room (such as a cab) on the ship, so that the gyroscope has an axis direction parallel to the forward direction of the ship body, then the axis x is parallel to the forward direction of the ship body, the axis y is perpendicular to the axis x and points to the left side of the ship body, and the axis z is perpendicular to the ground. And (3) measuring and acquiring ship pose angle information of the ship in real time through a gyroscope, and calculating according to the ship pose angle information through Euler angles to obtain a ship rotation matrix.
Step S106, calculating to obtain a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
step S108, acquiring an image containing a ship and a target point through a monocular camera, and processing to obtain pixel coordinates of pixel points in the image;
specifically, the coordinates corresponding to the pixel points in the pixel coordinate system are calculated. The pixel position is the coordinate of a pixel point on the camera imaging plane, in units of pixels (pixels).
Step S110, calculating camera physical coordinates of pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
in particular, by the formula
The physical coordinates of the camera are calculated, wherein,
as an internal parameter of the monocular camera, u and v are pixel coordinates, xc、ycAnd zcIs the camera physical coordinates, s is a constant that makes the third component of the equal-sign left-end vector 1.
It should be added that, in order to further improve the accuracy of the conversion between the pixel coordinate and the camera physical coordinate, at least 3 non-collinear points (a point with a z-axis coordinate of 0 in the sea surface coordinate system) may be taken on the sea surface, the coordinate of the point in the camera coordinate system is calculated, and then fitting is performed by the least square method, where the fitting formula is as follows:
wherein
，
T represents the transpose operation of the matrix, -1 represents the inverse operation of the matrix; solving the sea level equation of
。
Converting and calculating the physical coordinate x of the camera obtained by the pixel coordinates u and vc、ycAnd zcAre respectively set to x'c、y’cAnd z'cAre then combined
、
To obtain the real coordinates of the camera in the coordinate system
Then, step S112 is performed.
Step S112, calculating to obtain sea surface physical coordinates of the pixel points in a sea surface coordinate system according to the camera physical coordinates, the system rotation matrix and the system translation matrix of the pixel points;
in particular, by the formula
Calculating to obtain the sea surface physical coordinates, wherein r in the formulaijFor the system to rotate the ith row and jth column element in the matrix,
for the system translation matrix, xw、ywAnd zwAre the sea surface physical coordinates.
It should be added that, according to step S108 and step S110, the pixel coordinates and the sea surface physical coordinates can also be obtained through formulas
Direct conversion calculations are performed.
And step S114, finishing data measurement between the target point and the ship according to the sea surface physical coordinates.
Specifically, the data measurement between the target point and the ship according to the sea surface physical coordinates comprises the following steps:
calculating the distance between a target point and a ship, directly solving the Euclidean distance between the target point and the origin of coordinates according to the coordinates in a sea surface coordinate system to be used as the distance between the target point and the ship, if the distance between the ship and the target ship is calculated, combining a ship detector, using the middle point of the bottom edge of a detection frame as a reference point for calculating the distance of the target ship, and realizing the calculation of the distance between the ships after the position of the target ship is detected;
calculating the deflection angle of the ship, calculating the included angle between the connecting line of the target point and the ship (origin) and the x axis according to the coordinates under the sea surface coordinate system, wherein the absolute value of the included angle is the angle of the deflection angle, and the sign of the included angle represents the direction: positive is left deviation, negative is right deviation;
and calculating the width of the target ship, and calculating the distance between two end points on the bottom edge of the detection frame under the sea surface coordinate system as the width of the target ship after detecting the position of the target ship by combining a ship detector according to the coordinates under the sea surface coordinate system.
Through the steps S102 to S114 in the embodiment of the application, the problems of high distance measurement difficulty and large distance measurement error in a sea surface distance measurement scene are solved, the measurement of data such as the distance between a target ship and a target point, the azimuth angle of the target ship, the width of a ship body and the like in sea surface measurement is realized, and the accuracy of data measurement in the sea surface distance measurement is improved.
In some embodiments, the step S102 of measuring camera pose angle information of the monocular camera and calculating a camera rotation matrix according to the camera pose angle information includes:
measuring the camera pose angle information of the monocular camera, wherein the camera pose angle information comprises a deviation angle mu and a pitch angle
And roll angle
By the formula
Calculating to obtain a camera rotation matrix R2。
Specifically, at least 4 points on a ship deck are taken in the visual field range of the monocular camera, a space rectangular coordinate system is formulated, the space rectangular coordinate system is a ship coordinate system, the xOy plane of the space rectangular coordinate system is located on the deck, ship physical coordinates of the 4 points under the ship coordinate system are obtained, and then camera physical coordinates of the 4 points are obtained on a frame of the monocular camera.
Taking a vector parallel to the advancing direction of the ship under a deck coordinate system
A normal vector perpendicular to the deck
Based on the ship physical coordinates and the camera physical coordinates of the 4 points, vectors are converted by the n-point perspective principle
Sum vector
Respectively converted into coordinates in the camera coordinate system
And
further obtain the deflection angle of the monocular camera
And a pitch angle
And roll angle
；
Then by the formula
Calculating to obtain a camera rotation matrix R2。
In some embodiments, the step S104 of measuring and acquiring ship pose angle information of the ship in real time, and calculating a ship rotation matrix according to the ship pose angle information includes:
the method comprises the steps of measuring and acquiring ship pose angle information of a ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
By the formula
Calculating to obtain a ship rotation matrix R3。
Specifically, a gyroscope is arranged on the ground, such as a cab, which is tightly attached to a ship, so that the gyroscope has an axis direction which is parallel to the forward direction of a ship body, then the axis x is parallel to the forward direction of the ship body, the axis y is perpendicular to the axis x and points to the left side of the ship body, the axis z is perpendicular to the ground and faces upwards, and the gyroscope returns the angle of the axis y and the angle of the axis x in real time;
by passingThe gyroscope measures and obtains ship position and attitude angle information of a ship in real time, wherein the ship position and attitude angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
By the formula
Calculating to obtain a ship rotation matrix R3。
In some embodiments, the step S106 of calculating a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix includes:
according to the camera rotation matrix R2And ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
according to the altitude h and the system rotation matrix R, by formula
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
In some embodiments, in step S110, before calculating the camera physical coordinates of the pixel point in the camera coordinate system according to the pixel coordinates, and the internal parameter and distortion parameter of the monocular camera, the method further includes:
and acquiring an image containing a calibration plate through the monocular camera, and calculating internal parameters and distortion parameters of the monocular camera by using a Zhang Zhengyou camera calibration algorithm.
In particular, the intrinsic parameters of the monocular camera may be found
Wherein f isxAnd fyFocal lengths of the camera in the horizontal and vertical directions of the camera's picture, cxAnd cyRespectively the pixel coordinates of the projection of the camera optical center on the camera picture; distortion parameter of camera
Wherein k is1、k2And k3As a radial distortion parameter, p1And p2Is a tangential distortion parameter.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system, such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed in an order different than here.
Example 2
The embodiment of the application provides a monocular camera-based sea surface distance measuring system, and fig. 2 is a structural block diagram of the monocular camera-based sea surface distance measuring system according to the embodiment of the application, and the system includes a monocular camera module 21, a gyroscope module 22 and a host module 23, wherein the host module 23 is used for processing data information provided by the monocular camera module 21 and the gyroscope module 22, and calculating to generate a sea surface distance measuring result;
the monocular camera module 21 acquires an image containing a ship and a target point, and the host module 23 processes the image to obtain pixel coordinates of the pixel point in the image;
the host module 23 calculates camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
the monocular camera module 21 measures camera pose angle information and altitude of the monocular camera, and the host module 23 calculates a camera rotation matrix according to the camera pose angle information;
the gyroscope module 22 measures and acquires ship pose angle information of a ship in real time, and the host module 23 calculates according to the ship pose angle information to obtain a ship rotation matrix;
the host module 23 calculates a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
the host module 23 calculates to obtain the sea surface physical coordinates of the pixel points in the sea surface coordinate system according to the camera physical coordinates, the system rotation matrix and the system translation matrix of the pixel points;
the host module 23 completes data measurement between the target point and the ship according to the sea surface physical coordinates.
Through the monocular camera module 21, the gyroscope module 22 and the host module 23 in the embodiment of the application, the problems of high distance measurement difficulty and large distance measurement error in a sea surface distance measurement scene are solved, the measurement of data such as the distance between a target ship and a target point, the azimuth angle of the target ship, the width of a ship body and the like in sea surface measurement is realized, and the accuracy of data measurement in sea surface distance measurement is improved.
In some embodiments, the monocular camera module 21 measures camera pose angle information of the monocular camera, and the calculating, by the host module 23, the camera rotation matrix according to the camera pose angle information includes:
the monocular camera module 21 measures camera pose angle information of the monocular camera, the camera pose angle information including a yaw angle mu and a pitch angle
And roll angle
The host module 23 passes the formula
Calculating to obtain a camera rotation matrix R2。
In some embodiments, the gyroscope module 22 measures and acquires ship pose angle information of the ship in real time, and the calculating by the host module 23 according to the ship pose angle information to obtain the ship rotation matrix includes:
the gyroscope module 22 measures and acquires ship pose angle information of the ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
The host module 23 passes the formula
Calculating to obtain a ship rotation matrix R3。
In some embodiments, the calculating, by the host module 23, a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix includes:
the host module 23 rotates the matrix R according to the camera2And ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
the host module 23 passes through a formula based on the altitude h and the system rotation matrix R
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
In some embodiments, before the host module 23 calculates the camera physical coordinates of the pixel point in the camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera, the method further includes:
the monocular camera module 21 acquires an image including a calibration plate, and the host module 23 calculates internal parameters and distortion parameters of the monocular camera through a Zhang Zhen camera calibration algorithm.
The above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
Example 3
The present embodiment also provides an electronic device comprising a memory having a computer program stored therein and a processor configured to execute the computer program to perform the steps of any of the above method embodiments.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Alternatively, fig. 3 is a flow chart illustrating a processor according to the present embodiment executing a computer program in a memory, and as shown in fig. 3, the processor may be configured to execute the following steps by the computer program:
s1, measuring and calculating the camera pose and angle information and measuring and calculating the ship pose information;
s2, calculating a conversion matrix between the camera coordinate system and the sea surface coordinate system;
s3, calibrating camera internal parameters;
s4, converting and calculating the pixel coordinate, the camera physical coordinate and the sea surface physical coordinate;
and S5, measuring and calculating data between the ship and the target point.
It should be noted that, for specific examples in this embodiment, reference may be made to examples described in the foregoing embodiments and optional implementations, and details of this embodiment are not described herein again.
In addition, in combination with the monocular camera-based sea surface ranging method in the above embodiments, the embodiments of the present application may provide a storage medium to implement. The storage medium having stored thereon a computer program; the computer program when executed by a processor implements any of the monocular camera based sea-ranging methods of the above embodiments.
Example 4
In one embodiment, a computer device is provided, which may be a terminal. The computer device includes a processor, a memory, a network interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of an operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used for communicating with an external terminal through a network connection. The computer program is executed by a processor to implement a monocular camera based sea-ranging method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, a key, a track ball or a touch pad arranged on the shell of the computer equipment, an external keyboard, a touch pad or a mouse and the like.
In one embodiment, fig. 4 is a schematic diagram of an internal structure of an electronic device according to an embodiment of the present application, and as shown in fig. 4, there is provided an electronic device, which may be a server, and its internal structure diagram may be as shown in fig. 4. The electronic device comprises a processor, a network interface, an internal memory and a non-volatile memory connected by an internal bus, wherein the non-volatile memory stores an operating system, a computer program and a database. The processor is used for providing calculation and control capability, the network interface is used for communicating with an external terminal through network connection, the internal memory is used for providing an environment for an operating system and the running of a computer program, the computer program is executed by the processor to realize the monocular camera-based sea surface ranging method, and the database is used for storing data.
Those skilled in the art will appreciate that the configuration shown in fig. 4 is a block diagram of only a portion of the configuration associated with the present application, and does not constitute a limitation on the electronic device to which the present application is applied, and a particular electronic device may include more or less components than those shown in the drawings, or combine certain components, or have a different arrangement of components.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), Programmable ROM (PROM), Electrically Programmable ROM (EPROM), Electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), Rambus Direct RAM (RDRAM), direct bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
It should be understood by those skilled in the art that various features of the above-described embodiments can be combined in any combination, and for the sake of brevity, all possible combinations of features in the above-described embodiments are not described in detail, but rather, all combinations of features which are not inconsistent with each other should be construed as being within the scope of the present disclosure.
The above-mentioned embodiments only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (10)
1. A monocular camera-based sea surface ranging method is characterized by comprising the following steps:
measuring the camera pose angle information and the altitude of the monocular camera, and calculating to obtain a camera rotation matrix according to the camera pose angle information;
measuring and acquiring ship pose angle information of a ship in real time, and calculating according to the ship pose angle information to obtain a ship rotation matrix;
calculating to obtain a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
acquiring an image containing a ship and a target point through a monocular camera, and processing to obtain pixel coordinates of pixel points in the image;
calculating to obtain camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
calculating to obtain sea surface physical coordinates of the pixel points in a sea surface coordinate system according to the camera physical coordinates of the pixel points, the system rotation matrix and the system translation matrix;
and finishing data measurement between the target point and the ship according to the sea surface physical coordinate.
2. The method of claim 1, wherein measuring camera pose angle information for a monocular camera and computing a camera rotation matrix from the camera pose angle information comprises:
measuring phases of monocular cameraThe camera pose angle information comprises a deviation angle mu and a pitch angle
And roll angle
By the formula
Calculating to obtain a camera rotation matrix R2。
3. The method of claim 1, wherein the step of measuring and acquiring ship pose angle information of a ship in real time, and the step of calculating a ship rotation matrix according to the ship pose angle information comprises the steps of:
the method comprises the steps of measuring and acquiring ship pose angle information of a ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
By the formula
Calculating to obtain a ship rotation matrix R3。
4. The method of claim 1, wherein computing a system rotation matrix and a system translation matrix from the altitude, the camera rotation matrix, and the vessel rotation matrix comprises:
according to the camera rotation matrix R2And the ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
according to the altitude h and the system rotation matrix R, passing through a formula
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
5. The method of claim 1, wherein before calculating camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the intrinsic parameters and distortion parameters of the monocular camera, the method further comprises:
and acquiring an image containing a calibration plate through the monocular camera, and calculating internal parameters and distortion parameters of the monocular camera by using a Zhang Zhen you camera calibration algorithm.
6. A sea surface distance measuring system based on a monocular camera is characterized by comprising a monocular camera module, a gyroscope module and a host module, wherein the host module is used for processing data information provided by the monocular camera module and the gyroscope module and calculating to generate a sea surface distance measuring result;
the monocular camera module measures the camera pose angle information and the altitude of the monocular camera, and the host computer module calculates to obtain a camera rotation matrix according to the camera pose angle information;
the gyroscope module measures and acquires ship pose angle information of a ship in real time, and the host module calculates according to the ship pose angle information to obtain a ship rotation matrix;
the host module calculates a system rotation matrix and a system translation matrix according to the altitude, the camera rotation matrix and the ship rotation matrix;
the monocular camera module acquires an image containing a ship and a target point, and the host module processes the image to obtain pixel coordinates of pixel points in the image;
the host module calculates camera physical coordinates of the pixel points in a camera coordinate system according to the pixel coordinates and the internal parameters and distortion parameters of the monocular camera;
the host module calculates to obtain sea surface physical coordinates of the pixel points in a sea surface coordinate system according to the camera physical coordinates of the pixel points, the system rotation matrix and the system translation matrix;
and the host module completes data measurement between the target point and the ship according to the sea surface physical coordinates.
7. The system of claim 6, wherein the monocular camera module measures camera pose angle information for the monocular camera, and wherein the host module calculating the camera rotation matrix from the camera pose angle information comprises:
the monocular camera module measures camera pose angle information of the monocular camera, and the camera pose angle information comprises a deviation angle mu and a pitch angle
And roll angle
Said host module passing formula
Calculating to obtain a camera rotation matrix R2。
8. The system of claim 6, wherein the gyroscope module measures and acquires ship pose angle information of a ship in real time, and the host module calculates a ship rotation matrix according to the ship pose angle information, and comprises:
the gyroscope module measures and acquires ship pose angle information of a ship in real time, wherein the ship pose angle information comprises a longitudinal axis deflection angle
Angle of deflection of transverse axis
Said host module passing formula
Calculating to obtain a ship rotation matrix R3。
9. The system of claim 6, wherein the host module calculates a system rotation matrix and a system translation matrix from the altitude, the camera rotation matrix, and the vessel rotation matrix, comprising:
the host module rotates the matrix R according to the camera2And the ship rotation matrix R3By the formula
Calculating to obtain a system rotation matrix R;
the host module passes a formula according to the altitude h and the system rotation matrix R
Calculating to obtain a system translation matrix
Wherein the translation vector of the monocular camera in the sea surface coordinate system
。
10. The system of claim 6, wherein before the host module calculates the camera physical coordinates of the pixel points in the camera coordinate system according to the pixel coordinates and the intrinsic parameters and distortion parameters of the monocular camera, the system further comprises:
the monocular camera module acquires an image containing a calibration plate, and the host module calculates internal parameters and distortion parameters of the monocular camera through a Zhang Yongyou camera calibration algorithm.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111235602.1A CN113689492B (en) | 2021-10-22 | 2021-10-22 | Sea surface distance measurement method and system based on monocular camera |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111235602.1A CN113689492B (en) | 2021-10-22 | 2021-10-22 | Sea surface distance measurement method and system based on monocular camera |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113689492A CN113689492A (en) | 2021-11-23 |
| CN113689492B true CN113689492B (en) | 2022-02-11 |
Family
ID=78587748
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111235602.1A Active CN113689492B (en) | 2021-10-22 | 2021-10-22 | Sea surface distance measurement method and system based on monocular camera |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113689492B (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109509226A (en) * | 2018-11-27 | 2019-03-22 | 广东工业大学 | Three dimensional point cloud method for registering, device, equipment and readable storage medium storing program for executing |
| CN109993798A (en) * | 2019-04-09 | 2019-07-09 | 上海肇观电子科技有限公司 | Method, equipment and the storage medium of multi-cam detection motion profile |
| CN111429532A (en) * | 2020-04-30 | 2020-07-17 | 南京大学 | Method for improving camera calibration accuracy by utilizing multi-plane calibration plate |
| CN111681279A (en) * | 2020-04-17 | 2020-09-18 | 东南大学 | Driving suspension arm space pose measurement method based on improved lie group nonlinear optimization |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101482410B (en) * | 2008-01-10 | 2010-10-13 | 宝山钢铁股份有限公司 | Calibration method for image measuring system |
| CN109523592A (en) * | 2018-10-19 | 2019-03-26 | 天津大学 | A kind of interior flame localization method based on camera |
-
2021
- 2021-10-22 CN CN202111235602.1A patent/CN113689492B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109509226A (en) * | 2018-11-27 | 2019-03-22 | 广东工业大学 | Three dimensional point cloud method for registering, device, equipment and readable storage medium storing program for executing |
| CN109993798A (en) * | 2019-04-09 | 2019-07-09 | 上海肇观电子科技有限公司 | Method, equipment and the storage medium of multi-cam detection motion profile |
| CN111681279A (en) * | 2020-04-17 | 2020-09-18 | 东南大学 | Driving suspension arm space pose measurement method based on improved lie group nonlinear optimization |
| CN111429532A (en) * | 2020-04-30 | 2020-07-17 | 南京大学 | Method for improving camera calibration accuracy by utilizing multi-plane calibration plate |
Non-Patent Citations (2)
| Title |
|---|
| SMT坐标平面旋转和平移的计算方法;李勇;《第十一届中国高端SMT学术会议论文集》;20171230;全文 * |
| 野外大视场单相机空间坐标测量系统的快速标定;王向军;《光学精工程》;20170730;第25卷(第7期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113689492A (en) | 2021-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10594941B2 (en) | Method and device of image processing and camera | |
| WO2018153374A1 (en) | Camera calibration | |
| US20120308126A1 (en) | Corresponding image processing method for compensating colour | |
| US20210227132A1 (en) | Method for tracking target in panoramic video, and panoramic camera | |
| US8755624B2 (en) | Image registration device and method thereof | |
| CN111711756A (en) | Image anti-shake method, electronic equipment and storage medium | |
| CN111489288B (en) | Image splicing method and device | |
| CN113222878B (en) | Image stitching method | |
| CN111340737B (en) | Image correction method, device and electronic system | |
| WO2019232793A1 (en) | Two-camera calibration method, electronic device and computer-readable storage medium | |
| CN111429530B (en) | Coordinate calibration method and related device | |
| CN114299156A (en) | Method for calibrating and unifying coordinates of multiple cameras in non-overlapping area | |
| CN108444452B (en) | Method and device for detecting longitude and latitude of target and three-dimensional space attitude of shooting device | |
| CN112465877A (en) | Kalman filtering visual tracking stabilization method based on motion state estimation | |
| CN112629565B (en) | Method, device and equipment for calibrating rotation relation between camera and inertial measurement unit | |
| CN113689492B (en) | Sea surface distance measurement method and system based on monocular camera | |
| CN113436267A (en) | Visual inertial navigation calibration method and device, computer equipment and storage medium | |
| CN111476842B (en) | Method and system for estimating relative pose of camera | |
| KR20180131743A (en) | Method and Apparatus for Stereo Matching of Wide-Angle Images using SIFT Flow | |
| CN115174878B (en) | Projection picture correction method, apparatus and storage medium | |
| JP2005275789A (en) | Three-dimensional structure extraction method | |
| CN112254812B (en) | Method, device and equipment for calculating overlapping region of camera spectral bands and storage medium | |
| CN112927276B (en) | Image registration method, device, electronic equipment and storage medium | |
| WO2018100230A1 (en) | Method and apparatuses for determining positions of multi-directional image capture apparatuses | |
| Kudinov et al. | The algorithm for a video panorama construction and its software implementation using CUDA technology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| PE01 | Entry into force of the registration of the contract for pledge of patent right |
Denomination of invention: A sea surface ranging method and system based on monocular camera Effective date of registration: 20231202 Granted publication date: 20220211 Pledgee: Hangzhou High-tech Financing Guarantee Co.,Ltd. Pledgor: Zhejiang Jianmu Intelligent System Co.,Ltd. Registration number: Y2023330002915 |
|
| PE01 | Entry into force of the registration of the contract for pledge of patent right |