CN115760930A

CN115760930A - Multi-camera-based water surface target tracking method and device, computer equipment and storage medium

Info

Publication number: CN115760930A
Application number: CN202211646510.7A
Authority: CN
Inventors: 程宇威; 朱健楠; 许浒; 池雨豪; 虞梦苓
Original assignee: Shaanxi Orca Electronic Intelligent Technology Co ltd
Current assignee: Shaanxi Orca Electronic Intelligent Technology Co ltd
Priority date: 2022-12-21
Filing date: 2022-12-21
Publication date: 2023-03-07

Abstract

The invention relates to a multi-camera-based water surface target tracking method, a multi-camera-based water surface target tracking device, computer equipment and a storage medium, wherein the method comprises the following steps: acquiring image data of a plurality of cameras to obtain target detection results and image characteristics of the plurality of cameras; converting the target detection result into a detection result of a world coordinate system; merging the detection results of the world coordinate system to obtain a fusion position result, and extracting target appearance characteristics from the image characteristics; inputting the fusion position result and the target appearance characteristic into a multi-target tracker, and iterating tracker parameters to obtain a multi-target tracker result; and outputting a multi-camera target tracking result in real time at the next moment. The multi-camera multi-target tracking method based on the appearance characteristic association realizes multi-camera multi-target tracking results, in addition, the detection target can still be stably tracked after being shielded through appearance characteristic association, and the provided tracking result has better usability for subsequent path planning of the unmanned ship.

Description

Multi-camera-based water surface target tracking method and device, computer equipment and storage medium

Technical Field

The invention relates to the technical field of water surface target tracking, in particular to a multi-camera-based water surface target tracking method, a multi-camera-based water surface target tracking device, computer equipment and a storage medium.

Background

Unmanned ships have been widely used in scientific research and water surface work in recent years, and have been gradually developed toward intellectualization. With the deep learning technology, a great breakthrough is made in the field of image processing. The unmanned ship can also realize intelligent perception to surrounding waters environment by carrying the cameras, and the unmanned ship can carry a plurality of cameras to carry out 360-degree omnidirectional detection on surrounding ships and obstacles to obtain a detection result, so that the unmanned ship has important significance for safe navigation of the unmanned ship. In the unmanned ship target detection system, besides the detection target, a target number ID is acquired through a target tracking algorithm, and the target position and the target number ID are provided for a planning module for decision control.

The traditional target tracking algorithm is usually based on single camera for target tracking, and the problem of target tracking across multiple cameras cannot be solved. In addition, there may be different kinds of situations with multiple cameras carried by unmanned ships: such as different focal lengths and different view angles, so that a target tracking algorithm of the unmanned ship needs a more complex processing method.

Disclosure of Invention

The invention aims to overcome the defects of the prior art and provides a multi-camera-based water surface target tracking method, a multi-camera-based water surface target tracking device, computer equipment and a storage medium.

In order to solve the technical problem, the invention adopts the following technical scheme:

in a first aspect, the present embodiment provides a multi-camera based water surface target tracking method, including the following steps:

acquiring image data of a plurality of cameras to obtain target detection results and image characteristics of the plurality of cameras;

converting the target detection result into a detection result of a world coordinate system;

merging the detection results of the world coordinate system to obtain a fusion position result, and extracting target appearance characteristics from the image characteristics according to the fusion position result;

inputting the fusion position result and the target appearance characteristic into a multi-target tracker, and iterating tracker parameters to obtain a multi-target tracker result;

and outputting the target tracking result of the multiple cameras in real time at the next moment according to the result of the multi-target tracker.

The further technical scheme is as follows: the method for acquiring the image data of the multiple cameras to obtain the target detection results and the image characteristics of the multiple cameras comprises the following steps of:

establishing a world coordinate system by taking the position of the electrification time on the unmanned ship as an origin and taking the north and east directions as positive directions;

acquiring image data of n cameras at a set moment;

and inputting the image data of the n cameras into a target detection model to obtain a target detection result and image characteristics.

The further technical scheme is as follows: the method for converting the target detection result into the detection result of the world coordinate system comprises the following steps:

obtaining an external parameter matrix and an internal parameter matrix in advance through calibration;

establishing an unmanned ship coordinate system by taking the position of the unmanned ship at a set moment as an origin and taking the heading direction and the right-side direction of the unmanned ship as positive directions;

converting a target detection result into a coordinate system of the unmanned ship according to the external reference matrix and the internal reference matrix;

and transferring the target detection result from the unmanned ship coordinate system to a world coordinate system to obtain a detection result of the world coordinate system.

The further technical scheme is as follows: the method for combining the detection results of the world coordinate system to obtain a fusion position result and extracting the target appearance characteristics from the image characteristics according to the fusion position result comprises the following steps:

combining the detection results of the world coordinate system to obtain a fusion position result;

acquiring an image area corresponding to an original target detection result according to the index of the fusion position result;

and cutting out image set characteristics corresponding to the image areas according to the image characteristics, and performing dimension reduction extraction on the image set characteristics to obtain corresponding target appearance characteristics.

In a second aspect, the present embodiment provides a multi-camera based water surface target tracking device, including: the device comprises an acquisition unit, a conversion unit, a merging and extracting unit, an input iteration unit and an output unit;

the acquisition unit is used for acquiring image data of the multiple cameras to obtain target detection results and image characteristics of the multiple cameras;

the conversion unit is used for converting the target detection result into a detection result of a world coordinate system;

the merging and extracting unit is used for merging the detection results of the world coordinate system to obtain a fusion position result and extracting target appearance characteristics from the image characteristics according to the fusion position result;

the input iteration unit is used for inputting the fusion position result and the target appearance characteristic into the multi-target tracker and iterating the tracker parameters to obtain a multi-target tracker result;

and the output unit is used for outputting the multi-camera target tracking result in real time at the next moment according to the multi-target tracker result.

The further technical scheme is as follows: the acquisition unit includes: the device comprises a first establishing module, a first obtaining module and an input module;

the first establishing module is used for establishing a world coordinate system by taking the position of the electricity-collecting time on the unmanned ship as an origin and taking the north-righting direction and the east-righting direction as positive directions;

the first acquisition module is used for acquiring the image data of the n cameras at a set moment;

and the input module is used for inputting the image data of the n cameras into the target detection model so as to obtain a target detection result and image characteristics.

The further technical scheme is as follows: the merging extraction unit includes: the system comprises a calibration module, a second establishing module, a conversion module and a transfer module;

the calibration module is used for obtaining an external parameter matrix and an internal parameter matrix in advance through calibration;

the second establishing module is used for establishing a coordinate system of the unmanned ship by taking the position of the unmanned ship at the set moment as an origin and the orientation direction and the right-side direction of the unmanned ship as positive directions;

the conversion module is used for converting the target detection result into a coordinate system of the unmanned ship according to the external reference matrix and the internal reference matrix;

and the transfer module is used for transferring the target detection result from the unmanned ship coordinate system to the world coordinate system to obtain the detection result of the world coordinate system.

The further technical scheme is as follows: the input iteration unit includes: the cutting and dimension-reducing extraction module comprises a merging module, a second acquisition module and a cutting and dimension-reducing extraction module;

the merging module is used for merging the detection results of the world coordinate system to obtain a fusion position result;

the second acquisition module is used for acquiring an image area corresponding to the original target detection result according to the index of the fusion position result;

and the cutting dimension reduction extraction module is used for cutting out image set features corresponding to the image areas according to the image features and performing dimension reduction extraction on the image set features to obtain corresponding target appearance features.

In a third aspect, the present embodiment provides a computer device, which includes a memory and a processor, where the memory stores a computer program, and the processor implements the multi-camera based water surface target tracking method as described above when executing the computer program.

In a fourth aspect, the present embodiments provide a storage medium storing a computer program comprising program instructions that, when executed by a processor, may implement the multi-camera based water surface target tracking method as described above.

Compared with the prior art, the invention has the beneficial effects that: the target detection result is obtained by carrying a plurality of cameras on the unmanned ship to carry out 360-degree omnidirectional target detection on the water surface, the multi-target tracking result of the plurality of cameras is realized, and when the plurality of cameras are crossed for a large target, a plurality of detection results can be fused, the error tracking of the plurality of targets is avoided, in addition, the correlation is realized through appearance characteristics, the detection target can still be stably tracked after being shielded, the tracking target ID does not jump, and the provided tracking result has better usability for the follow-up path planning of the unmanned ship.

The invention is further described below with reference to the accompanying drawings and specific embodiments.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a first schematic flow diagram of a multi-camera-based water surface target tracking method according to an embodiment of the present invention;

fig. 2 is a schematic flow diagram of a multi-camera-based water surface target tracking method according to an embodiment of the present invention;

fig. 3 is a schematic view of an application scenario of a multi-camera-based water surface target tracking method according to an embodiment of the present invention;

fig. 4 is a third schematic flowchart of a multi-camera-based water surface target tracking method according to an embodiment of the present invention;

fig. 5 is a schematic flow diagram of a fourth method for tracking a water surface target based on multiple cameras according to an embodiment of the present invention;

FIG. 6 is a block diagram of a sampled multilayer perceptron according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of a process of tracker optimization iteration provided by an embodiment of the present invention;

FIG. 8 is a first schematic block diagram of a multi-camera based water surface target tracking device according to an embodiment of the present invention;

FIG. 9 is a second schematic block diagram of a multi-camera based water surface target tracking device according to an embodiment of the present invention;

fig. 10 is a schematic block diagram three of a multi-camera-based water surface target tracking device provided in an embodiment of the present invention;

fig. 11 is a fourth schematic block diagram of a multi-camera based water surface target tracking device provided by the embodiment of the invention;

FIG. 12 is a schematic block diagram of a computer device provided by an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

Referring to the specific embodiment shown in fig. 1, the invention discloses a water surface target tracking method based on multiple cameras, which comprises the following steps:

s1, acquiring image data of a plurality of cameras to obtain target detection results and image characteristics of the plurality of cameras;

in an embodiment, referring to fig. 2 to 3, the acquiring image data of a plurality of cameras to obtain target detection results and image features of the plurality of cameras includes the following steps:

s11, establishing a world coordinate system by taking the position of the unmanned ship at the electricity-collecting time as an origin and taking the due north direction and the due east direction as positive directions;

in particular, the position of the electricity time on the unmanned ship

As an origin, in the true north direction

In the positive direction of the axis, the east direction is

The positive direction of the axis establishes a world coordinate system.

S12, acquiring image data of n cameras at a set moment;

specifically, the time is set to

At any moment, the unmanned ship acquires image data of n cameras

Wherein, the acquisition of the n images at the same time can be triggered synchronously by hardware or can be realized by matching the number of the images adjacent to the frame number through softwareAccordingly, the method is simple and convenient.

And S13, inputting the image data of the n cameras into a target detection model to obtain a target detection result and image characteristics.

Specifically, the camera image data

Inputting the data into a target detection model to obtain a target detection result

And an image feature F. And the image feature F is obtained by extracting a network according to the main features in the target detection model.

Further, when processing image data of the camera, each image may be sequentially input into the object detection model to obtain a single image result, or all images may be simultaneously input into one large object detection model to obtain all object detection results simultaneously.

Carrying out tracking pretreatment on a target detection result, and obtaining the target detection result

Including the type of the detection target frame and the position of the target detection frame. Firstly, normalizing the target frame to obtain a new target frame position (x, y, a, b):

；

；

；

；

wherein, (x 1, y 1), (x 2, y 2) are the coordinates of the upper left corner and the lower right corner of the target detection frame position box, respectively, and w and h are the width and the height of the image. x, y, a and b are respectively the normalized abscissa center, the normalized ordinate center, the normalized abscissa width and the normalized ordinate height.

S2, converting the target detection result into a detection result of a world coordinate system;

in an embodiment, referring to fig. 4, the converting the target detection result into the detection result of the world coordinate system includes the following steps:

s21, obtaining an external reference matrix and an internal reference matrix in advance through calibration;

specifically, the unmanned ship obtains an external parameter matrix through calibration in advance

And internal reference matrix

。

Specifically, the unmanned ship is placed indoors and is static, calibration checkerboards are placed around the unmanned ship, checkerboard data are collected through a camera, and an external reference matrix and an internal reference matrix of the unmanned ship camera are obtained through an internal reference and external reference calibration algorithm (the internal reference and external reference calibration algorithm adopts the conventional algorithm).

S22, establishing a coordinate system of the unmanned ship by taking the position of the unmanned ship at the set moment as an origin and the orientation direction and the right-side direction of the unmanned ship as positive directions;

in particular, considering that the images acquired by the unmanned ship are based on the unmanned ship coordinate system, the results of all cameras need to be unified into the world coordinate system so as to facilitate the subsequent data integration and fusion processing. Firstly, establishing a coordinate system of the unmanned ship to use the unmanned ship

The time position is the origin, and the orientation direction of the unmanned ship is used as the origin

The positive direction of the axis is right to the unmanned ship

And establishing a coordinate system of the unmanned ship in the positive direction of the axis.

S23, converting a target detection result into a coordinate system of the unmanned ship according to the external reference matrix and the internal reference matrix;

specifically, the target is detected

And (3) converting into an unmanned ship coordinate system:

；

wherein the content of the first and second substances,

and the detection frame is the ith target detection frame under the coordinate system of the unmanned ship.

And S24, transferring the target detection result from the unmanned ship coordinate system to a world coordinate system to obtain a detection result of the world coordinate system.

Specifically, the target detection result is transferred from the coordinate system of the unmanned ship to the world coordinate system, and the unmanned ship acquires the target detection result

GPS position of time of day

And yaw angle data

Detecting the target

Along the edge

Rotating the angle in the opposite direction, and then carrying out position compensation to obtain a target detection result of the world coordinate system

:

。

S3, combining the detection results of the world coordinate system to obtain a fusion position result, and extracting target appearance characteristics from the image characteristics according to the fusion position result;

in an embodiment, referring to fig. 5, the merging the detection results of the world coordinate system to obtain a fusion position result, and extracting the target appearance feature from the image feature according to the fusion position result includes the following steps:

s31, combining the detection results of the world coordinate system to obtain a fusion position result;

specifically, the detection results of the world coordinate system are combined to obtain a fusion position result D, the combination rule is that the intersection ratio IOU of the position of one target frame and the positions of the other n-1 target frames is calculated in sequence, if the IOU is larger than 0.2, the two targets are considered to be overlapped into one target, and otherwise, the two targets are processed according to the two targets.

Furthermore, strategy design can be performed according to the category of the target frame, and only targets in the same category are fused, for example, only detected manned cruise ship targets are fused by an IOU, or a plurality of category targets are selected to form a large category for the IOU.

S32, acquiring an image area corresponding to the original target detection result according to the index of the fusion position result;

specifically, according to the index j of the fusion position result D, the original target detection result is obtained

Corresponding image area

。

And S33, cutting out image set characteristics corresponding to the image areas according to the image characteristics, and performing dimension reduction extraction on the image set characteristics to obtain corresponding target appearance characteristics.

Specifically, cutting out image set characteristics corresponding to the image area according to the image characteristics F, and performing dimension reduction extraction on the image set characteristics

To obtain corresponding target appearance characteristics

；

。

Further, referring to fig. 6, the dimensionality reduction extraction network is a sampling multilayer perceptron, and the dimensionality reduction image appearance feature is obtained after the layers are activated through a 3 × 3 convolution layer and four times of upsampling layers. When the temperature is higher than the set temperature

When the temperature of the water is higher than the set temperature,

obtaining all target appearance characteristics for the fused target frame number

。

S4, inputting the fusion position result and the target appearance characteristics into the multi-target tracker, and iterating tracker parameters to obtain a multi-target tracker result;

specifically, referring to FIG. 7, a tracker state space X is first constructed, including the abscissa X, the ordinate y, the transverse width a, the longitudinal length b, and the abscissa velocity of the target frame in the world coordinate system

Rate of ordinate

Rate of change of transverse width

Rate of change of longitudinal length

:

；

Constructing a state transition matrix according to a dynamic model of the unmanned ship

Then according to the state transition matrix

Prediction

State space of moments

：

；

Specifically, the unmanned ship dynamics model is a common newton kinematics motion model, i.e. new position = original position + velocity × time difference. The state transition matrix is the state transition from time t to time t + 1. dt is the time difference between time t and time t + 1.

According to

State space of moments

And state transition matrix

Prediction

State space of moments

:

；

Further, a covariance matrix L of the state space is calculated, and then calculated

Optimized estimation of state space of time instants

：

；

Wherein, the first and the second end of the pipe are connected with each other,

a noise estimation matrix is defined as normal distribution noise by default;

by passing

Optimized estimation of state space of time instants

Performing Euclidean distance calculation with the state space X (the Euclidean distance calculation is a common calculation method in the prior art and is not specifically described here) to obtain a position matching result

；

Matching the positions to the result

And appearance feature matching result

Adding the addition coefficients to obtain the result of the multi-target tracker

：

；

Wherein the content of the first and second substances,

for hyper-parametric, multi-target tracker results

Included

The position of the tracking target frame at the moment and the type of the tracking target.

And S5, outputting a multi-camera target tracking result in real time at the next moment according to the multi-target tracker result.

Specifically, the unmanned ship outputs the multi-camera target tracking result in real time through the steps S1 to S4 at the next moment.

Specifically, the unmanned ship obtains trained target detection model weights from a deep learning cloud server, loads the trained target detection model weights to the unmanned ship, obtains a plurality of camera image raw data, GPS position raw data and IMU raw data from a sensor in real time, and then performs data preprocessing and analysis to obtain image data, GPS data and yaw angle data collected by a plurality of cameras. And further, performing real-time reasoning on the preprocessed sensor data, and outputting a multi-camera target tracking result R in real time by the unmanned ship through the steps S1-S4.

According to the invention, the unmanned ship carries a plurality of cameras to carry out 360-degree omnidirectional target detection on the water surface to obtain the target detection result, so that the multi-target tracking result for the plurality of cameras is realized, and when a large target spans the plurality of cameras, the plurality of detection results can be fused to avoid tracking the plurality of targets by mistake.

Referring to fig. 8, the present invention also discloses a water surface target tracking device based on multiple cameras, which includes: the device comprises an acquisition unit 10, a conversion unit 20, a merging extraction unit 30, an input iteration unit 40 and an output unit 50;

the acquiring unit 10 is configured to acquire image data of multiple cameras to obtain target detection results and image features of the multiple cameras;

the conversion unit 20 is used for converting the target detection result into a detection result of a world coordinate system;

the merging and extracting unit 30 is configured to merge the detection results of the world coordinate system to obtain a fusion position result, and extract a target appearance feature from the image feature according to the fusion position result;

the input iteration unit 40 is used for inputting the fusion position result and the target appearance characteristic into the multi-target tracker, and iterating the tracker parameters to obtain a multi-target tracker result;

and the output unit 50 is used for outputting the multi-camera target tracking result in real time at the next moment according to the multi-target tracker result.

In one embodiment, referring to fig. 9, the obtaining unit 10 includes: the system comprises a first establishing module 11, a first obtaining module 12 and an input module 13;

the first establishing module 11 is configured to establish a world coordinate system by using the position of the unmanned ship at the time of electrification as an origin and using the due north direction and the due east direction as positive directions;

the first obtaining module 12 is configured to obtain image data of n cameras at a set time;

the input module 13 is configured to input image data of the n cameras into the target detection model to obtain a target detection result and image features.

In one embodiment, referring to fig. 10, the merge extraction unit 20 includes: the calibration module 21, the second establishing module 22, the conversion module 23 and the transfer module 24;

the calibration module 21 is configured to obtain an external parameter matrix and an internal parameter matrix in advance through calibration;

the second establishing module 22 is configured to establish a coordinate system of the unmanned ship by using the position of the unmanned ship at the set time as an origin and by using the heading direction and the right-side direction of the unmanned ship as positive directions;

the conversion module 23 is used for converting the target detection result into the coordinate system of the unmanned ship according to the external parameter matrix and the internal parameter matrix;

the transferring module 24 is configured to transfer the target detection result from the unmanned ship coordinate system to a world coordinate system to obtain a detection result of the world coordinate system.

In one embodiment, please refer to fig. 11, the input iteration unit 30 includes: a merging module 31, a second obtaining module 32 and a cutting dimension-reducing extraction module 33;

the merging module 31 is configured to merge the detection results of the world coordinate system to obtain a result of the fusion location;

the second obtaining module 32 is configured to obtain an image area corresponding to the original target detection result according to the index of the fusion position result;

the cutting dimension reduction extraction module 33 is configured to cut out image set features corresponding to the image regions according to the image features, and perform dimension reduction extraction on the image set features to obtain corresponding target appearance features.

It should be noted that, as can be clearly understood by those skilled in the art, for the specific implementation process of the above-mentioned multi-camera-based water surface target tracking apparatus and each unit, reference may be made to the corresponding description in the foregoing method embodiment, and for convenience and simplicity of description, details are not repeated here.

The multi-camera based surface target tracking device described above may be implemented in the form of a computer program that may be run on a computer device as shown in fig. 12.

Referring to fig. 12, fig. 12 is a schematic block diagram of a computer device according to an embodiment of the present application; the computer device 500 may be a terminal or a server, where the terminal may be an electronic device with a communication function, such as a smart phone, a tablet computer, a notebook computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of a plurality of servers.

Referring to fig. 12, the computer device 500 includes a processor 502, memory, and a network interface 505 connected by a system bus 501, where the memory may include a non-volatile storage medium 503 and an internal memory 504.

The non-volatile storage medium 503 may store an operating system 5031 and computer programs 5032. The computer programs 5032 include program instructions that, when executed, cause the processor 502 to perform a multi-camera based surface target tracking method.

The processor 502 is used to provide computing and control capabilities to support the operation of the overall computer device 500.

The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503, and when the computer program 5032 is executed by the processor 502, the processor 502 can be enabled to execute a multi-camera based water surface target tracking method.

The network interface 505 is used for network communication with other devices. Those skilled in the art will appreciate that the configuration shown in fig. 12 is a block diagram of only a portion of the configuration associated with the present application and does not constitute a limitation of the computer device 500 to which the present application may be applied, and that a particular computer device 500 may include more or less components than those shown, or may combine certain components, or have a different arrangement of components.

Wherein the processor 502 is configured to run the computer program 5032 stored in the memory to perform the steps of:

the method comprises the following steps of S1, acquiring image data of a plurality of cameras to obtain target detection results and image characteristics of the plurality of cameras;

s2, converting a target detection result into a detection result of a world coordinate system;

It should be understood that in the embodiment of the present Application, the Processor 502 may be a Central Processing Unit (CPU), and the Processor 502 may also be other general-purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, and the like. Wherein a general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It will be understood by those skilled in the art that all or part of the flow of the method implementing the above embodiments may be implemented by a computer program instructing associated hardware. The computer program includes program instructions, and the computer program may be stored in a storage medium, which is a computer-readable storage medium. The program instructions are executed by at least one processor in the computer system to implement the flow steps of the embodiments of the method described above.

Accordingly, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program, wherein the computer program comprises program instructions that, when executed by a processor, may implement the multi-camera based water surface target tracking method described above. The storage medium stores a computer program comprising program instructions which, when executed by a processor, implement the method described above. The program instructions include the steps of:

The storage medium may be a usb disk, a removable hard disk, a Read-Only Memory (ROM), a magnetic disk, or an optical disk, which can store various computer readable storage media.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the technical solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.

In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative. For example, the division of each unit is only one logic function division, and there may be another division manner in actual implementation. For example, various elements or components may be combined or may be integrated into another system, or some features may be omitted, or not implemented.

The steps in the method of the embodiment of the invention can be sequentially adjusted, combined and deleted according to actual needs. The units in the device of the embodiment of the invention can be merged, divided and deleted according to actual needs. In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a storage medium. Based on such understanding, the technical solution of the present invention essentially or partially contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a terminal, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention.

The above embodiments are preferred implementations of the present invention, and besides, the present invention can be implemented in other ways, and any obvious substitutions without departing from the concept of the present invention are within the protection scope of the present invention.

Claims

1. A water surface target tracking method based on multiple cameras is characterized by comprising the following steps:

and outputting a multi-camera target tracking result in real time at the next moment according to the multi-target tracker result.

2. The multi-camera based water surface target tracking method according to claim 1, wherein the acquiring of the image data of the plurality of cameras to obtain the target detection results and the image characteristics of the plurality of cameras comprises the following steps:

acquiring image data of n cameras at a set moment;

3. The multi-camera based water surface target tracking method according to claim 2, wherein the step of converting the target detection result into a detection result of a world coordinate system comprises the following steps:

obtaining an external reference matrix and an internal reference matrix in advance through calibration;

establishing an unmanned ship coordinate system by taking the position of the unmanned ship at a set moment as an origin and the orientation direction and the right-side direction of the unmanned ship as positive directions;

converting a target detection result into a coordinate system of the unmanned ship according to the external parameter matrix and the internal parameter matrix;

4. The multi-camera based water surface target tracking method according to claim 3, wherein the detection results of the world coordinate system are combined to obtain a fusion position result, and the target appearance feature is extracted from the image feature according to the fusion position result, comprising the following steps:

5. Surface of water target tracking means based on many cameras, its characterized in that includes: the device comprises an acquisition unit, a conversion unit, a merging and extracting unit, an input iteration unit and an output unit;

6. The multi-camera based water surface target tracking device of claim 5, wherein the acquisition unit comprises: the device comprises a first establishing module, a first obtaining module and an input module;

the first acquisition module is used for acquiring image data of the n cameras at a set moment;

7. The multi-camera based water surface target tracking device of claim 6, wherein the merging extraction unit comprises: the system comprises a calibration module, a second establishing module, a conversion module and a transfer module;

the second establishing module is used for establishing a coordinate system of the unmanned ship by taking the position of the unmanned ship at the set moment as an origin and the direction of the unmanned ship and the right-side direction as positive directions;

8. The multi-camera based water surface target tracking device of claim 7, wherein the input iteration unit comprises: the merging module, the second acquisition module and the cutting dimension reduction extraction module;

9. A computer device, characterized in that the computer device comprises a memory having stored thereon a computer program and a processor which, when executing the computer program, implements a multi-camera based water surface target tracking method according to any one of claims 1-4.

10. A storage medium, characterized in that the storage medium stores a computer program comprising program instructions which, when executed by a processor, implement the multi-camera based water surface target tracking method of any one of claims 1-4.