CN107369178B - Underwater target tracking method and underwater equipment - Google Patents
Underwater target tracking method and underwater equipment Download PDFInfo
- Publication number
- CN107369178B CN107369178B CN201710615572.4A CN201710615572A CN107369178B CN 107369178 B CN107369178 B CN 107369178B CN 201710615572 A CN201710615572 A CN 201710615572A CN 107369178 B CN107369178 B CN 107369178B
- Authority
- CN
- China
- Prior art keywords
- image
- tracking
- information
- tracking target
- position information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/292—Multi-camera tracking
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S15/00—Systems using the reflection or reradiation of acoustic waves, e.g. sonar systems
- G01S15/66—Sonar tracking systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
Abstract
The invention provides a tracking method of an underwater target and underwater equipment, wherein the method comprises the following steps: acquiring a tracking image comprising a tracking target, wherein the tracking image comprises a first image and a second image which are respectively acquired by adopting different cameras; calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image, and tracking the tracking target based on the first relative position information; judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not; if yes, second relative position information between the underwater equipment and the tracking target is calculated by transmitting an inquiry signal to the tracking target and receiving a corresponding response signal transmitted by the tracking target, and the tracking target is tracked based on the second relative position information. The invention combines binocular tracking and acoustic wave tracking to realize distance tracking of the tracked target, further ensures the safety of the target and has better practicability.
Description
Technical Field
The invention relates to the technical field of target tracking, in particular to a tracking method of an underwater target and underwater equipment.
Background
In the research of marine science, underwater objects (e.g., divers) are often tracked using underwater devices. The underwater machine vision is a general technology, is widely used for assisting underwater application, and plays an important role in the aspects of robots, unmanned planes and the like.
In underwater machine vision, the position of a target in each frame of image is often required to be located, and target tracking is a key technology. The related underwater target tracking method shoots a target through image acquisition equipment so as to track the target according to the shot image. When the image acquisition device is too far away from the target, the tracking failure possibility exists due to the fact that the target cannot be effectively identified, so that the safety of the target cannot be further ensured, and the practicability is poor.
Disclosure of Invention
In view of this, the present invention provides a method for tracking an underwater target and an underwater device, which combine binocular tracking and acoustic wave tracking to achieve distance tracking of the tracked target, further ensure the safety of the target, and have good practicability.
In a first aspect, an embodiment of the present invention provides a method for tracking an underwater target, where the method includes:
the method comprises the steps that underwater equipment obtains a tracking image comprising a tracking target, wherein the tracking image comprises a first image and a second image which are respectively collected by different cameras;
calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image, and tracking the tracking target based on the first relative position information;
judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not;
if yes, second relative position information between the underwater equipment and the tracking target is calculated by transmitting an inquiry signal to the tracking target and receiving a response signal which is transmitted by the tracking target and corresponds to the inquiry signal, and the tracking target is tracked based on the second relative position information.
With reference to the first aspect, an embodiment of the present invention provides a first possible implementation manner of the first aspect, where the preset threshold is determined by an image attribute of the first image and/or an image attribute of the second image, where the image attribute of the first image at least includes a first image sharpness, and the image attribute of the second image at least includes a second image sharpness.
With reference to the first aspect, an embodiment of the present invention provides a second possible implementation manner of the first aspect, where the number of the response signals is multiple; the calculating of the second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal corresponding to the inquiry signal transmitted by the tracking target comprises:
calculating a phase difference between any two of the plurality of reply signals;
determining second azimuth information between the tracking target and the underwater equipment according to the calculated phase difference;
calculating second distance information between the underwater equipment and the tracking target according to a time difference between a first time of transmitting the inquiry signal and a second time of receiving the corresponding response signal;
determining the second distance information and the second orientation information as second relative position information.
With reference to the first aspect, an embodiment of the present invention provides a third possible implementation manner of the first aspect, where the calculating, according to the first image and the second image, first relative position information between the underwater device and the tracking target includes:
calculating first distance information between the underwater equipment and the tracking target according to the first image and the second image;
traversing and searching the first image, and determining first coordinate position information of the tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
determining first orientation information of the tracking target relative to the underwater equipment according to the first coordinate position information or the second coordinate position information;
determining the first distance information and the first orientation information as first relative position information.
With reference to the third possible implementation manner of the first aspect, an embodiment of the present invention provides a fourth possible implementation manner of the first aspect, where the performing a traversal search on the first image to determine first coordinate position information of the tracking target in the first image includes:
determining the scale information of the tracking target in the current first image based on first distance information corresponding to a previous first image of the current first image, the scale information of the tracking target in the previous first image and the first distance information corresponding to the current first image, wherein the current first image and the previous first image are images acquired by the same camera;
and traversing and searching the current first image according to the determined scale information, and determining first coordinate position information of the tracking target in the current first image.
With reference to the third possible implementation manner of the first aspect, an embodiment of the present invention provides a fifth possible implementation manner of the first aspect, where the calculating, according to the first image and the second image, first distance information between the underwater device and the tracking target includes:
extracting first image feature information from the first image and extracting second image feature information from the second image;
taking the first image characteristic information as reference information, and performing stereo matching processing on all the second image characteristic information and any one of the reference information respectively to obtain a corresponding matching point relation;
according to the corresponding relation of the matching points, calculating parallax information between the first image and the second image;
and obtaining first distance information between the underwater equipment and the tracking target according to the parallax information.
In a second aspect, an embodiment of the present invention further provides an underwater apparatus, where the apparatus includes:
the tracking system comprises an image acquisition module, a tracking module and a tracking module, wherein the image acquisition module is used for acquiring a tracking image comprising a tracking target, and the tracking image comprises a first image and a second image which are respectively acquired by adopting different cameras;
the first calculation tracking module is used for calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image and tracking the tracking target based on the first relative position information;
the judging module is used for judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not;
and the second calculation tracking module is used for calculating second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal which is transmitted by the tracking target and corresponds to the inquiry signal when the first distance information is judged to be greater than the preset threshold value, and tracking the tracking target based on the second relative position information.
With reference to the second aspect, an embodiment of the present invention provides a first possible implementation manner of the second aspect, where the first computation tracking module includes:
a first distance calculation unit, configured to calculate first distance information between the underwater device and the tracking target according to the first image and the second image;
the traversal search unit is used for performing traversal search on the first image and determining first coordinate position information of the tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
the first orientation determining unit is used for determining first orientation information of the tracking target relative to the underwater equipment according to the first coordinate position information or the second coordinate position information;
a first position determination unit configured to determine the first distance information and the first orientation information as first relative position information.
With reference to the first possible implementation manner of the second aspect, an embodiment of the present invention provides a second possible implementation manner of the second aspect, where the traversal search unit includes:
the scale confirmation subunit is configured to determine, based on first distance information corresponding to a previous first image of a current first image, scale information of a tracking target in the previous first image, and first distance information corresponding to the current first image, scale information of the tracking target in the current first image, where the current first image and the previous first image are images acquired by using the same camera;
and the traversal search subunit is configured to perform traversal search on the current first image according to the determined scale information, and determine first coordinate position information of the tracking target in the current first image.
With reference to the first possible implementation manner of the second aspect, an embodiment of the present invention provides a third possible implementation manner of the second aspect, where the second computation tracking module includes:
a calculation unit for receiving the plurality of reply signals and calculating a phase difference between any two of the plurality of reply signals;
a second azimuth determination unit configured to determine second azimuth information between the tracking target and the underwater device according to the calculated phase difference;
the second distance calculation unit is used for calculating second distance information between the underwater equipment and the tracking target according to a time difference between first time of transmitting the inquiry signal and second time of receiving the corresponding response signal;
a second position determination unit configured to determine the second distance information and the second orientation information as second relative position information.
Compared with the prior art that when the image acquisition equipment is too far away from the target, the tracking failure possibility exists due to the fact that the target cannot be effectively identified, and the practicability is poor, the underwater target tracking method and the underwater target tracking equipment firstly acquire a tracking image comprising the tracking target, wherein the tracking image comprises a first image and a second image which are acquired by different cameras respectively; then, calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image, and tracking the tracking target based on the first relative position information; then judging whether first distance information between the underwater device and the tracking target corresponding to the first relative position information is larger than a preset threshold value, calculating second relative position information between the underwater device and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal corresponding to the inquiry signal transmitted by the tracking target when the first distance information is judged to be larger than the preset threshold value, tracking the tracking target based on the second relative position information, switching the binocular tracking mode to the acoustic wave tracking mode when the first distance information between the underwater device and the tracking target is judged to be larger than the preset threshold value, calculating second relative position information between the underwater device and the tracking target by transmitting the inquiry signal to the tracking target and receiving the corresponding response signal transmitted by the tracking target, and tracking based on the second relative position information, the binocular tracking and the acoustic wave tracking are combined to realize the distance tracking of the tracked target, the safety of the target is further ensured, and the practicability is better.
In order to make the aforementioned and other objects, features and advantages of the present invention comprehensible, preferred embodiments accompanied with figures are described in detail below.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present invention and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained according to the drawings without inventive efforts.
FIG. 1 is a flow chart illustrating a method for tracking an underwater target according to an embodiment of the present invention;
FIG. 2 is a flow chart illustrating another method for tracking an underwater target according to an embodiment of the present invention;
FIG. 3 is a flow chart illustrating another method for tracking an underwater target according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating another method for tracking an underwater target according to an embodiment of the present invention;
FIG. 5 is a flow chart illustrating another method for tracking an underwater target according to an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating modular components of a subsea installation provided by an embodiment of the present invention;
FIG. 7 is a schematic diagram illustrating a modular configuration of a first computation tracking module in an underwater apparatus according to an embodiment of the present invention;
FIG. 8 is a schematic diagram illustrating a module composition of a traversal search unit in underwater equipment according to an embodiment of the present invention;
FIG. 9 is a schematic diagram showing the modular components of a first distance calculating unit in the underwater apparatus according to the embodiment of the present invention;
fig. 10 is a schematic diagram illustrating a module composition of a second computation tracking module in a subsea equipment according to an embodiment of the present invention.
Description of the main element symbols:
11. an image acquisition module; 22. a first computation tracking module; 33. a judgment module; 44. a second computational tracking module; 221. a first distance calculation unit; 222. traversing the search unit; 223. a first orientation determining unit; 224. a first position determination unit; 441. a calculation unit; 442. a second orientation determination unit; 443. a second distance calculation unit; 444. a second position determination unit; 2221. a scale confirmation subunit; 2222. traversing the search subunit; 2211. a feature extraction subunit; 2212. a matching processing subunit; 2213. a parallax calculation subunit; 2214. a distance calculation subunit.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. The components of embodiments of the present invention generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the present invention, presented in the figures, is not intended to limit the scope of the invention, as claimed, but is merely representative of selected embodiments of the invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments of the present invention without making any creative effort, shall fall within the protection scope of the present invention.
In the related art, when the image acquisition device is too far away from the target, the tracking failure may occur due to the failure to effectively recognize the target, so that the safety of the target cannot be further ensured, and the practicability is poor. Based on the above, the embodiment of the invention provides a tracking method of an underwater target and underwater equipment, which combine binocular tracking and acoustic wave tracking to realize distance tracking of the tracked target, further ensure the safety of the target and have better practicability.
Referring to a flowchart of a method for tracking an underwater target provided by an embodiment of the present invention shown in fig. 1, an execution subject of the method is an underwater device or a controller or other control chip in the underwater device, and the method specifically includes the following steps:
s101, acquiring a tracking image comprising a tracking target, wherein the tracking image comprises a first image and a second image which are respectively acquired by adopting different cameras;
specifically, in consideration of a specific application scenario of the underwater target tracking method provided by the embodiment of the present invention, the tracking images in the embodiment of the present invention include two images of the tracking target, where the two images are images synchronously acquired by two different cameras or two binocular cameras, but the corresponding shooting angles of the two images are different.
S102, calculating first relative position information between the underwater equipment and a tracking target according to the first image and the second image, and tracking the tracking target based on the first relative position information;
specifically, the underwater target tracking method provided by the embodiment of the invention calculates the first relative position information between the underwater device and the tracked target through a binocular vision algorithm, and tracks the tracked target based on the first relative position information. The first relative position information comprises first distance information between the underwater equipment and the tracking target obtained according to the first image and the second image, and first direction information of the tracking target relative to the underwater target.
S103, judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not;
and S104, if yes, calculating second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal which is transmitted by the tracking target and corresponds to the inquiry signal, and tracking the tracking target based on the second relative position information.
Specifically, the underwater target tracking method provided by the embodiment of the present invention tracks and aerial-shoots divers using image data acquired through binocular acquisition, and when an object tracked through binocular is further away from an underwater device, pixels of the object imaged in a camera are smaller and smaller, an imaging effect is gradually deteriorated, and a tracking effect is slowly decreased, that is, when it is determined that first distance information is greater than a preset threshold value, an acoustic wave tracking method is added, and remote tracking is performed based on second relative position information calculated by transmitting an interrogation signal to a tracked target and receiving a corresponding response signal transmitted by the tracked target, so that a good tracking effect can be achieved at a long distance. The embodiment of the invention adopts a sound wave mode to track the target at a long distance, and preferably adopts a sonar tracking mode.
When the target is tracked in a binocular mode, the target is limited to different preset thresholds corresponding to application scenes in different water areas, wherein the preset thresholds in the embodiment of the invention are determined by the image attributes of the first image and/or the second image, and the image attributes are changed based on different application scenes. In addition, the image definition is preferably used as the image attribute in the embodiment of the present invention. For example, if the definition of an image acquired by the underwater device in a turbid water area is poorer than that of an image acquired by the underwater device in a clean water area, and the ability of the underwater device for performing remote tracking in the turbid water area is obviously insufficient, the preset threshold value should be properly reduced to achieve the optimal binocular tracking effect, and vice versa. For the same set of underwater equipment, when the preset threshold value is determined to be 5m in the turbid water area, the clean water area can be determined to be 8 m.
In addition, when the second distance information in the second relative position information exceeds a preset range, a sonar alarm signal is sent to the tracked target through a sonar emission end, the tracked target can receive the sonar alarm signal through data communication between a sonar receiving end carried by the tracked target and the sonar emission end, the tracked target can know the too far distance from the underwater equipment according to the sonar alarm signal, the tracked target slowly waits for the underwater equipment to shorten the distance between the underwater equipment and the tracked target, binocular tracking is carried out again to obtain more detailed information, accurate positioning tracking is achieved, and the practicability is better.
Compared with the prior art that when the image acquisition equipment is too far away from the target, the tracking failure possibility exists due to the fact that the target cannot be effectively identified, and the practicability is poor, when the first distance information between the underwater equipment and the tracking target is judged to be larger than a preset threshold value, the binocular tracking mode is switched to the sound wave tracking mode, the second relative position information between the underwater equipment and the tracking target is calculated by transmitting an inquiry signal to the tracking target and receiving a corresponding response signal transmitted by the tracking target, tracking is carried out based on the second relative position information, distance tracking of the tracking target is achieved by combining the binocular tracking and the sound wave tracking, the safety of the target is further guaranteed, and the practicability is good.
Considering that the tracking method of the underwater target provided by the embodiment of the present invention tracks the tracking target according to the first relative position information and the second relative position information between the underwater device and the tracking target, it can be seen that the determination of the relative position information is a key step of the embodiment of the present invention. The second relative position information calculating process is specifically implemented by the following steps, referring to a flowchart shown in fig. 2, where the method further includes:
s201, calculating a phase difference between any two response signals in the plurality of response signals;
s202, determining second azimuth information between the tracking target and the underwater equipment according to the calculated phase difference;
s203, calculating second distance information between the underwater equipment and the tracking target according to the time difference between the first time of transmitting the inquiry signal and the second time of receiving the corresponding response signal;
and S204, determining the second distance information and the second direction information as second relative position information.
Specifically, the embodiment of the invention measures the second distance information according to the round-trip time of the speed of the sound wave propagating in the water and simultaneously measures the phase difference to determine the second azimuth information. In order to determine the second distance information and the second azimuth information, the method for tracking the underwater target provided by the embodiment of the invention adopts an ultra-short baseline positioning system. The positioning system comprises an acoustic measurement device and a data acquisition and processing device. The acoustic measuring device consists of an acoustic transducer arranged on the underwater equipment and an acoustic transponder arranged on a tracking target, wherein the acoustic transducer transmits an inquiry signal to the transponder, and the transponder transmits a response signal which is different from the inquiry signal back to the transducer after receiving the inquiry signal. In addition, in the embodiment of the invention, the plurality of units of the transducer array matrix are arrayed according to a triangle or a right angle, taking the triangle as an example, and the embodiment of the invention also takes the plane of the triangle as the plane of the reference coordinate system, determines second azimuth information between the underwater target and the tracking target by measuring and comparing phase differences among the response signals received by the units, and simultaneously measures second distance information to obtain second relative position information.
The above calculation process of the first relative position information is specifically realized by the following steps, referring to a flowchart shown in fig. 3, where the method further includes:
s301, calculating first distance information between the underwater equipment and the tracking target according to the first image and the second image;
s302, performing traversal search on the first image, and determining first coordinate position information of a tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
s303, determining first orientation information of the tracking target relative to the underwater equipment according to the first coordinate position information or the second coordinate position information;
s304, determining the first distance information and the first direction information as first relative position information.
Specifically, the underwater target tracking method provided by the embodiment of the present invention first calculates first distance information corresponding to the first relative position information according to the first image and the second image, then determines first orientation information corresponding to the first relative position information according to a traversal search result of the first image or the second image, and finally determines the first relative position information based on the first distance information and the first orientation information.
In order to avoid the problem of large calculation amount caused by the fact that when the first image is subjected to traversal search to search for the tracking target, the image needs to be zoomed in or zoomed out to perform traversal search, the embodiment of the invention determines the first coordinate position information of the tracking target in the first image through the following steps, and refer to fig. 4.
S401, determining the scale information of the tracking target in the current first image based on first distance information corresponding to a previous first image of the current first image, the scale information of the tracking target in the previous first image and the first distance information corresponding to the current first image, wherein the current first image and the previous first image are images acquired by the same camera;
s402, performing traversal search on the current first image according to the determined scale information, and determining first coordinate position information of the tracking target in the current first image.
Specifically, in the embodiment of the present invention, first, based on first distance information corresponding to a previous first image of a current first image, scale information of a tracking target in the previous first image, and first distance information corresponding to the current first image, scale information of the tracking target in the current first image is determined, and then, traversal search is performed on the current first image according to the determined scale information, so as to determine first coordinate position information of the tracking target in the current first image. Therefore, the embodiment of the invention determines the scale information of the current image through the scale information and the distance information of the previous image and the distance information of the current image, and can ensure that the image is zoomed near the determined scale information according to the corresponding relation between the scale information and the size of the image, thereby greatly reducing the calculation amount when determining the first coordinate position information.
Similarly, the embodiment of the present invention determines the second coordinate position information in the second image by the following steps, and determines the scale information of the tracking target in the current second image based on the first distance information corresponding to the previous second image of the current second image and the scale information of the tracking target in the previous second image, and the first distance information corresponding to the current second image, wherein the current second image and the previous second image are images acquired by using the same camera; and traversing and searching the current second image according to the determined scale information, and determining second coordinate position information of the tracking target in the current second image. The specific implementation manner is the same as the determination manner of the first coordinate position information, and is not described herein again.
In order to better calculate the first distance information between the underwater device and the tracking target, the calculating process of S301 is specifically implemented by the following steps, referring to a flowchart shown in fig. 5, and the method further includes:
s501, extracting first image characteristic information from a first image and extracting second image characteristic information from a second image;
s502, taking the first image characteristic information as reference information, and performing stereo matching processing on all second image characteristic information and any one reference information respectively to obtain a corresponding matching point relation;
s503, calculating parallax information between the first image and the second image according to the corresponding matching point correspondence;
s504, first distance information between the underwater equipment and the tracking target is obtained according to the parallax information.
Specifically, it is considered that the first image and the second image are both composed of a plurality of pixel points, and each pixel point corresponds to corresponding characteristic information. According to the underwater target tracking method provided by the embodiment of the invention, for the first image and the second image shot by the binocular camera, the first image is taken as the reference image, and the corresponding first image characteristic information is taken as the reference information. And selecting any pixel point from the first image, wherein the pixel point corresponds to the first image characteristic information of the pixel point, and selecting the pixel point of which the second image characteristic information is matched with the first image characteristic information from all the pixel points in the second image, namely, the pixel point matched with any pixel point in the first image is found from the second image, so that the corresponding relation of the matched point is obtained.
In the binocular vision algorithm in the embodiment of the invention, the projection center of the left camera is a coordinate origin, the projection center from the origin to the right camera is an X axis, the front vertical to the imaging plane of the camera is a Z axis, and the downward vertical to the X-Z plane is a Y axis. After the stereo matching is carried out, the parallax of each matching point can be obtained, and the first distance information between the underwater equipment and the tracking target is obtained by combining the camera parameter matrix.
Compared with the prior art that when the image acquisition equipment is too far away from the target, the tracking failure possibility exists due to the fact that the target cannot be effectively identified, and the practicability is poor, when the first distance information between the underwater equipment and the tracking target is judged to be larger than a preset threshold value, the binocular tracking mode is switched to the sound wave tracking mode, the second relative position information between the underwater equipment and the tracking target is calculated by transmitting an inquiry signal to the tracking target and receiving a corresponding response signal transmitted by the tracking target, tracking is carried out based on the second relative position information, distance tracking of the tracking target is achieved by combining the binocular tracking and the sound wave tracking, the safety of the target is further guaranteed, and the practicability is good.
An embodiment of the present invention further provides an underwater apparatus, configured to perform the above-mentioned method for tracking an underwater target, and referring to fig. 6, the apparatus includes:
the image acquisition module 11 is configured to acquire a tracking image including a tracking target, where the tracking image includes a first image and a second image that are acquired by using different cameras respectively;
the first calculation tracking module 22 is used for calculating first relative position information between the underwater equipment and a tracking target according to the first image and the second image and tracking the tracking target based on the first relative position information;
the judging module 33 is configured to judge whether first distance information between the underwater device corresponding to the first relative position information and the tracking target is greater than a preset threshold;
and the second calculation tracking module 44 is configured to calculate second relative position information between the underwater device and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal corresponding to the inquiry signal and transmitted by the tracking target when it is determined that the first distance information is greater than the preset threshold, and track the tracking target based on the second relative position information.
Considering that the underwater device provided by the embodiment of the present invention tracks the tracking target according to the first relative position information and the second relative position information between the underwater device and the tracking target, it can be seen that the determination of the relative position information is a key step of the embodiment of the present invention. Referring to fig. 7, the first computation tracking module 22 includes:
a first distance calculation unit 221, configured to calculate first distance information between the underwater device and the tracking target according to the first image and the second image;
the traversal search unit 222 is configured to perform traversal search on the first image, and determine first coordinate position information of the tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
a first orientation determining unit 223 for determining first orientation information of the tracking target with respect to the underwater device according to the first coordinate position information or the second coordinate position information;
a first position determination unit 224 configured to determine the first distance information and the first direction information as first relative position information.
In order to avoid the problem of large calculation amount caused by the need to zoom or enlarge the image for performing traversal search when performing traversal search on the first image to search for the tracking target, referring to fig. 8, the traversal search unit 222 in the embodiment of the present invention includes:
a scale determining subunit 2221, configured to determine, based on first distance information corresponding to a previous first image of a current first image, scale information of a tracking target in the previous first image, and first distance information corresponding to the current first image, scale information of the tracking target in the current first image, where the current first image and the previous first image are images acquired by using the same camera;
the traversal search subunit 2222 is configured to perform traversal search on the current first image according to the determined scale information, and determine first coordinate position information of the tracking target in the current first image.
In addition, the scale confirming subunit 2221 in this embodiment of the present invention is further configured to determine, based on the first distance information corresponding to the previous second image of the current second image, the scale information of the tracked target in the previous second image, and the first distance information corresponding to the current second image, the scale information of the tracked target in the current second image, where the current second image and the previous second image are images acquired by using the same camera; the traversal search subunit 2222 is further configured to perform traversal search on the current second image according to the determined scale information, and determine second coordinate position information of the tracking target in the current second image. The specific implementation manner is the same as the determination manner of the first coordinate position information, and is not described herein again.
In order to better calculate the first distance information between the underwater device and the tracking target, referring to fig. 9, the first distance calculating unit 221 includes:
a feature extraction subunit 2211 configured to extract first image feature information from the first image and second image feature information from the second image;
a matching processing subunit 2212, configured to perform stereo matching on all the second image feature information and any one of the reference information, respectively, with the first image feature information as the reference information, to obtain a corresponding matching relationship;
a disparity calculating subunit 2213 configured to calculate disparity information between the first image and the second image according to the matching point correspondence;
and the distance calculation subunit 2214 is configured to obtain first distance information between the underwater device and the tracking target according to the parallax information.
Additionally, referring to FIG. 10, the second computation tracking module 44 in an embodiment of the invention includes:
a calculating unit 441, configured to receive the plurality of response signals and calculate a phase difference between any two response signals in the plurality of response signals;
a second direction determination unit 442 for determining second direction information between the tracking target and the underwater device according to the calculated phase difference;
a second distance calculation unit 443 for calculating second distance information between the underwater device and the tracking target according to a time difference between a first time of transmitting the interrogation signal and a second time of receiving the corresponding response signal;
a second position determination unit 444 for determining the second distance information and the second orientation information as second relative position information.
Compared with the prior art that when the image acquisition equipment is too far away from the target, the tracking failure possibility exists due to the fact that the target cannot be effectively identified, and the practicability is poor, when the first distance information between the underwater equipment and the tracking target is judged to be larger than a preset threshold value, the binocular tracking mode is switched to the sound wave tracking mode, the second relative position information between the underwater equipment and the tracking target is calculated by transmitting an inquiry signal to the tracking target and receiving a corresponding response signal transmitted by the tracking target, tracking is carried out based on the second relative position information, distance tracking of the tracking target is achieved by combining binocular tracking and sound wave tracking, the safety of the target is further guaranteed, and the practicability is good.
The computer program product for performing the underwater target tracking method provided by the embodiment of the present invention includes a computer readable storage medium storing a program code, where instructions included in the program code may be used to execute the method described in the foregoing method embodiment, and specific implementation may refer to the method embodiment, and will not be described herein again.
The apparatus provided by the embodiment of the present invention may be specific hardware on the device, or software or firmware installed on the device, etc. The device provided by the embodiment of the present invention has the same implementation principle and technical effect as the method embodiments, and for the sake of brief description, reference may be made to the corresponding contents in the method embodiments without reference to the device embodiments. It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the foregoing systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and method may be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the division of the units is only one logical division, and there may be other divisions when actually implemented, and for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection of devices or units through some communication interfaces, and may be in an electrical, mechanical or other form.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments provided by the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.
The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus once an item is defined in one figure, it need not be further defined and explained in subsequent figures, and moreover, the terms "first", "second", "third", etc. are used merely to distinguish one description from another and are not to be construed as indicating or implying relative importance.
Finally, it should be noted that: the above-mentioned embodiments are only specific embodiments of the present invention, which are used for illustrating the technical solutions of the present invention and not for limiting the same, and the protection scope of the present invention is not limited thereto, although the present invention is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: any person skilled in the art can modify or easily conceive the technical solutions described in the foregoing embodiments or equivalent substitutes for some technical features within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the present invention in its spirit and scope. Are intended to be covered by the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (6)
1. A method of tracking an underwater target, comprising:
the method comprises the steps that underwater equipment obtains a tracking image comprising a tracking target, wherein the tracking image comprises a first image and a second image which are respectively collected by different cameras;
calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image, and tracking the tracking target based on the first relative position information;
judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not;
if yes, calculating second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal which is transmitted by the tracking target and corresponds to the inquiry signal, and tracking the tracking target based on the second relative position information;
wherein the calculating first relative position information between the underwater device and the tracking target according to the first image and the second image comprises:
calculating first distance information between the underwater equipment and the tracking target according to the first image and the second image;
traversing and searching the first image, and determining first coordinate position information of the tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
determining first orientation information of the tracking target relative to the underwater equipment according to the first coordinate position information or the second coordinate position information;
determining the first distance information and the first orientation information as first relative position information;
the step of performing traversal search on the first image to determine first coordinate position information of the tracking target in the first image includes:
determining the scale information of the tracking target in the current first image based on first distance information corresponding to a previous first image of the current first image, the scale information of the tracking target in the previous first image and the first distance information corresponding to the current first image, wherein the current first image and the previous first image are images acquired by the same camera;
and traversing and searching the current first image according to the determined scale information, and determining first coordinate position information of the tracking target in the current first image.
2. The method of claim 1,
the preset threshold is determined by the image attribute of the first image and/or the image attribute of the second image, wherein the image attribute of the first image at least comprises a first image definition, and the image attribute of the second image at least comprises a second image definition.
3. The method of claim 1, wherein the reply signal is plural; the calculating of the second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal corresponding to the inquiry signal transmitted by the tracking target comprises:
calculating a phase difference between any two of the plurality of reply signals;
determining second azimuth information between the tracking target and the underwater equipment according to the calculated phase difference;
calculating second distance information between the underwater equipment and the tracking target according to a time difference between a first time of transmitting the inquiry signal and a second time of receiving the corresponding response signal;
determining the second distance information and the second orientation information as second relative position information.
4. The method of claim 1, wherein the calculating first distance information between the underwater device and the tracking target from the first image and the second image comprises:
extracting first image feature information from the first image and extracting second image feature information from the second image;
taking the first image characteristic information as reference information, and performing stereo matching processing on all the second image characteristic information and any one of the reference information respectively to obtain a corresponding matching point relation;
according to the corresponding relation of the matching points, calculating parallax information between the first image and the second image;
and obtaining first distance information between the underwater equipment and the tracking target according to the parallax information.
5. An underwater apparatus, comprising:
the tracking system comprises an image acquisition module, a tracking module and a tracking module, wherein the image acquisition module is used for acquiring a tracking image comprising a tracking target, and the tracking image comprises a first image and a second image which are respectively acquired by adopting different cameras;
the first calculation tracking module is used for calculating first relative position information between the underwater equipment and the tracking target according to the first image and the second image and tracking the tracking target based on the first relative position information;
the judging module is used for judging whether first distance information between the underwater equipment corresponding to the first relative position information and the tracking target is larger than a preset threshold value or not;
the second calculation tracking module is used for calculating second relative position information between the underwater equipment and the tracking target by transmitting an inquiry signal to the tracking target and receiving a response signal which is transmitted by the tracking target and corresponds to the inquiry signal when the first distance information is judged to be larger than the preset threshold value, and tracking the tracking target based on the second relative position information;
wherein the first computation tracking module comprises:
a first distance calculation unit, configured to calculate first distance information between the underwater device and the tracking target according to the first image and the second image;
the traversal search unit is used for performing traversal search on the first image and determining first coordinate position information of the tracking target in the first image; or, performing traversal search on the second image, and determining second coordinate position information of the tracking target in the second image;
the first orientation determining unit is used for determining first orientation information of the tracking target relative to the underwater equipment according to the first coordinate position information or the second coordinate position information;
a first position determination unit configured to determine the first distance information and the first orientation information as first relative position information;
wherein, the traversal search unit comprises:
the scale confirmation subunit is configured to determine, based on first distance information corresponding to a previous first image of a current first image, scale information of a tracking target in the previous first image, and first distance information corresponding to the current first image, scale information of the tracking target in the current first image, where the current first image and the previous first image are images acquired by using the same camera;
and the traversal search subunit is configured to perform traversal search on the current first image according to the determined scale information, and determine first coordinate position information of the tracking target in the current first image.
6. The subsea equipment of claim 5, wherein the second computing tracking module comprises:
a calculation unit for receiving the plurality of reply signals and calculating a phase difference between any two of the plurality of reply signals;
a second azimuth determination unit configured to determine second azimuth information between the tracking target and the underwater device according to the calculated phase difference;
the second distance calculation unit is used for calculating second distance information between the underwater equipment and the tracking target according to a time difference between first time of transmitting the inquiry signal and second time of receiving the corresponding response signal;
a second position determination unit configured to determine the second distance information and the second orientation information as second relative position information.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710615572.4A CN107369178B (en) | 2017-07-26 | 2017-07-26 | Underwater target tracking method and underwater equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710615572.4A CN107369178B (en) | 2017-07-26 | 2017-07-26 | Underwater target tracking method and underwater equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107369178A CN107369178A (en) | 2017-11-21 |
| CN107369178B true CN107369178B (en) | 2020-06-26 |
Family
ID=60308478
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710615572.4A Active CN107369178B (en) | 2017-07-26 | 2017-07-26 | Underwater target tracking method and underwater equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107369178B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108415441B (en) * | 2018-03-05 | 2020-03-10 | 中国海洋大学 | Following method of underwater robot target following system based on monocular vision |
| CN110262565B (en) * | 2019-05-28 | 2023-03-21 | 深圳市吉影科技有限公司 | Target tracking motion control method and device applied to underwater six-push unmanned aerial vehicle |
| CN110490909B (en) * | 2019-08-27 | 2020-10-16 | 吉林大学 | AUV (autonomous Underwater vehicle) and target tracking method, device and medium |
| CN117590824B (en) * | 2024-01-19 | 2024-04-02 | 杭州市水务集团有限公司 | Sand setting tank control method and system in water purification process |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202770991U (en) * | 2012-09-10 | 2013-03-06 | 北京引创科技有限责任公司 | Large-scale underwater hoisting structure position posture real-time measurement apparatus |
| CN106127117A (en) * | 2016-06-16 | 2016-11-16 | 哈尔滨工程大学 | Based on binocular vision quick high robust identification, location automatically follow luggage case |
| CN206249096U (en) * | 2016-12-06 | 2017-06-13 | 北京臻迪科技股份有限公司 | A kind of underwater robot |
| CN106891341A (en) * | 2016-12-06 | 2017-06-27 | 北京臻迪科技股份有限公司 | A kind of underwater robot and catching method |
| CN106970618A (en) * | 2017-04-06 | 2017-07-21 | 北京臻迪科技股份有限公司 | A kind of unmanned boat control method and system |
| CN109188421A (en) * | 2018-07-25 | 2019-01-11 | 江苏科技大学 | A kind of maritime search and rescue system and method for unmanned rescue boat |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP5766641B2 (en) * | 2012-03-22 | 2015-08-19 | 株式会社東芝 | Tracking device |
-
2017
- 2017-07-26 CN CN201710615572.4A patent/CN107369178B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN202770991U (en) * | 2012-09-10 | 2013-03-06 | 北京引创科技有限责任公司 | Large-scale underwater hoisting structure position posture real-time measurement apparatus |
| CN106127117A (en) * | 2016-06-16 | 2016-11-16 | 哈尔滨工程大学 | Based on binocular vision quick high robust identification, location automatically follow luggage case |
| CN206249096U (en) * | 2016-12-06 | 2017-06-13 | 北京臻迪科技股份有限公司 | A kind of underwater robot |
| CN106891341A (en) * | 2016-12-06 | 2017-06-27 | 北京臻迪科技股份有限公司 | A kind of underwater robot and catching method |
| CN106970618A (en) * | 2017-04-06 | 2017-07-21 | 北京臻迪科技股份有限公司 | A kind of unmanned boat control method and system |
| CN109188421A (en) * | 2018-07-25 | 2019-01-11 | 江苏科技大学 | A kind of maritime search and rescue system and method for unmanned rescue boat |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107369178A (en) | 2017-11-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107369178B (en) | Underwater target tracking method and underwater equipment | |
| CN108352056B (en) | System and method for correcting erroneous depth information | |
| CN107564033B (en) | Underwater target tracking method, underwater equipment and wearable equipment | |
| EP2884460B1 (en) | Image capturing apparatus, monitoring system, image processing apparatus, image capturing method, and non-transitory computer readable recording medium | |
| US6847392B1 (en) | Three-dimensional structure estimation apparatus | |
| CN107980138B (en) | False alarm obstacle detection method and device | |
| US20150035820A1 (en) | 3d modeling motion parameters | |
| KR101544021B1 (en) | Apparatus and method for generating 3d map | |
| US8531505B2 (en) | Imaging parameter acquisition apparatus, imaging parameter acquisition method and storage medium | |
| JP2019190974A (en) | Calibration device, calibration method and program | |
| CN108510528B (en) | Method and device for registration and fusion of visible light and infrared image | |
| US9628777B2 (en) | Method of 3D reconstruction of a scene calling upon asynchronous sensors | |
| CN112106111A (en) | Calibration method, calibration equipment, movable platform and storage medium | |
| CN110751685A (en) | Depth information determination method, determination device, electronic device and vehicle | |
| CN112837207A (en) | Panoramic depth measuring method, four-eye fisheye camera and binocular fisheye camera | |
| TWI528783B (en) | Methods and systems for generating depth images and related computer products | |
| JP5445069B2 (en) | 3D position measuring apparatus and 3D position measuring method | |
| EP3819815A1 (en) | Human body recognition method and device, as well as storage medium | |
| CN112513766A (en) | Method, tracking device, storage medium and computer program product for path planning | |
| CN111583338B (en) | Positioning method and device for unmanned equipment, medium and unmanned equipment | |
| US9245343B1 (en) | Real-time image geo-registration processing | |
| JP2006309450A (en) | Image recognition device and image recognition method | |
| US20140307053A1 (en) | Method of prompting proper rotation angle for image depth establishing | |
| US20210402616A1 (en) | Information processing apparatus, information processing method, mobile robot, and non-transitory computer-readable storage medium | |
| EP3522520B1 (en) | Image processing method, electronic device, and non-transitory computer readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CP03 | Change of name, title or address | ||
| CP03 | Change of name, title or address |
Address after: 264200 Zone E, blue venture Valley, No. 40, Yangguang Road, Nanhai new area, Weihai City, Shandong Province Patentee after: Zhendi Technology Co., Ltd Address before: Unit 301, unit a, 9 Fulin Road, Chaoyang District, Beijing 100107 Patentee before: POWERVISION TECH Inc. |