CN114248893A - An operational underwater robot for sea cucumber fishing and its control method - Google Patents

Abstract

The invention relates to an operation type underwater robot for sea cucumber catching and a control method thereof, belonging to the field of underwater robots. In the grabbing process, a sea cucumber recognition and tracking algorithm based on MobileNet-transform-GCN is used for recognizing and continuously tracking sea cucumbers to be caught, meanwhile, the sea cucumbers to be caught are positioned in real time, a path from an operation type underwater robot to a target point is planned by adopting a rapid search tree algorithm, the operation type underwater robot is controlled to move according to the path based on an Actor-Critic reinforcement learning model, and accurate control and autonomous grabbing of the sea cucumber catching robot in a complex underwater environment are achieved.

Description

An operational underwater robot for sea cucumber fishing and its control method

technical field

The invention relates to the field of underwater robots, in particular to an operation-type underwater robot for sea cucumber fishing and a control method thereof.

Background technique

With the increasing demand for high-quality seafood such as sea cucumbers, scallops, and sea urchins, artificial fishing is not only slow and low in yield, but also has a high cost and a high risk factor. Therefore, the technology of catching seafood through intelligent automatic equipment instead of humans has received extensive attention. Automated and intelligent fishing equipment can greatly improve the efficiency of sea cucumber fishing, reduce the dependence on labor, maximize the output of sea cucumbers, reduce the risk of sea cucumber fishing, and reduce unnecessary losses. Operational underwater robot is a kind of intelligent equipment for sea cucumber fishing. Through an underwater robot equipped with a robotic arm, combined with target recognition, detection, tracking and corresponding stable control algorithms, the robot can automatically grasp sea cucumbers and greatly reduce the difficulty of sea cucumber fishing. However, the design and control of underwater robots are very complex. On the one hand, the underwater environment is very complex, which is not only affected by its own buoyancy and gravity, but also affected by ocean currents and tides. These factors will affect the balance and stability of underwater robots. Stability increases the difficulty of control. On the other hand, the underwater environment is poor, the brightness is low, the red light decays quickly, and the color shift is serious, so the recognition degree and visible range of the underwater environment are low, which also increases the recognition Sea cucumbers and the difficulty of fishing.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide an operational underwater robot for sea cucumber fishing and a control method thereof, so as to realize precise control and autonomous grasping of the sea cucumber fishing robot in a complex underwater environment.

For achieving the above object, the present invention provides the following scheme:

An operation type underwater robot for sea cucumber fishing, the operation type underwater robot comprises: a body frame, a propeller, a second control board, a first control board, a camera mechanism and a grabbing mechanism;

The propeller, the second control board, the first control board and the camera mechanism are all arranged on the body frame, and the fixed end of the grabbing mechanism is connected with the body frame;

The camera mechanism is connected with the signal input end of the second control board, and the signal output end of the second control board is respectively connected with the first control board and the grabbing mechanism; the camera mechanism is used for photographing the forward vision of the operation-type underwater robot image and the lower visual image, and transmit both the forward visual image and the lower visual image to the second control board; the second control board is used to determine the three-dimensional position information of the sea cucumber according to the lower visual image, and according to the three-dimensional position of the sea cucumber Information and forward visual images for online path planning;

The first control board is connected to the propeller, and the first control board is used to adjust the propeller according to the online path plan, so that the operation-type underwater robot moves according to the online path plan;

The second control board is further configured to control the grasping mechanism to fish for sea cucumbers when the operation-type underwater robot reaches the target point of the online path planning.

Optionally, the camera mechanism includes: a front-view monocular camera and a top-down binocular camera;

The forward-looking monocular camera and the top-view binocular camera are both connected to the signal input end of the second control board; the forward-looking monocular camera is used to capture the forward visual image of the operation-type underwater robot, and the forward visual The image is transmitted to the second control board; the top-view binocular camera is used to capture the lower visual image of the operation-type underwater robot, and transmit the lower visual image to the second control board.

Optionally, the propellers include: a first propeller, a second propeller, a third propeller, a fourth propeller, a fifth propeller, a sixth propeller, and a seventh propeller and the eighth propeller;

The first propeller, the second propeller, the third propeller and the fourth propeller are arranged in the horizontal direction as vector type propellers, and the first propeller and the second propeller are arranged on the body frame The front of the horizontal direction, the third propeller and the fourth propeller are arranged at the rear of the body frame in the horizontal direction;

The first propeller, the second propeller, the third propeller, the fourth propeller, the fifth propeller, the sixth propeller, the seventh propeller and the eighth propeller are all the same as the first propeller. a control panel connection;

The first control board is used to control the first propeller, the second propeller, the third propeller and the fourth propeller to provide thrust in the horizontal front and rear directions for the operation-type underwater robot, and to control the The fifth propeller, the sixth propeller, the seventh propeller and the eighth propeller provide the operation-type underwater robot with thrust in the vertical direction, so that the operation-type underwater robot can follow the online Path planning for movement.

Optionally, the grabbing mechanism includes: a base, a shoulder joint, an elbow joint, a forearm joint, a gripper, a first steering gear, a second steering gear, a third steering gear, and a fourth steering gear;

The first steering gear, the second steering gear, the third steering gear and the fourth steering gear are all connected to the second control board;

The shoulder joint is fixed on the body frame through the base; the base is equipped with a first steering gear, and the first steering gear is used to drive the shoulder joint to rotate around the vertical axis direction under the control of the second control board;

The elbow joint is connected in series with the shoulder joint, and a second steering gear is installed between the shoulder joint and the elbow joint, and the second steering gear is used to drive the elbow joint around the axis perpendicular to the shoulder joint under the control of the second control board. direction rotation;

The forearm joint is connected in series with the elbow joint, and a third steering gear is installed between the forearm joint and the elbow joint, and the third steering gear is used to drive the forearm joint to rotate around the elbow perpendicular to the elbow Rotation in the direction of the joint centerline;

The clamping jaw and the fourth steering gear are mounted above the forearm joint; the clamping jaw includes two oppositely arranged mesh structures; the fourth steering gear is used to drive the two opposing gears under the control of the second control board The set mesh structure rotates in the opposite direction to realize the opening and closing control of the clamping jaws.

Optionally, the operation-type underwater robot further includes: a loading cage and a fifth steering gear;

The loading net box is arranged on the body frame, and the loading net box is arranged opposite to the grabbing mechanism;

The loading cage is connected to the second control board through the fifth steering gear, and the loading cage is used to pass the second control board under the control of the second control board when the grabbing mechanism moves to a preset position after grabbing. Five steering gears drive the loading cage to automatically open, and load the sea cucumbers caught by the grabbing mechanism.

A control method of an operation-type underwater robot for sea cucumber fishing, the control method comprising:

Real-time acquisition of the forward visual image and the lower visual image of the operational underwater robot captured by the camera mechanism;

According to the lower visual image acquired in real time, the sea cucumber identification and tracking algorithm based on MobileNet-Transformer-GCN is used to identify and continuously track the sea cucumbers to be caught, and at the same time locate the pixel coordinates of the sea cucumbers to be caught in real time;

Convert the pixel coordinates into world coordinates through binocular stereo matching to obtain the three-dimensional position of the sea cucumber to be caught;

Set the three-dimensional position of the sea cucumber to be fished as the target point, and use the fast search tree algorithm to plan the path between the operational underwater robot and the target point;

According to the forward visual image, the operation-type underwater robot is controlled based on the Actor-Critic reinforcement learning model to move according to the path, and the operation-type underwater robot suspends after running to the target point; the Actor-Critic reinforcement learning model passes the Gaussian mixture The model performs clustering compression on the sample space;

According to inverse kinematics, the sea cucumber to be caught is grabbed by the grabbing mechanism of the operational underwater robot.

Optionally, the real-time acquisition of the front visual image and the bottom visual image of the operation-type underwater robot photographed by the camera mechanism further includes:

The underwater image enhancement algorithm based on the Siamese convolutional neural network is used to perform color correction and dehazing enhancement on the front visual image and the lower visual image; the Siamese convolutional neural network includes a first branch convolutional neural network and a second branch convolutional neural network. neural network; the first branch convolutional neural network is constrained by the color feature of the label image and is responsible for the color correction of the image; the second branch convolutional neural network is constrained by the texture feature and is responsible for the clarity of the image; the The first branch convolutional neural network and the second branch convolutional neural network perform convolution feature transformation operations after being constrained by features respectively, and finally the two branch features are spliced by point multiplication, and after splicing, a layer of convolution is performed. The product transform produces the final sharpened image.

Optionally, according to the lower visual image obtained in real time, the sea cucumber identification and tracking algorithm based on MobileNet-Transformer-GCN is used to identify and continuously track the sea cucumber to be fished, and simultaneously locate the pixel coordinates of the sea cucumber to be fished in real time, specifically including:

Zoom the lower visual image obtained in real time to obtain the zoomed lower visual image;

Input the zoomed lower visual image into the first lightweight module, the second lightweight module, the third lightweight module, the first Transformer-GCN module, the fourth lightweight module, the second Transformer-GCN module, and the fifth lightweight module. Quantization module and global pooling module, output feature map;

Mapping the feature map to obtain a prediction result of the sea cucumber to be caught; the prediction result includes a target location, a target category and a confidence level;

Inputting the feature map into a fully connected module to obtain a deep identity feature;

Extract gradient histogram features from the scaled lower visual image as artificial identity features;

Using principal component analysis to map artificial identity features to the same dimensions as deep identity features;

Fusing the mapped artificial identity features and deep identity features to obtain the fused identity features;

Input the fused identity feature into the filtering module, and calculate the response value of each detection target in the zoomed lower visual image;

The detection target with the largest response value in the zoomed lower visual image is selected and determined as the currently tracked sea cucumber to be caught.

Optionally, converting the pixel coordinates into world coordinates through binocular stereo matching to obtain the three-dimensional position of the sea cucumber to be fished specifically includes:

The camera is calibrated by Zhang Zhengyou's calibration method, and the internal parameter matrix and distortion coefficient of the camera are obtained;

Using the internal parameter matrix to convert the lower visual image pixels to the camera coordinate system;

In the camera coordinate system, the lower visual image pixels are converted by the distortion coefficient, and the converted lower visual image pixels are converted to the pixel coordinate system;

Use the formula D = fT / d to calculate the distance D from a point in the space to the camera plane; where f is the focal length obtained by calibration, T is the distance between the two cameras in the binocular camera, and d is the parallax value;

，将所述像素坐标转换为世界坐标，获得待捕捞海参的三维位置；其中，(X,Y,Z)为待捕捞海参的三维位置坐标，(x ₁,y ₁)和(x ₂,y ₂)分别为双目视觉中待捕捞海参在两个相机拍摄的图像中的像素坐标。According to the distance from a point in space to the camera plane, use the formula

, convert the pixel coordinates into world coordinates to obtain the three-dimensional position of the sea cucumber to be fished; wherein ( X , Y , Z ) are the three-dimensional position coordinates of the sea cucumber to be fished, ( x ₁ , y ₁ ) and ( x ₂ , y ) ₂ ) are respectively the pixel coordinates of the sea cucumber to be caught in the images captured by the two cameras in binocular vision.

Optionally, the operation-type underwater robot is controlled based on the Actor-Critic reinforcement learning model to move according to the path, and the operation-type underwater robot suspends after running to the target point, specifically including:

Obtain the current motion posture of the operational underwater robot, and transmit the current motion posture to the online strategy network in the action network; the current motion posture includes yaw angle, pitch angle, roll angle, three-dimensional coordinates, angular velocity and line speed;

Calculate the current reward value according to the reward function of the Actor-Critic reinforcement learning model and the path; the reward function is R = r ₀ - ρ ₁ ||Δφ,Δ θ ,Δ ψ ||- ρ ₂ ||Δ x ,Δ y ,Δ z || ₂ ; among them, Δ x ,Δ y ,Δ z are three-dimensional coordinate state quantities, Δφ,Δ θ ,Δ ψ are yaw angle state quantities, pitch angle state quantities, roll angle state quantities, respectively , r ₀ is the reward constant, ρ ₁ ||Δφ,Δ θ ,Δ ψ || is the second norm of the relative orientation error, ρ ₂ ||Δ x ,Δ y ,Δ z || ₂ is the second norm of the relative position error norm, ρ ₁ and ρ ₂ are the first coefficient and the second coefficient, respectively;

The current reward value and the state function are fused into a training sample and added to the sample space; the state function is s =[ g ,Δ x ,Δ y ,Δ z ,Δφ,Δ θ ,Δ ψ , u , v , w , p , q , r ]; among them, g is the state constant, u , v , w are the linear velocity state quantities, p , q , r are the angular velocity state quantities, and s is the state function;

；其中，P(x)为压缩后的样本空间，K为压缩前的样本空间中样本的类别，

是类分布概率，

与σ _k 分别为类均值与类方差，（R _i ,s _i ）为压缩前的样本空间中的第i个样本，N为子模型的高斯分布密度函数；The samples in the sample space are fused by the Gaussian mixture model to compress the sample space; the Gaussian mixture model is

; Among them, P ( x ) is the sample space after compression, K is the category of the sample in the sample space before compression,

is the class distribution probability,

and σ _k are the class mean and class variance, respectively, ( R _i , s _i ) is the ith sample in the sample space before compression, and N is the Gaussian distribution density function of the sub-model;

Input the samples in the compressed sample space into the target-action network in the evaluation network, perform gradient calculation, and update the parameters of the online state-action network;

The online strategy network in the action network is optimized and updated by the gradient calculated by the evaluation network, and after accumulating multiple gradients, the parameters of the target strategy network are updated through the gradient of the online strategy network;

Generate a new state function through the online policy network in the action network for the control of the manipulator and the propeller;

The above steps are repeated until convergence, so that the motion result of the operation-type underwater robot conforms to the path.

According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

The invention discloses an operation-type underwater robot facing sea cucumber fishing and a control method thereof. A camera mechanism captures a front visual image and a lower visual image of the operation-type underwater robot, and a second control board determines three-dimensional position information of the sea cucumber according to the lower visual image. , and carry out online path planning according to the three-dimensional position information of the sea cucumber and the forward visual image. The first control board adjusts the propeller according to the online path planning, so that the operation-type underwater robot moves according to the online path planning, and the second control board is in the operation-type underwater vehicle. When the robot reaches the target point of the online path planning, it controls the grasping mechanism to fish the sea cucumber. In the grabbing process, the sea cucumber identification and tracking algorithm based on MobileNet-Transformer-GCN is used to identify and continuously track the sea cucumbers to be caught, and at the same time locate the pixel coordinates of the sea cucumbers to be caught in real time, and use the fast search tree algorithm to plan the operation-type underwater robot to the target point. The path between the two is based on the Actor-Critic reinforcement learning model to control the operation-type underwater robot to move according to the described path, and according to inverse kinematics, the grasping mechanism of the operation-type underwater robot grabs the sea cucumbers to be caught, and realizes complex underwater Precise control and autonomous grasping of sea cucumber fishing robots in the environment.

Description of drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings required in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some of the present invention. In the embodiments, for those of ordinary skill in the art, other drawings can also be obtained according to these drawings without creative labor.

Fig. 1 is the front view of the operation type underwater robot that faces sea cucumber fishing provided by the present invention;

Fig. 2 is the rear view of the operation-type underwater robot facing sea cucumber fishing provided by the present invention;

Fig. 3 is the bottom view of the operation-type underwater robot facing sea cucumber fishing provided by the present invention;

4 is a side view of an operation-type underwater robot facing sea cucumber fishing provided by the present invention;

5 is a schematic structural diagram of a grabbing mechanism provided by the present invention;

6 is a frame diagram of a control method of an operation-type underwater robot for sea cucumber fishing provided by the present invention;

7 is a schematic diagram of an underwater image enhancement algorithm provided by the present invention;

8 is a structural block diagram of a sea cucumber identification and tracking algorithm provided by the present invention;

9 is a schematic structural diagram of a lightweight module provided by the present invention;

10 is a schematic structural diagram of Transformer-GCN provided by the present invention;

11 is a schematic diagram of a sea cucumber identification and tracking algorithm provided by the present invention;

FIG. 12 is a schematic diagram of the Actor-Critic reinforcement learning model provided by the present invention.

Symbol description: 1-first control cabin, 2-power compartment, 31-first propeller, 32-second propeller, 33-third propeller, 34-fourth propeller, 41-th Five propellers, 42-sixth propeller, 43-seventh propeller, 44-eighth propeller, 5-overhead binocular camera, 6-second control cabin, 7-loading cage, 81 - jaws, 82- forearm joint, 83- elbow joint, 84- shoulder joint, 85- base, 9- body frame.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

The purpose of the present invention is to provide an operational underwater robot for sea cucumber fishing and a control method thereof, so as to realize precise control and autonomous grasping of the sea cucumber fishing robot in a complex underwater environment.

In order to make the above objects, features and advantages of the present invention more clearly understood, the present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

The present invention provides an operation-type underwater robot for sea cucumber fishing. As shown in Figures 1-4, the operation-type underwater robot includes: a body frame 9, a propeller, a second control board, a first control board, and a camera mechanism and grabbing mechanism.

The pusher, the second control board, the first control board and the camera mechanism are all arranged on the main body frame 9 , and the fixed end of the grabbing mechanism is connected to the main body frame 9 . The camera mechanism is connected with the signal input end of the second control board, and the signal output end of the second control board is connected with the first control board and the grasping mechanism respectively; image, and transmit both the forward visual image and the lower visual image to the second control board; the second control board is used to determine the 3D position information of the sea cucumber according to the lower visual image, and carry out the online path according to the 3D position information of the sea cucumber and the forward visual image planning. The first control board is connected with the propeller, and the first control board is used for adjusting the propeller according to the online path planning, so that the operation-type underwater robot moves according to the online path planning. The second control board is also used for controlling the grasping mechanism to fish the sea cucumber when the operation-type underwater robot reaches the target point of the online path planning.

As shown in FIG. 5 , the camera mechanism includes: a front-view monocular camera and a top-down binocular camera 5 . The forward-looking monocular camera and the top-down binocular camera 5 are both connected to the signal input end of the second control board; the forward-looking monocular camera is used to capture the forward visual image of the operation-type underwater robot, and transmit the forward visual image to the second control board. Control board; the top-view binocular camera 5 is used to capture the lower visual image of the operation-type underwater robot, and transmit the lower visual image to the second control board.

The front-view monocular high-definition camera is fixed on the pan/tilt in the first control cabin 1, and its viewing angle can be adjusted by the steering gear, which is used to obtain the vision information in front of the movement of the operational sea cucumber fishing robot, so as to realize the tracking and avoidance of the robot. obstacles and other tasks. The top-view binocular camera 5 is fixed in the middle of the robot frame, and is connected to the first control compartment through a watertight connection line. The visual information processing module in the first control cabin can be used for the processing and transmission of the video information obtained by the binocular camera and the monocular camera. In addition, the first control cabin 1 also includes a nine-axis inertial sensor and a depth sensor for acquiring robot attitude information and depth information.

The second control board is located in the second control compartment. The second control board has a graphics processing unit, a sensor processor, and a network communication module. It has low power consumption and a maximum power consumption of 30 watts. It can quantify and accelerate the deep network model and can run Target recognition, target detection and tracking algorithms can identify obstacles in the front-view field of view and targets in the top-down field of view in real time, and can calculate the three-dimensional position information and attitude information of the target through the binocular camera. The second control board can also run an autonomous control algorithm and a navigation algorithm to realize the autonomous control of the sea cucumber fishing robot.

Below the first control cabin is the power supply compartment 2 of the underwater robot for sea cucumber fishing, which provides the necessary power for autonomous control of the robot; below the power supply compartment 2 is the binocular camera in the robot's vision system, which is used for grasping by the robotic arm. Take the time to obtain the three-dimensional position information and attitude information of the target. The lower part behind the frame is a second control cabin 6 , a second control board is arranged in the second control cabin 6 , and the second control cabin 6 and the first control cabin 1 are connected by a watertight connection line.

The propellers include: a first propeller propeller 31, a second propeller propeller 32, a third propeller propeller 33, a fourth propeller propeller 34, a fifth propeller propeller 41, a sixth propeller propeller 42, and a seventh propeller propeller 43 and the eighth propeller 44. The first propeller 31, the second propeller 32, the third propeller 33 and the fourth propeller 34 are arranged in a vector type in the horizontal direction, the first propeller 31 and the second propeller 32 The third propeller 33 and the fourth propeller 34 are arranged at the rear of the body frame 9 in the horizontal direction. The first propeller 31, the second propeller 32, the third propeller 33, the fourth propeller 34, the fifth propeller 41, the sixth propeller 42, the seventh propeller 43 and the third propeller The eight propellers 44 are all connected to the first control board. The first control board is used to control the first propeller 31, the second propeller 32, the third propeller 33 and the fourth propeller 34 to provide thrust in the horizontal front and rear directions for the operation-type underwater robot, and to control the The fifth propeller 41, the sixth propeller 42, the seventh propeller 43 and the eighth propeller 44 provide vertical thrust for the operation-type underwater robot, so that the operation-type underwater robot can be planned according to the online path Exercise.

The first propeller 31 to the eighth propeller 44 are all six-degree-of-freedom vector propellers, which are used to provide power for the sea cucumber fishing robot to drive the robot's motion and precise attitude control. The first propeller 31 to the eighth propeller 44 are connected to the first control compartment through a watertight connection line, and the control compartment can drive the propeller to rotate through a driving signal to control the movement direction of the robot.

The grasping mechanism includes: a base 85, a shoulder joint 84, an elbow joint 83, a forearm joint 82, a clamping jaw 81, a first steering gear, a second steering gear, a third steering gear and a fourth steering gear. The first steering gear, the second steering gear, the third steering gear and the fourth steering gear are all connected with the second control board. The shoulder joint 84 is fixed to the body frame 9 through the base 85; the base 85 is equipped with a first steering gear, and the first steering gear is used to drive the shoulder joint 84 to rotate around the vertical axis direction under the control of the second control board. The elbow joint 83 is connected to the shoulder joint 84 in series, a second steering gear is installed between the shoulder joint 84 and the elbow joint 83, and the second steering gear is used to drive the elbow joint 83 to be perpendicular to the shoulder joint 84 under the control of the second control board. Rotation in the direction of the axis. The forearm joint 82 is connected in series with the elbow joint 83, and a third steering gear is installed between the forearm joint 82 and the elbow joint 83, and the third steering gear is used to drive the forearm joint 82 to rotate perpendicular to The elbow joint 83 rotates in the direction of the center line. The clamping jaw 81 and the fourth steering gear are installed above the forearm joint 82; the clamping jaw 81 includes two oppositely arranged mesh structures; the fourth steering gear is used to drive the two oppositely arranged meshes under the control of the second control board The like structure rotates in the opposite direction to realize the opening and closing control of the clamping jaws 81 .

The clamping jaw 81 is a sea cucumber-shaped strip-shaped mesh structure, and the structure is a flexible structure, and a silicone sleeve is covered on the outer layer of the plastic frame, which is suitable for the fishing of sea cucumbers.

The grasping mechanism is a three-degree-of-freedom mechanical arm, which includes a series motor drive structure, a mechanical rigid body transmission structure and a gripper 81. The drive structure drives the gripper 81 through the drive structure to grasp the sea cucumber, and the series motor drive structure is driven by the first rudder. The mechanical rigid body transmission structure is composed of a base 85 , a shoulder joint 84 , an elbow joint 83 and a forearm joint 82 . All drive servos are connected to the second control compartment through watertight cables, and a control board is installed in the compartment to drive the robotic arm servos to rotate.

The operation-type underwater robot also includes: loading the cage 7 and the fifth steering gear. The loading net box 7 is arranged on the main body frame 9, and the loading net box 7 is arranged opposite to the grabbing mechanism. The loading cage 7 is connected to the second control board through the fifth steering gear, and the loading cage 7 is used to be driven by the fifth steering gear under the control of the second control board when the grasping mechanism ends and moves to the preset position. The loading cage 7 is automatically opened, and the sea cucumbers caught by the grasping mechanism are loaded, so as to facilitate the recovery of the grasped sea cucumbers.

Preferably, the loading cage 7 is made of acrylic material. The loading cage 7 is fixed under the front of the robot.

The operation-type underwater robot also includes: a low-power searchlight, a front bracket fixed on the body frame 9, and a lower part of the bracket. It is used to illuminate the front under darker underwater conditions and expand the field of vision of the sea cucumber fishing robot. The second control board can judge the change of brightness in the environment where the robot is located by the obtained visual information. When the brightness is lower than the threshold, the lighting system is automatically turned on to fill in the light, which expands the robot's ability to operate in dark underwater scenes. view range.

The second control board executes underwater image enhancement algorithm based on twin network; sea cucumber identification and detection algorithm and sea cucumber target tracking and positioning algorithm based on MobileNet-Transformer-GCN; underwater robot online motion planning and coordinated control algorithm; sea cucumber tracking control and suspension grasping Taking the algorithm, the autonomous control can be realized without the upper computer.

Based on the aforementioned operation-type underwater robot for sea cucumber fishing, the present invention also provides a control method for the operation-type underwater robot for sea cucumber fishing, as shown in FIG. 6 , the control method includes:

In step 1, the forward visual image and the lower visual image of the operation-type underwater robot captured by the camera mechanism are acquired in real time.

Visual images of the workspace in front of and below the robot are obtained through binocular and monocular cameras. The monocular camera is used to guide the underwater robot to move from the current position to the target position of the sea cucumber, which is realized by the target detection and tracking algorithm, and the movement process is realized by the planning algorithm and control algorithm; the binocular camera is used for the robot to reach the target position after the The grasping control is realized by target detection and tracking and positioning, and the grasping process is calculated by inverse kinematics.

After step 1 also include:

The underwater image enhancement algorithm based on the Siamese convolutional neural network is used to perform color correction and dehazing enhancement on the forward vision image and the lower vision image. As shown in Figure 7, the twin convolutional neural network includes a first branch convolutional neural network and a second branch convolutional neural network; the first branch convolutional neural network is constrained by the color feature of the label image, and is responsible for the color correction of the image; The second branch convolutional neural network is constrained by texture features and is responsible for the clarity of the image; the first branch convolutional neural network and the second branch convolutional neural network are respectively constrained by the features to perform convolution feature transformation operations, and finally pass the point The two branch features are spliced by multiplication, and after splicing, a layer of convolution transformation is performed to generate the final clear image.

The color features used are the first-order moment of color, the second-order moment of color and the third-order moment of color, and the extraction formula is specifically expressed as:

The texture features used are those extracted by the local binary pattern operator. The feature extraction is performed by convolution extraction through the operator template. Taking the 3*3 convolution operator as an example, the eight surrounding pixel values are compared with the central pixel, and the value greater than the central pixel value is 1, and the value is less than the central pixel value. is 0. After the two branches are constrained by the two features, the convolution feature transformation operation is further performed, and finally the two branch features are spliced by point multiplication. After splicing, a layer of convolution transformation is performed to generate the final clear image.

The designed underwater image enhancement algorithm can be adapted to the situation where the sea cucumber has a large difference in the background color in the underwater environment. In this case, the edge features of the sea cucumber are obvious, and better texture feature extraction can be performed.

Step 2: According to the real-time acquired visual image below, the sea cucumber identification and tracking algorithm based on MobileNet-Transformer-GCN is used to identify and continuously track the sea cucumbers to be fished, and simultaneously locate the pixel coordinates of the sea cucumbers to be fished in real time.

The algorithm takes the underwater images collected by binocular and monocular cameras as input, and detects and outputs the position information of sea cucumbers in real time. The designed sea cucumber detection and sea cucumber target tracking algorithm is a single-step algorithm, which can detect and track at the same time, and can be better applied to the robot platform. Pick.

The designed sea cucumber detection and sea cucumber target tracking algorithm is a single-step algorithm, which can detect and track at the same time, and can be better applied to the robot platform. Pick. The detection and recognition algorithm involved combines the characteristics of MobileNet, Transformer and GCN. The head part is first extracted through the convolution layer. The convolution used is decomposed convolution. The decomposed convolution is divided into two steps, depth convolution and point convolution. Convolution, the calculation amount of the model can be reduced by decomposing the convolution, and the lightweight model makes it suitable for underwater operation robots. In the deep convolution part, the ordinary convolution is replaced by the expanded convolution, and the expanded convolution can further make the model lightweight. , which can not only obtain a larger receptive field, improve the detection accuracy, but also reduce the amount of calculation; by decomposing the convolution, the image features can be mapped from low-dimensional to high-dimensional, input into the Transformer model, and further feature extraction, in the Transformer Instead of using a fully connected structure, the GCN structure is used. While extracting features, the weights of the edges of the graph are trained, the relationship between different features is learned, and the induction bias is increased; at the end of the model, it is divided into two parts. Branches, one branch is the detection result, the other branch is the extraction of identity features, and the identity features can be used for target tracking. In the tracking model, for the problem that the similarity between underwater sea cucumbers is large and the extraction of identity features is difficult, artificial features are fused. For the extraction of identity features, for the problem that the similarity is large and the model deviation is easy to increase, the method of simulated annealing and backtracking is used for correction, and the tracking speed is improved through correlation filtering, making it suitable for sea cucumber fishing robots.

Detection, recognition and tracking algorithms are used to obtain the coordinate position of the sea cucumber within the vision range of the robot, that is, the pixel coordinates. The designed detection and recognition algorithm integrates the lightweight module and the Transformer-GCN module.

The designed algorithm consists of a lightweight module and a Transformer-GCN module.

×C _e 维度的特征图；最后经过1×1×C _p 维度的卷积核卷积计算，输出

维度的特征图作为这一模块的输出特征图。Referring to Figure 9, each lightweight module consists of three convolution kernels. The input of n × n × C _i dimension is first subjected to the convolution calculation of the convolution kernel of 1 × 1 × C _e dimension, and becomes n × n × C The feature map of the _e dimension; then through the 3 × 3 × C _e dimension convolution kernel convolution calculation, it becomes

The feature map of × C _e dimension; finally, the convolution kernel of 1 × 1 × C _p dimension is calculated by convolution, and the output

The dimension feature map is used as the output feature map of this module.

×C _i 维度的特征图，此特征图标记为T ₁；然后再经过1×1×C _t 维度卷积核卷积计算，变为

×C _t 维度的特征图；再次将特征图拉伸展开变为

×C _t 维度的特征图，此时特征集合可以表示为F=[f ₀,f ₁,• • •,f _Ct ]，每个子特征f _i 的维度为

；针对特征集合构建强连通图，连通图边向量表示为W通过余弦相似度计算，既

，其中iϵ(1,• • •,C _t )，jϵ(1,• • •,C _t )，此时特征图可以转化为图卷积特征，既G=(F,W)，利用卷积核通过图卷积计算，既G _e =G*θ ₁，可以得到该层图卷积网络特征，其中θ ₁为卷积核，此时图卷积特征为F '=[f ₀',f ₁',• • •,f _Ct ']，每个子特征f _i '维度为n _g ；再利用图卷积核θ ₂计算下一层图卷积特征，此时图卷积特征为F ''=[f ₀'',f ₁'',• • •,f _Ct '']，每个子特征f _i ''维度为

；将图卷积特征拼接为

×C _t 维度的特征图；再经过1×1×C _i 维度的卷积核卷积计算，输出

×C _i 维度的特征图，此特征图标记为T ₂；最后再将T ₁与T ₂进行拼接作为模块的最后输出，拼接后特征图维度为

×2C _i 。Referring to Figure 10, the input of the Transformer-GCN module n × n × C _i dimension is first calculated by the convolution kernel of the 3 × 3 × C _i dimension, and becomes

× C _i dimension feature map, this feature map is marked as T ₁ ; then after 1 × 1 × C _t dimension convolution kernel convolution calculation, it becomes

The feature map of × C _t dimension; the feature map is stretched and expanded again to become

× C _t dimension feature map, the feature set can be expressed as F =[ f ₀ , f ₁ ,• • •, f _Ct ], and the dimension of each sub-feature f _i is

; Construct a strongly connected graph for the feature set, and the edge vector of the connected graph is expressed as W calculated by cosine similarity, both

, where i ϵ(1,• • •, C _t ), j ϵ(1,• • •, C _t ), the feature map can be converted into a graph convolution feature, that is, G = ( F , W ), using The convolution kernel is calculated by graph convolution, that is, _Ge = G *θ ₁ , the graph convolution network feature of this layer can be obtained, where θ ₁ is the convolution kernel, and the graph convolution feature is F '=[ f ₀ ' , f ₁ ',• • •, f _Ct '], the dimension of each sub-feature f _i ' is n _g ; then the graph convolution kernel θ ₂ is used to calculate the graph convolution feature of the next layer, at this time, the graph convolution feature is F ''=[ f ₀ '', f ₁ '',• • •, f _Ct ''], the dimension of each sub-feature f _i '' is

; concatenate the graph convolution features as

The feature map of × C _t dimension; and then through the convolution kernel convolution calculation of 1 × 1 × C _i dimension, the output

× The feature map of dimension C _i , this feature map is marked as T ₂ ; finally, T ₁ and T ₂ are spliced as the final output of the module, and the dimension of the feature map after splicing is

×2 C _i .

The output end of the designed algorithm includes two branches, which can complete the function of target detection and tracking at the same time.

After the captured image is zoomed to 640×640×3, it is input to the lightweight module 1, the lightweight module 2, the lightweight module 3, the Transformer-GCN module 1, the lightweight module 4, the Transformer-GCN module 2, and the lightweight module. 5, the dimension of the output depth feature is 4×4×2048, and then the feature is changed to 1×1×2048 by global pooling.

For the target detection branch, as shown in Figure 11, the global pooled features are mapped into 1 × 6-dimensional prediction results through the fully connected layer, including the target position of the sea cucumber (the center point coordinates x, y and the target frame width and High w, h), the category of the target (whether it is a sea cucumber), and the confidence (probability of being detected as a sea cucumber) six prediction results.

For the tracking branch, the global pooled features are first calculated through the fully connected layer, and the features are mapped into 1×1×256-dimensional features, which are called deep identity features; at the same time, the gradient histogram feature is extracted from the original image. As an artificial identity feature, the feature dimension of the extracted gradient histogram is uncertain, and the feature dimension is mapped to 1×1×256 by principal component analysis. The artificial identity features and deep identity features after fusion and mapping are input into the correlation filter for prediction. The correlation filter calculation formula is as follows:

w y = x

w

Where x is the fusion feature, w is the filter template, the response value y of each target is calculated by convolution, and the target with the largest response value is the current tracking target.

In one example, referring to FIG. 8 , step 2 specifically includes:

2-1, zoom the lower visual image obtained in real time to obtain the zoomed lower visual image;

2-2. Input the scaled lower visual image into the first lightweight module, the second lightweight module, the third lightweight module, the first Transformer-GCN module, the fourth lightweight module, and the second Transformer-GCN module , the fifth lightweight module and the global pooling module, output feature map;

2-3, map the feature map to obtain the prediction result of the sea cucumber to be caught; the prediction result includes the target location, target category and confidence;

2-4, input the feature map into the fully connected module to obtain deep identity features;

2-5, extract the gradient histogram feature from the zoomed lower visual image as an artificial identity feature;

2-6, using principal component analysis to map artificial identity features to the same dimension as deep identity features;

2-7, fuse the mapped artificial identity features and deep identity features to obtain the fused identity features;

2-8, input the fused identity feature into the filtering module, and calculate the response value of each detection target in the zoomed lower visual image;

2-9, select the detection target with the largest response value in the zoomed lower visual image and determine it as the currently tracked sea cucumber to be caught.

In step 3, the pixel coordinates are converted into world coordinates through binocular stereo matching to obtain the three-dimensional position of the sea cucumber to be fished.

In the grabbing process, after obtaining the pixel coordinates of the tracking target, it is necessary to convert the pixel coordinates to the world coordinates through binocular stereo matching, and obtain the real world coordinates of the sea cucumber according to the detection and tracking results.

In one example, this includes:

3-1. Use Zhang Zhengyou's calibration method to calibrate the camera to obtain the camera's internal parameter matrix [ f , 1/ d _x , 1/ d _y , c _x , c _y ] and distortion coefficients [ k ₁ , k ₂ , k ₃ , p ₁ , p ₂ ], and also includes the external parameter rotation matrix R and translation vector t .

3-2, use the internal parameter matrix to convert the pixels of the lower visual image to the camera coordinate system. In order to align the pixel positions of the left and right views of the binocular camera, it is necessary to perform distortion correction and imaging origin alignment on the image according to the camera internal parameters, rotation matrix and translation vector obtained from the camera calibration. The method firstly converts the image pixels to the camera coordinate system through the camera internal parameter matrix, and then converts the image pixels to the pixel coordinate system after conversion by the distortion coefficient in the camera coordinate system.

3-3, in the camera coordinate system, convert the lower visual image pixels by the distortion coefficient, and convert the converted lower visual image pixels to the pixel coordinate system;

3-4, after eliminating distortion and pixel alignment, it is necessary to calculate the disparity map to obtain the three-dimensional coordinates of the target, and realize the conversion from pixel coordinates to world coordinates. Use the formula D = fT / d to calculate the distance D from a point in the space to the camera plane; where f is the focal length obtained by calibration, T is the distance between the two cameras in the binocular camera, and d is the parallax value;

，将所述像素坐标转换为世界坐标，获得待捕捞海参的三维位置；其中，(X,Y,Z)为待捕捞海参的三维位置坐标，(x ₁,y ₁)和(x ₂,y ₂)分别为双目视觉中待捕捞海参在两个相机拍摄的图像中的像素坐标。利用上式依次遍历视差图上每一个像素点，求得目标的深度图，可以求取每一个像素点在世界坐标系下的坐标。3-5, according to the distance from a point in space to the camera plane, use the formula

, convert the pixel coordinates into world coordinates to obtain the three-dimensional position of the sea cucumber to be fished; wherein ( X , Y , Z ) are the three-dimensional position coordinates of the sea cucumber to be fished, ( x ₁ , y ₁ ) and ( x ₂ , y ) ₂ ) are respectively the pixel coordinates of the sea cucumber to be caught in the images captured by the two cameras in binocular vision. Using the above formula to traverse each pixel on the disparity map in turn, to obtain the depth map of the target, the coordinates of each pixel in the world coordinate system can be obtained.

The designed sea cucumber detection model and target tracking model can realize the sea cucumber detection and tracking tasks at the same time, and the designed detection and tracking model is suitable for the sea cucumber fishing task.

After obtaining the three-dimensional position information of the target, the motion planning is carried out through the motion planning algorithm of the underwater robot.

Step 4: Set the three-dimensional position of the sea cucumber to be fished as the target point, and use the fast search tree algorithm to plan the path between the operation-type underwater robot and the target point.

The path between the robot and the target is planned by the fast search tree algorithm, and the process is as follows:

4-1, initialize the root node q _int .

4-2, in each outer loop, randomly select a random point q _rand , after generating a random point, traverse each node in the tree, find the node q _near closest to the random point, define the step variable eps , and then In each inner loop of , the step size eps is extended from q _near to q _rand , reaching a new node q _new , checking whether the path collides, if no collision occurs, add a new node q _new to the tree and enter the next inner loop, if it occurs Collision then does not add new nodes and goes to the next inner loop until q _rand is reached, starting the outer loop again.

4-3, repeat steps 4-2 until reaching the end point, and obtain the path from the current sea cucumber fishing robot to the target point.

The coordinated motion planning and control of the sea cucumber fishing robot is completed by the task priority algorithm. According to the sea cucumber fishing task, the motion priority of the underwater robot can be divided into four levels. The first level is the movement guided by the forward-looking camera. Through the online motion planning algorithm, the position and path of the fishing robot to the target point are planned in real time. The first level is the movement in the three-dimensional position; For the movement guided by the camera, after the robot moves to a certain distance near the target, the sea cucumber is positioned by binoculars, and online path planning is performed at the same time to adjust the position of the fishing robot so that the target is located in the movement space of the robotic arm; the third level is the mechanical The adjustment of the arm posture maximizes the working space of the robotic arm; the fourth level is the movement of the robotic arm to fish for sea cucumbers.

Step 5: According to the forward visual image, the operation-type underwater robot is controlled to move according to the path based on the Actor-Critic reinforcement learning model, and the operation-type underwater robot is suspended after running to the target point; the Actor-Critic reinforcement learning model uses the Gaussian mixture model to analyze the sample space for clustering compression.

Based on the designed Actor-Critic reinforcement learning model, sea cucumber tracking control and suspension grabbing algorithms are implemented according to the planned path. The motion control of the sea cucumber fishing robot is three-dimensional control, so it needs to be controlled at the same time in three attitude angles, three-dimensional coordinates, angular velocity and linear velocity.

As shown in Figure 12, the specific process of the designed model is as follows:

5-1. After the propeller thruster and the manipulator move according to the random initial state function, the current motion posture is obtained, and it is passed into the action network, and the current reward value is calculated according to the expected planning and reward function obtained in step 3. The designed reward function is as follows:

R = r ₀ - ρ ₁ ||Δφ,Δ θ ,Δ ψ ||- ρ ₂ ||Δ x ,Δ y ,Δ z || ₂

Among them, Δ x , Δ y , Δ z are the three-dimensional coordinate state quantities, Δφ, Δ θ , Δ ψ are the yaw angle state quantities, pitch angle state quantities, and roll angle state quantities, respectively, r ₀ is the reward constant, ρ ₁ || Δφ, Δ θ , Δ ψ || are the two-norm of the relative orientation error, and ρ ₂ ||Δ x , Δ y , Δ z || ₂ are the two-norm of the relative position error.

5-2. Integrate the reward value and the state function into training samples and add them to the training set space. At the same time, the Gaussian mixture model is used to fuse the samples to compress the sample space. The state function is represented as follows:

s =[ g ,Δ x ,Δ y ,Δ z ,Δφ,Δ θ ,Δ ψ , u , v , w , p , q , r ]

Where g is the state constant, u , v , w are the linear velocity state quantities, p , q , r are the angular velocity state quantities. So the current training sample space can be expressed as

[ R , s ]=[[ R ₁ , s ₁ ],• • •,[ R _n , s _n ]]

The sample space will gradually increase with time, increasing the amount of calculation. The algorithm designed in the present invention performs clustering and compression on the sample space through the Gaussian mixture model. The Gaussian mixture model for this sample space can be expressed as

=1，

与σ _k 分别为类均值与类方差。需要对

、σ _k 、

进行求解，其中求解出的

作为重构后的样本。There are a total of K categories of samples in this sample space, π _k is the class distribution probability, that is, the weight of the mixture of Gaussian components, so it satisfies

=1,

and σ _k are the class mean and class variance, respectively. need to

, σ _k ,

solve, where the solved

as a reconstructed sample.

The log-likelihood function of this Gaussian mixture model can be expressed as:

For the Gaussian mixture model, it can be solved by the expectation-maximization algorithm. The specific calculation process is as follows:

与σ _k 来求解某个样本的条件概率，即：Step 1: By initializing

and σ _k to solve the conditional probability of a sample, namely:

与σ _k ：Step 2: After obtaining the posterior probability, it can be further optimized by minimizing the log-likelihood function

with σ _k :

即为重构样本空间，Q为中间变量。Repeat these two steps until convergence, and the solved

is the reconstructed sample space, and Q is the intermediate variable.

5-3, input the sample into the target-action network in the evaluation network, perform gradient calculation, and update the parameters of the online state-action network. In addition, after accumulating multiple gradients, the target state-action network parameters are updated through the online state-action network gradient.

5-4, optimize and update the parameters of the online policy network through the gradient obtained by the evaluation network. In addition, after accumulating multiple gradients, the target policy network parameters are updated through the online policy network gradient.

5-5, generate a new state function through the action network for the control of the robotic arm and the propeller.

5-6, loop 5-1 to 5-5 until convergence, that is, the motion results of the robotic arm and the propeller are in line with the motion trajectory and attitude planned in step 3, so as to realize the grasping of the sea cucumber.

Step 6, according to inverse kinematics, the sea cucumber is to be caught by the gripper 81 of the grasping mechanism of the operation-type underwater robot.

The invention can realize precise control and autonomous grasping of a sea cucumber fishing robot in a complex underwater environment.

The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the various embodiments can be referred to each other.

In this paper, specific examples are used to illustrate the principles and implementations of the present invention. The descriptions of the above embodiments are only used to help understand the methods and core ideas of the present invention; meanwhile, for those skilled in the art, according to the present invention There will be changes in the specific implementation and application scope. In conclusion, the contents of this specification should not be construed as limiting the present invention.

Claims (10)

1. An operation type underwater robot for sea cucumber fishing, characterized by comprising: the device comprises a body frame, a propeller, a second control plate, a first control plate, a camera mechanism and a grabbing mechanism;

the propeller, the second control plate, the first control plate and the camera shooting mechanism are all arranged on the body frame, and the fixed end of the grabbing mechanism is connected with the body frame;

the camera shooting mechanism is connected with a signal input end of the second control board, and a signal output end of the second control board is respectively connected with the first control board and the grabbing mechanism; the camera shooting mechanism is used for shooting a front visual image and a lower visual image of the operation type underwater robot and transmitting the front visual image and the lower visual image to the second control board; the second control board is used for determining three-dimensional position information of the sea cucumber according to the lower visual image and carrying out online path planning according to the three-dimensional position information of the sea cucumber and the front visual image;

the first control board is connected with the propeller and used for adjusting the propeller according to the on-line path plan so that the operation type underwater robot moves according to the on-line path plan;

the second control board is also used for controlling the grabbing mechanism to catch the sea cucumbers when the operation type underwater robot reaches a target point of the online path planning.

2. The working underwater robot for sea cucumber fishing according to claim 1, wherein the camera mechanism comprises: a forward looking monocular camera and a look down binocular camera;

the front-view monocular camera and the overlooking binocular camera are both connected with the signal input end of the second control board; the front monocular camera is used for shooting a front visual image of the operation type underwater robot and transmitting the front visual image to the second control board; and the overlooking binocular camera is used for shooting a lower visual image of the operation type underwater robot and transmitting the lower visual image to the second control board.

3. The working type underwater robot for sea cucumber fishing according to claim 1, wherein the propeller comprises: a first propeller thruster, a second propeller thruster, a third propeller thruster, a fourth propeller thruster, a fifth propeller thruster, a sixth propeller thruster, a seventh propeller thruster, and an eighth propeller thruster;

the first propeller thruster, the second propeller thruster, the third propeller thruster and the fourth propeller thruster are arranged in the horizontal direction according to the vector thruster, the first propeller thruster and the second propeller thruster are arranged in front of the horizontal direction of the body frame, and the third propeller thruster and the fourth propeller thruster are arranged behind the horizontal direction of the body frame;

the first propeller thruster, the second propeller thruster, the third propeller thruster, the fourth propeller thruster, the fifth propeller thruster, the sixth propeller thruster, the seventh propeller thruster and the eighth propeller thruster are all connected with the first control board;

the first control board is used for controlling the first propeller thruster, the second propeller thruster, the third propeller thruster and the fourth propeller thruster to provide thrust in the horizontal front-back direction for the operation type underwater robot, and controlling the fifth propeller thruster, the sixth propeller thruster, the seventh propeller thruster and the eighth propeller thruster to provide thrust in the vertical direction for the operation type underwater robot, so that the operation type underwater robot moves according to the on-line path planning.

4. The working type underwater robot for sea cucumber fishing according to claim 1, wherein the catching mechanism comprises: the device comprises a base, a shoulder joint, an elbow joint, a forearm joint, a clamping jaw, a first steering engine, a second steering engine, a third steering engine and a fourth steering engine;

the first steering engine, the second steering engine, the third steering engine and the fourth steering engine are all connected with the second control panel;

the shoulder joint is fixed to the body frame through a base; the base is provided with a first steering engine, and the first steering engine is used for driving the shoulder joint to rotate around the direction of the vertical axis under the control of the second control board;

the elbow joint is connected in series with the shoulder joint, a second steering engine is arranged between the shoulder joint and the elbow joint and used for driving the elbow joint to rotate around the direction vertical to the axis of the shoulder joint under the control of a second control board;

the forearm joint is connected in series with the elbow joint, a third steering engine is arranged between the forearm joint and the elbow joint and used for driving the forearm joint to rotate around the direction vertical to the elbow joint center line under the control of the second control board;

the clamping jaw and the fourth steering engine are arranged above the small arm joint; the clamping jaw comprises two oppositely arranged net structures; the fourth steering wheel is used for driving the two oppositely-arranged net-shaped structures to rotate in opposite directions under the control of the second control board, so that the opening and closing control of the clamping jaw is realized.

5. The working underwater robot for sea cucumber fishing according to claim 1, further comprising: the loading net cage and a fifth steering engine;

the loading net cage is arranged on the body frame, and the loading net cage is arranged opposite to the grabbing mechanism;

the loading net box is connected with the second control board through a fifth steering engine and is used for driving the loading net box to be automatically opened under the control of the second control board when the grabbing mechanism finishes grabbing and moves to a preset position, and loading the sea cucumbers caught by the grabbing mechanism.

6. A control method for a sea cucumber fishing operation type underwater robot is characterized by comprising the following steps:

acquiring a front visual image and a lower visual image of the operation type underwater robot shot by a camera shooting mechanism in real time;

according to the lower visual image obtained in real time, a sea cucumber identification and tracking algorithm based on MobileNet-Transformer-GCN is utilized to identify and continuously track the sea cucumber to be caught, and meanwhile, the pixel coordinates of the sea cucumber to be caught are positioned in real time;

converting the pixel coordinates into world coordinates through binocular stereo matching to obtain the three-dimensional position of the sea cucumber to be caught;

setting the three-dimensional position of the sea cucumber to be caught as a target point, and planning a path from the operation type underwater robot to the target point by adopting a rapid search tree algorithm;

controlling the operation type underwater robot to move according to the path based on an Actor-Critic reinforcement learning model according to the front visual image, and suspending the operation type underwater robot after the operation type underwater robot runs to a target point; the Actor-Critic reinforcement learning model performs clustering compression on a sample space through a Gaussian mixture model;

according to inverse kinematics, the sea cucumbers to be caught are caught by a grabbing mechanism of the operation type underwater robot.

7. The control method of the working underwater robot for sea cucumber catching according to claim 6, wherein the real-time acquisition of the front visual image and the lower visual image of the working underwater robot photographed by the photographing means further comprises:

carrying out color correction and defogging enhancement on the front visual image and the lower visual image by adopting an underwater image enhancement algorithm based on a twin convolutional neural network; the twin convolutional neural network comprises a first branch convolutional neural network and a second branch convolutional neural network; the first branch convolutional neural network is constrained by the color characteristics of the label image and is responsible for color correction of the image; the second branch convolutional neural network is constrained by texture features and is responsible for the image definition; and the first branch convolutional neural network and the second branch convolutional neural network are subjected to convolutional characteristic transformation operation after characteristic constraint, finally the two branch characteristics are spliced in a dot-product mode, and a final clear image is generated through one layer of convolutional transformation after splicing.

8. The control method of the working underwater robot for sea cucumber fishing according to claim 6, wherein the method for identifying and continuously tracking the sea cucumber to be fished and simultaneously locating the pixel coordinates of the sea cucumber to be fished in real time by using a sea cucumber identification and tracking algorithm based on MobileNet-transform-GCN according to the real-time acquired lower visual image specifically comprises the following steps:

zooming the real-time acquired lower visual image to obtain a zoomed lower visual image;

inputting the zoomed lower visual image into a first lightweight module, a second lightweight module, a third lightweight module, a first transform-GCN module, a fourth lightweight module, a second transform-GCN module, a fifth lightweight module and a global pooling module in sequence, and outputting a characteristic diagram;

mapping the characteristic graph to obtain a prediction result of the sea cucumber to be caught; the prediction result comprises a target position, a target category and a confidence coefficient;

inputting the feature map into a full-connection module to obtain a depth identity feature;

extracting gradient histogram features from the zoomed lower visual image as artificial identity features;

mapping the artificial identity features to dimensions the same as the depth identity features by principal component analysis;

fusing the mapped artificial identity features and the depth identity features to obtain fused identity features;

inputting the fused identity characteristics into a filtering module, and calculating the response value of each detection target in the zoomed lower visual image;

and selecting the detection target with the maximum response value in the zoomed lower visual image to determine the current tracked sea cucumber to be caught.

9. The control method for the working underwater robot for sea cucumber fishing according to claim 6, wherein the pixel coordinates are converted into world coordinates through binocular stereo matching to obtain the three-dimensional position of the sea cucumber to be fished, and specifically comprises the following steps:

calibrating the camera by adopting a Zhangyingyou calibration method to obtain an internal reference matrix and a distortion coefficient of the camera;

converting the lower visual image pixels to a camera coordinate system using the internal reference matrix;

converting the lower visual image pixels in a camera coordinate system through a distortion coefficient, and converting the converted lower visual image pixels into a pixel coordinate system;

using formulasD=fT/dCalculating the distance from a point in space to the plane of the cameraD(ii) a Wherein,fin order to calibrate the acquired focal length,Tis the distance of two of the binocular cameras,dis a parallax value;

according to the distance from a point in space to the plane of the camera, using a formula

Converting the pixel coordinates into world coordinates to obtain the three-dimensional position of the sea cucumber to be caught; wherein (A), (B), (C), (D), (C), (B), (C)X,Y,Z) For the three-dimensional position coordinates of the sea cucumber to be caught (x ₁,y ₁) And (a)x ₂,y ₂) Respectively is a picture of the sea cucumber to be caught in binocular vision shot by two camerasPixel coordinates in the image.

10. The control method for the sea cucumber catching-oriented working underwater robot as claimed in claim 6, wherein the Actor-Critic-based reinforcement learning model controls the working underwater robot to move according to the path and to suspend after the working underwater robot runs to a target point, specifically comprises:

acquiring the current motion attitude of the operation type underwater robot, and transmitting the current motion attitude into an online strategy network in a mobile network; the current motion attitude comprises a yaw angle, a pitch angle, a roll angle, a three-dimensional coordinate, an angular velocity and a linear velocity;

calculating a current reward value according to a reward function and the path of the Actor-critical reinforcement learning model; the reward function isR=r ₀-ρ ₁||Δφ,Δθ,Δψ||-ρ ₂||Δx,Δy,Δz||₂(ii) a Wherein, Deltax,Δy,ΔzIs a three-dimensional coordinate state quantity, Δ φ, Δθ,ΔψRespectively a yaw angle state quantity, a pitch angle state quantity and a roll angle state quantity,r ₀in order to award the constant amount of the prize,ρ ₁||Δφ,Δθ,Δψ| | is a two-norm relative orientation error,ρ ₂||Δx,Δy,Δz||₂is a two-norm of the relative position error,ρ ₁andρ ₂a first coefficient and a second coefficient, respectively;

fusing the current reward value and the state function into a training sample and adding the training sample into a sample space; the state function iss=[g,Δx,Δy,Δz,Δφ,Δθ,Δψ,u,v,w,p,q,r](ii) a Wherein,gis a constant value of the state, and the state,u,v,was the state quantity of the linear velocity,p,q,ras the state quantity of the angular velocity,sis a function of the state;

fusing the samples in the sample space through a Gaussian mixture model, and compressing the sample space; the Gaussian mixture model is

(ii) a Wherein,P(x) In order to obtain a compressed sample space,Kfor the class of samples in the sample space before compression,

is the probability of the class distribution,

andσ _k respectively, mean class and variance class: (R _i ,s _i ) Is the first in sample space before compressioniThe number of the samples is one,Nis a Gaussian distribution density function of the submodel;

inputting the compressed samples in the sample space into a target-behavior network in an evaluation network, performing gradient calculation, and updating parameters of the online state-behavior network;

optimizing and updating parameters of an online policy network in the action network through the gradient obtained by evaluating network calculation, and updating parameters of a target policy network through the gradient of the online policy network after accumulating the gradient for multiple times;

generating a new state function through an online strategy network in the action network, wherein the new state function is used for controlling the mechanical arm and the propeller;

and circulating the steps until convergence, so that the motion result of the operation type underwater robot conforms to the path.

Images