CN116363494B - Fish quantity monitoring and migration tracking method and system - Google Patents

Fish quantity monitoring and migration tracking method and system Download PDF

Info

Publication number
CN116363494B
CN116363494B CN202310634836.6A CN202310634836A CN116363494B CN 116363494 B CN116363494 B CN 116363494B CN 202310634836 A CN202310634836 A CN 202310634836A CN 116363494 B CN116363494 B CN 116363494B
Authority
CN
China
Prior art keywords
fish
frame
tracking
detection
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202310634836.6A
Other languages
Chinese (zh)
Other versions
CN116363494A (en
Inventor
刘浩兵
周明
江鹏
程和煦
黄金
徐珏
金靖芳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Ruike Environmental Technology China Co ltd
Original Assignee
Ruike Environmental Technology China Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ruike Environmental Technology China Co ltd filed Critical Ruike Environmental Technology China Co ltd
Priority to CN202310634836.6A priority Critical patent/CN116363494B/en
Publication of CN116363494A publication Critical patent/CN116363494A/en
Application granted granted Critical
Publication of CN116363494B publication Critical patent/CN116363494B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/05—Underwater scenes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
    • Y02A40/81—Aquaculture, e.g. of fish

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a method and a system for monitoring the quantity of fish and tracking migration, wherein the method comprises the following steps: training a deep learning model through the underwater fish image data set to obtain a fish detection model; predicting a first fish detection frame and a corresponding confidence coefficient through a fish detection model; calculating a fish probability map in continuously acquired fish images to be detected through a Gaussian mixture model, and extracting a second fish detection frame; calculating the total number of the first fish detection frame and the second fish detection frame as a preliminary fish quantity statistical value; comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating the repeatedly counted fishes to obtain a fish quantity monitoring value; and (5) carrying out fish migration tracking according to the fish track. According to the fish quantity monitoring method based on the deep learning model and the Gaussian mixture model, the fish quantity monitoring and the track tracking are combined, missing detection and repeated counting can be avoided, and the accuracy of fish quantity monitoring is improved.

Description

Fish quantity monitoring and migration tracking method and system
Technical Field
The invention belongs to the technical field of ecological environment detection, and particularly relates to a method and a system for monitoring the quantity of fish and tracking migration.
Background
In the detection of the water ecological environment, the detection of fish is an important index for evaluating ecological variation, and the quantity and migration of the fish are closely related to the water quality. Environmental workers need to analyze the water quality according to the indexes, know the influence caused by environmental change, and can scientifically formulate an ecological fishing scheme. At present, fish detection mainly comprises two modes 1. Partial water seedlings are adopted, and the total fish quantity is estimated by sampling the quantity of the partial water seedlings; 2. and observing the quantity of the fish in the local water area, and estimating the quantity of the total fish by observing the quantity of the fish through human eyes. The method is strong in subjectivity, has repeated statistics, greatly influences the monitoring quantity due to the change of the external environment, and is low in efficiency due to the need of manual participation.
In the prior art, some schemes for estimating the quantity of fish by using an image processing method also appear, for example, patent application number CN114255203A discloses a method and a system for estimating the quantity of fish fry, which are used for collecting a plurality of images of fish fry to be identified through image collecting devices such as an optical camera and a video camera, carrying out thermodynamic diagram processing on the fish fry images and predicting the quantity of fish through a neural network model. However, the variety scheme is generally only suitable for a scene with clear water body and acquired fry images, and under a common water quality analysis scene, the underwater environment is complex, the water body is often turbid, the acquired images are low in quality, the fishes swim back and forth, the detection omission or repeated counting is easy, and the quantity estimation is difficult to accurately perform.
Therefore, there is a need for a method for monitoring and tracking the migration of fish with more accurate fish count.
Disclosure of Invention
In view of the above, the invention provides a method and a system for monitoring and migration tracking of fish quantity, which are used for solving the problem of low accuracy of fish quantity estimation under the condition of water turbidity.
The invention discloses a method for monitoring the quantity of fish and tracking migration, which comprises the following steps:
building a deep learning model, and training the deep learning model through an underwater fish image data set to obtain a fish detection model;
inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and a corresponding confidence coefficient;
calculating a fish probability map in continuously acquired fish images to be detected through a Gaussian mixture model, and extracting a second fish detection frame;
calculating the total number of the first fish detection frames and the second fish detection frames as a preliminary statistical value of the fish quantity, and extracting depth characteristics of fish in all the fish detection frames;
comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes;
and (5) carrying out fish migration tracking according to the fish track.
On the basis of the technical scheme, preferably, the deep learning model is formed by connecting 32 basic network units in series, and each basic network unit comprises a convolution layer, a BatchNorm layer and an activation layer which are sequentially connected.
On the basis of the above technical solution, preferably, calculating a fish probability map in continuously collected fish images to be detected by a gaussian mixture model, and extracting a second fish detection frame specifically includes:
is provided withtRepresents the firsttThe image of the fish to be measured is framed,is the firsttFrame of fish image to be measurediLine 1jPixel values for pixel locations of the columns;
initializing each pixel positioni, j) Pixel value atIs>Sum of variances->；
For the firsttFraming the fish image to be measured according to the firstt-1 frame of probability that the average value and variance of the fish image to be detected calculate each pixel position as the foreground；
When (when)When according to the firstt-1 frame of pixel position of fish image to be measuredi, j) Pixel mean +.>Update the firsttFrame fish image pixel position to be measuredi, j) Pixel mean +.>Sum of variances->；
According to the firsttPixel position of frame fish image to be measuredi, j) Pixel mean atSum of variances->Recalculating the probability of each pixel position as foreground +.>；
Based on the probability that each pixel position is the foreground calculated againUpdating the corresponding pixel positioni, j) Pixel values of (2):
wherein ,is pixel position #)i, j) Pixel values at;
and obtaining a fish probability map according to pixel values at each pixel position, and extracting a second fish detection frame by solving a connected domain.
On the basis of the technical proposal, preferably, the probability that each pixel position is foreground is calculated through normal distribution。
On the basis of the above technical solution, preferably, the method according to the first aspectt-1 frame of pixel position of fish image to be measuredi, j) Pixel mean atUpdate the firsttFrame fish image pixel position to be measuredi, j) Pixel mean +.>Sum of variances->The formula of (2) is:
wherein ,is the firstt-1 frame of pixel position in the fish image to be measuredi, j) Pixel mean at>、/>Respectively the firsttPixel position in frame fish image to be measuredi, j) Pixel mean and variance at.
On the basis of the above technical solution, preferably, comparing the depth characteristics of the fish in the corresponding fish detection frames at different moments, and tracking the track of the fish specifically includes:
is provided withtTime of day (time)sThe depth characteristic vector of the fish in the individual fish detection frame isIs provided witht-1 timerThe depth characteristic vector of the fish in the individual fish detection frame is +.>, wherein />，N'Is thatt-1 total number of fish detection frames corresponding to time instant;
when (when)In the case of the same fish, the same fish is considered to have a trace which is not repeatedly counted, wherein +.>。
On the basis of the above technical solution, preferably, the tracking fish migration according to the fish trajectory specifically includes:
is provided withIs the firstsFront fishtTrace vector of each momenttThe trajectory vector of the moment is expressed as:
the direction angle is as follows:
wherein ,respectively->In a fish track coordinate systemo-xyTransverse axis of (2)xComponent on the vertical axisyThe upper component;
when (when)For forward migration, add>Is reverse migration.
In a second aspect of the present invention, a system for monitoring and tracking the number of fish is disclosed, the system comprising:
the first detection frame extraction module: the method comprises the steps of constructing a deep learning model, and training the deep learning model through an underwater fish image data set to obtain a fish detection model; inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and a corresponding confidence coefficient;
the second detection frame extraction module: the fish probability map in the continuously acquired fish image to be detected is calculated through the Gaussian mixture model, and a second fish detection frame is extracted;
the fish number calculating module is used for: the method comprises the steps of calculating the total number of a first fish detection frame and a second fish detection frame to serve as a preliminary fish quantity statistical value, and extracting depth characteristics of fish in all fish detection frames; comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes;
fish migration tracking module: is used for carrying out fish migration tracking according to the fish track.
In a third aspect of the present invention, an electronic device is disclosed, comprising: at least one processor, at least one memory, a communication interface, and a bus;
the processor, the memory and the communication interface complete communication with each other through the bus;
the memory stores program instructions executable by the processor which the processor invokes to implement the method according to the first aspect of the invention.
In a fourth aspect of the invention, a computer-readable storage medium is disclosed, storing computer instructions that cause a computer to implement the method according to the first aspect of the invention.
Compared with the prior art, the invention has the following beneficial effects:
1) According to the method, the fish quantity is monitored by combining the fish detection based on the deep learning model and the fish detection based on the Gaussian mixture model, the fish probability map is calculated through the Gaussian mixture model, the problem that the deep learning model cannot well detect all fish targets in the background under the condition of turbid water quality is solved, so that missed detection is avoided, meanwhile, the track of the fish is tracked by comparing the depth characteristics of the fish in the corresponding fish detection frames at different moments, repeated counting can be avoided, and the accuracy of monitoring the fish quantity is improved;
2) According to the method, the mean value and the variance of the pixel positions corresponding to the fish image to be detected in the next frame are updated according to the mean value of the pixel positions corresponding to the fish image to be detected in the first frame, so that the water background images at different moments are associated, the method is suitable for the dynamic change of the underwater environment, the fish probability map is calculated in the dynamic environment, and the accuracy of the detection frame for extracting the rest fish is improved;
3) According to the invention, the migration direction of the fish can be judged according to fish tracking, the method is not influenced by environment and weather, the quantity and migration state of the fish can be systematically observed, and the calculation efficiency is high.
Drawings
In order to more clearly illustrate the embodiments of the invention or the technical solutions in the prior art, the drawings that are required in the embodiments or the description of the prior art will be briefly described, it being obvious that the drawings in the following description are only some embodiments of the invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic diagram of a detection framework of the fish quantity monitoring and migration tracking method of the invention;
FIG. 2 is a schematic illustration of labeling fish samples;
FIG. 3 is a schematic diagram of a deep learning model structure;
fig. 4 is a partial screenshot of an image of fish to be tested and a corresponding fish probability map.
Detailed Description
The following description of the embodiments of the present invention will clearly and fully describe the technical aspects of the embodiments of the present invention, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, are intended to fall within the scope of the present invention.
Existing machine-based learning/The monitoring of the fish quantity in deep learning is generally only suitable for the condition of better water quality and clear acquired image, but for the monitoring of the fish quantity used for water quality detection and analysis, the accuracy of the method is obviously reduced when the water body is turbid or the underwater environment is changed, the method is subject to missed detection or repeated counting, the estimation of the fish quantity is inaccurate, and the migration tracking of the fish is also an important ring of water quality analysis, and the existing monitoring of the fish quantity is seldom designed for tracking the track. The invention provides a method and a system for monitoring the quantity of fish and tracking migration, which can accurately detect the quantity of fish and track migration and improve the calculation efficiency.
The invention provides a method for monitoring the quantity of fish and tracking migration, which adopts a detection framework combining a deep learning model and a Gaussian mixture model, and fig. 1 is a schematic diagram of the detection framework, and the method comprises the following steps:
s1, acquiring underwater fish images and labeling samples to construct a data set.
A series of underwater fish images are collected and labeled, as shown in fig. 2, which is a fish sample labeling schematic diagram, wherein four round dots around the fish are four corner points of a labeling frame, and a data set is constructed after the fish sample and the negative sample are labeled. The data set may be divided into a training set for training the deep learning model of step S2, a verification set for verifying and testing the trained fish detection model, and a test set for testing the trained fish detection model.
S2, constructing a deep learning model, and training the deep learning model through a data set to obtain a fish detection model.
The deep learning model is shown in fig. 3, and is composed of 32 basic networks connected in series, wherein each basic network comprises 5 subunits, in the basic network on the right side of fig. 2, 5 Sigmoid (BN (Conv (x) _w [) units) from top to bottom are sequentially a first subunit, a second subunit, a third subunit, a fourth subunit and a fifth subunit, wherein the first subunit, the second subunit, the third subunit and the fourth subunit are sequentially connected in series, the output result of the second subunit and the output result of the fourth subunit are added to obtain a first intermediate result, the output result of the first subunit is input to the fifth subunit to obtain a second intermediate result, and the first intermediate result and the second intermediate result are added to obtain a feature map extracted by each basic network.
Each subunit comprises a convolutional layer Conv (x), a BN layer and an activation layer which are sequentially connected, wherein the BN layer represents BatchNorm, the activation layer adopts a Sigmoid function, so that each subunit can be represented by Sigmoid (BN (Conv (x) _w [. Cndot ]), w [. Cndot. ] represents the weight vector size corresponding to the convolutional layer Conv (x), and the weight vector size corresponding to the convolutional layer Conv (x) of each subunit is different.
S3, inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and corresponding confidence.
Continuously collecting images of fish to be detected, and extracting a first fish detection frame through a fish detection model.
Is provided withThe characteristic of the fish image matrix to be detected, which is input by the input layer and is subjected to one layer of convolution layer, BN layer and activation function, is expressed as follows:
the features after 161 layers can be expressed as:
the features of layer 161 are divided into two parts:
wherein Representing the confidence level of the fish->And the parameters representing the fish detection frame are used for representing the position of the first fish detection frame in the fish image to be detected.
S4, calculating a fish probability map in the continuously acquired fish image to be detected through the Gaussian mixture model, and extracting a second fish detection frame.
Under the conditions of turbid water quality, water body environment change and the like, the fish detection model based on the deep learning model has the condition of omission, the fish probability map of the fish image to be detected is calculated through the Gaussian mixture model, namely, secondary detection is carried out through the Gaussian mixture model, and a second fish detection frame is extracted.
The specific mode for extracting the second fish detection frame through the Gaussian mixture model is as follows:
s41 is provided withtRepresents the firsttThe image of the fish to be measured is framed,is the firsttFrame of fish image to be measurediLine 1jPixel position of column [ ]i, j) Pixel values at.
Setting the image of the fish to be tested to be sharedmRow of linesnThe number of columns, then,i=1,2,...,m，j=1,2,...,n。
s42, initializing each pixel positioni, j) Pixel value atIs>Sum of variances->：
Wherein the images of the fishes to be detected are sharedTA frame of +1,t=0,1,2,...,T。
s43 for the firsttFraming the fish image to be measured according to the firstt-1 frame of probability that the average value and variance of the fish image to be detected calculate each pixel position as the foreground：
wherein ,representing normal distribution，/>Respectively the firstt-1 frame of fish image to be measured at pixel positioni, j) Pixel mean and variance at.
S44, whenWhen according to the firstt-1 frame of pixel position of fish image to be measuredi, j) Pixel mean atUpdate the firsttFrame fish image pixel position to be measuredi, j) Pixel mean +.>Sum of variances->；
wherein ,is the firstt-1 frame of pixel position in the fish image to be measuredi, j) Pixel mean at>、/>Respectively the firsttPixel position in frame fish image to be measuredi, j) Pixel mean and variance at.
S45 according to the firsttPixel position of frame fish image to be measuredi, j) Pixel mean atSum of variances->Recalculating the probability of each pixel position as foreground +.>。
Based on the firsttPixel position of frame fish image to be measuredi, j) Pixel mean atSum of variances->Calculating the position of each pixel based on normal distributioni,j) Probability of being foreground->。
Based on the probability that each pixel position is the foreground calculated againUpdating the corresponding pixel positioni, j) Pixel values of (2):
wherein ,is pixel position #)i, j) Pixel values at.
Obtaining a fish probability map according to pixel values at each pixel position, wherein the fish probability map is a partial screenshot of a fish image to be detected and a corresponding fish probability map as shown in fig. 4, and extracting a second fish detection frame by solving a connected domain, wherein ,/>The connected domain of (a) represents the fish target, otherwise is the background image.
According to the method, the mean value and the variance of the pixel positions corresponding to the fish image to be detected in the next frame are updated according to the mean value of the pixel positions corresponding to the fish image to be detected in the first frame, so that the water background images at different moments are associated, the method is suitable for the dynamic change of the underwater environment, the fish probability map is calculated in the dynamic environment, and the accuracy of the detection frame for extracting the rest fish is improved.
S5, calculating the total number of the first fish detection frame and the second fish detection frame as a preliminary statistical value of the fish quantity, and extracting depth characteristics of fish in all the fish detection frames.
Specifically, all detected detection frames are extractedCalculating the total number of the first fish detection frame and the second fish detection frameNAs a preliminary statistic of fish quantity.
Initializing the fish track, and respectively corresponding the positions of the first fish detection frame and the second fish detection frame to the 160 th layer deep learning model, so that the characteristics of the positions of the corresponding detection frames can be extracted.
S6, comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating the repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes.
Is provided withtTime of day (time)sThe depth characteristic vector of the fish in the individual fish detection frame isIs provided witht-1 timerThe depth characteristic vector of the fish in the individual fish detection frame is +.>, wherein />，N'Is thatt-1 total number of fish detection frames corresponding to time instant. Computing depth feature vector +.> and />Similarity of->，. When (when)And when the fish is considered as the track of the same fish, the track of the same fish is not repeatedly counted, and the condition of repeated counting is removed from the preliminary statistical value of the fish quantity to obtain a corrected fish quantity monitoring value.
According to the fish quantity monitoring method, the fish quantity monitoring is carried out by combining the fish detection based on the deep learning model and the fish detection based on the Gaussian mixture model, the fish probability map is calculated through the Gaussian mixture model, the problem that all fish targets in the background cannot be well detected by the deep learning model under the conditions of turbid water quality and water body environment change is solved, so that missed detection is avoided, meanwhile, the track of the fish is tracked by comparing the depth characteristics of the fish in the corresponding fish detection frames at different moments, repeated counting can be avoided, and the accuracy of fish quantity monitoring is improved.
S7, carrying out migration tracking on the fish according to the track of the fish.
Is provided withIs the firstsFront fishtTrace vector of each momenttThe trajectory vector of the moment is expressed as:
the direction angle is as follows:
wherein ,respectively->In a fish track coordinate systemo-xyTransverse axis of (2)xComponent on the vertical axisyThe upper component;
when (when)For forward migration, add>Is reverse migration.
According to the invention, the characteristics of the fish are compared through the artificial intelligence algorithm, the influence of environmental weather can be avoided, the track of the fish can be accurately tracked, the condition of repeated statistics is avoided, the fish migration analysis can be simultaneously carried out, the quantity and migration state of the fish can be systematically observed, and the calculation accuracy is high.
Corresponding to the embodiment of the method, the invention also provides a system for monitoring the quantity of fish and tracking migration, which comprises:
sample marking module: the method comprises the steps of acquiring underwater fish images, marking samples, and constructing a data set;
the first detection frame extraction module: the method comprises the steps of constructing a deep learning model, and training the deep learning model through a data set to obtain a fish detection model; inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and a corresponding confidence coefficient;
the second detection frame extraction module: the fish probability map in the continuously acquired fish image to be detected is calculated through the Gaussian mixture model, and a second fish detection frame is extracted;
the fish number calculating module is used for: the method comprises the steps of calculating the total number of a first fish detection frame and a second fish detection frame to serve as a preliminary fish quantity statistical value, and extracting depth characteristics of fish in all fish detection frames; comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes;
fish migration tracking module: is used for carrying out fish migration tracking according to the fish track.
The system embodiments and the method embodiments are in one-to-one correspondence, and the brief description of the system embodiments is just to refer to the method embodiments.
The invention also discloses an electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus; the processor, the memory and the communication interface complete communication with each other through the bus; the memory stores program instructions executable by the processor that the processor invokes to implement the aforementioned methods of the present invention.
The invention also discloses a computer readable storage medium storing computer instructions for causing a computer to implement all or part of the steps of the methods of the embodiments of the invention. The storage medium includes: a usb disk, a removable hard disk, a ROM, a RAM, a magnetic disk, or an optical disk, or other various media capable of storing program codes.
The system embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, i.e., may be distributed over a plurality of network elements. One of ordinary skill in the art may select some or all of the modules according to actual needs without performing any inventive effort to achieve the objectives of the present embodiment.
The foregoing description of the preferred embodiments of the invention is not intended to be limiting, but rather is intended to cover all modifications, equivalents, alternatives, and improvements that fall within the spirit and scope of the invention.

Claims (9)

1. A method for monitoring and migration tracking of fish quantity, which is characterized by comprising the following steps:
building a deep learning model, and training the deep learning model through an underwater fish image data set to obtain a fish detection model;
inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and a corresponding confidence coefficient;
calculating a fish probability map in continuously acquired fish images to be detected through a Gaussian mixture model, and extracting a second fish detection frame;
calculating the total number of the first fish detection frames and the second fish detection frames as a preliminary statistical value of the fish quantity, and extracting depth characteristics of fish in all the fish detection frames;
comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes;
carrying out fish migration tracking according to the fish track; the fish migration tracking according to the fish track specifically comprises the following steps:
is provided withIs the firstsFront fishtTrace vector of each momenttThe trajectory vector of the moment is expressed as:
；
the direction angle is as follows:
；
wherein ,respectively->In a fish track coordinate systemo-xyTransverse axis of (2)xComponent on the vertical axisyThe upper component;
when (when)For forward migration, add>Is reverse migration.
2. The method for monitoring and migration tracking of fish quantity according to claim 1, wherein the deep learning model is composed of 32 basic network units in series, and each basic network unit comprises a convolution layer, a BatchNorm layer and an activation layer which are sequentially connected.
3. The method for monitoring and migration tracking of fish numbers according to claim 1, wherein the calculating a fish probability map in continuously collected images of fish to be detected by using a gaussian mixture model, and extracting a second fish detection frame specifically comprises:
is provided withtRepresents the firsttThe image of the fish to be measured is framed,is the firsttFrame of fish image to be measurediLine 1jPixel values for pixel locations of the columns;
initializing each pixel positioni, j) Pixel value atIs>Sum of variances->；
For the firsttFraming the fish image to be measured according to the firstt-1 frame of probability that the average value and variance of the fish image to be detected calculate each pixel position as the foreground；
When (when)When according to the firstt-1 frame of pixel position of fish image to be measuredi, j) Pixel mean +.>Update the firsttFrame fish image pixel position to be measuredi, j) Pixel mean +.>Sum of variances->；
According to the firsttPixel position of frame fish image to be measuredi, j) Pixel mean atSum of variances->Recalculating the probability of each pixel position as foreground +.>；
Based on the probability that each pixel position is the foreground calculated againUpdating the corresponding pixel positioni, j) Pixel values of (2):
；
wherein ,is pixel position #)i, j) Pixel values at;
and obtaining a fish probability map according to the pixel values at each pixel position, and extracting a second fish detection frame by solving a connected domain for the fish probability map.
4. A method for monitoring fish quantity and tracking migration as claimed in claim 3, wherein the probability of each pixel position as foreground is calculated by normal distribution。
5. The method for monitoring and tracking the migration of fish according to claim 4, wherein the current time is the following timeWhen according to the firstt-1 frame of pixel position of fish image to be measuredi, j) Pixel mean +.>Update the firsttFrame fish image pixel position to be measuredi, j) Pixel mean +.>Sum of variances->The formula of (2) is:
；
wherein ,is the firstt-1 frame of pixel position in the fish image to be measuredi, j) Pixel mean at>、/>Respectively the firsttPixel position in frame fish image to be measuredi, j) Pixel mean and variance at.
6. The method for monitoring and tracking the number of fish as claimed in claim 4, wherein the comparing the depth characteristics of the fish in the corresponding fish detection frames at different moments, and tracking the track of the fish specifically comprises:
is provided withtTime of day (time)sThe depth characteristic vector of the fish in the individual fish detection frame isIs provided witht-1 timerThe depth characteristic vector of the fish in the individual fish detection frame is +.>, wherein />，N'Is thatt-1 total number of fish detection frames corresponding to time instant;
when (when)In the case of the same fish, the same fish is considered to have a trace which is not repeatedly counted, wherein +.>。
7. A system for fish quantity monitoring and migration tracking using the method of any one of claims 1-6, the system comprising:
the first detection frame extraction module: the method comprises the steps of constructing a deep learning model, and training the deep learning model through an underwater fish image data set to obtain a fish detection model; inputting a plurality of continuously acquired fish images to be detected into a fish detection model, and predicting to obtain a first fish detection frame and a corresponding confidence coefficient;
the second detection frame extraction module: the fish probability map in the continuously acquired fish image to be detected is calculated through the Gaussian mixture model, and a second fish detection frame is extracted;
the fish number calculating module is used for: the method comprises the steps of calculating the total number of a first fish detection frame and a second fish detection frame to serve as a preliminary fish quantity statistical value, and extracting depth characteristics of fish in all fish detection frames; comparing the depth characteristics of the fishes in the corresponding fish detection frames at different moments, tracking the track of the fishes, and eliminating repeatedly counted fishes from the preliminary statistical value of the number of the fishes to obtain a monitoring value of the number of the fishes;
fish migration tracking module: is used for carrying out fish migration tracking according to the fish track.
8. An electronic device, comprising: at least one processor, at least one memory, a communication interface, and a bus;
the processor, the memory and the communication interface complete communication with each other through the bus;
the memory stores program instructions executable by the processor, the processor invoking the program instructions to implement the method of any of claims 1-6.
9. A computer readable storage medium storing computer instructions for causing a computer to implement the method of any one of claims 1 to 6.
CN202310634836.6A 2023-05-31 2023-05-31 Fish quantity monitoring and migration tracking method and system Active CN116363494B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310634836.6A CN116363494B (en) 2023-05-31 2023-05-31 Fish quantity monitoring and migration tracking method and system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310634836.6A CN116363494B (en) 2023-05-31 2023-05-31 Fish quantity monitoring and migration tracking method and system

Publications (2)

Publication Number Publication Date
CN116363494A CN116363494A (en) 2023-06-30
CN116363494B true CN116363494B (en) 2023-08-04

Family

ID=86924057

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310634836.6A Active CN116363494B (en) 2023-05-31 2023-05-31 Fish quantity monitoring and migration tracking method and system

Country Status (1)

Country Link
CN (1) CN116363494B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117218161B (en) * 2023-11-09 2024-01-16 聊城市敏锐信息科技有限公司 Fish track tracking method and system in fish tank

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805064A (en) * 2018-05-31 2018-11-13 中国农业大学 A kind of fish detection and localization and recognition methods and system based on deep learning
WO2021114702A1 (en) * 2019-12-10 2021-06-17 中国银联股份有限公司 Target tracking method, apparatus and system, and computer-readable storage medium
WO2021217934A1 (en) * 2020-04-28 2021-11-04 平安国际智慧城市科技股份有限公司 Method and apparatus for monitoring number of livestock, and computer device and storage medium
CN113689488A (en) * 2021-08-19 2021-11-23 安徽工大信息技术有限公司 Offshore culture fish frame number and area statistical method based on deep learning

Family Cites Families (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103324977B (en) * 2012-03-21 2016-03-30 日电（中国）有限公司 A kind of destination number detection method and equipment
CN108038872B (en) * 2017-12-22 2021-08-31 中国海洋大学 Dynamic and static target detection and real-time compressed sensing tracking research method
KR20200060868A (en) * 2018-11-23 2020-06-02 주식회사 월드씨엔에스 multi-view monitoring system using object-oriented auto-tracking function
CN112070799A (en) * 2020-05-29 2020-12-11 清华大学 Fish trajectory tracking method and system based on artificial neural network
CN112487908B (en) * 2020-11-23 2022-11-18 东南大学 Front vehicle line pressing behavior detection and dynamic tracking method based on vehicle-mounted video
CN113139620A (en) * 2021-05-14 2021-07-20 重庆理工大学 End-to-end multi-target detection and tracking joint method based on target association learning
CN113674328B (en) * 2021-07-14 2023-08-25 南京邮电大学 Multi-target vehicle tracking method
CN113674306A (en) * 2021-07-29 2021-11-19 杭州宇泛智能科技有限公司 Pedestrian trajectory acquisition method, system, device and medium based on fisheye lens
CN113689459B (en) * 2021-07-30 2023-07-18 南京信息工程大学 Real-time tracking and mapping method based on GMM and YOLO under dynamic environment
CN114202563A (en) * 2021-12-15 2022-03-18 中国农业大学 Fish multi-target tracking method based on balance joint network
CN114863323A (en) * 2022-04-13 2022-08-05 中国农业科学院农业信息研究所 Real-time production measurement remote monitoring method and system for orchard
CN114782481A (en) * 2022-04-19 2022-07-22 国能大渡河大数据服务有限公司 Method for predicting number and direction of fish swims based on visual technology
CN114782875B (en) * 2022-05-18 2023-04-21 电子科技大学 Fish fine granularity information acquisition method based on fishway construction
CN115424292A (en) * 2022-07-29 2022-12-02 华南农业大学 Improved YOLOX and DeepsORT based fish school tracking method
CN115496780A (en) * 2022-09-21 2022-12-20 浙江大学 Fish identification counting, speed measurement early warning and access quantity statistical system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108805064A (en) * 2018-05-31 2018-11-13 中国农业大学 A kind of fish detection and localization and recognition methods and system based on deep learning
WO2021114702A1 (en) * 2019-12-10 2021-06-17 中国银联股份有限公司 Target tracking method, apparatus and system, and computer-readable storage medium
WO2021217934A1 (en) * 2020-04-28 2021-11-04 平安国际智慧城市科技股份有限公司 Method and apparatus for monitoring number of livestock, and computer device and storage medium
CN113689488A (en) * 2021-08-19 2021-11-23 安徽工大信息技术有限公司 Offshore culture fish frame number and area statistical method based on deep learning

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
基于图像处理的鱼群运动监测方法研究;袁永明;施;;南方水产科学(第05期);111-116 *
基于改进的YOLO与GMM相结合的地铁行人检测算法;田青;齐自强;高国飞;刘婧怡;;工业控制计算机(第04期);49-55 *

Also Published As

Publication number Publication date
CN116363494A (en) 2023-06-30

Similar Documents

Publication Publication Date Title
CN104766320B (en) Many Bernoulli Jacob under thresholding is measured filter Faint target detection and tracking
CN107844797A (en) A kind of method of the milking sow posture automatic identification based on depth image
CN115661943B (en) Fall detection method based on lightweight attitude assessment network
CN116363494B (en) Fish quantity monitoring and migration tracking method and system
CN110827304B (en) Traditional Chinese medicine tongue image positioning method and system based on deep convolution network and level set method
CN113298023B (en) Insect dynamic behavior identification method based on deep learning and image technology
CN114463675B (en) Underwater fish group activity intensity identification method and device
CN112818969A (en) Knowledge distillation-based face pose estimation method and system
CN109376589A (en) ROV deformation target and Small object recognition methods based on convolution kernel screening SSD network
CN111141653B (en) Tunnel leakage rate prediction method based on neural network
CN113610046B (en) Behavior recognition method based on depth video linkage characteristics
CN112861666A (en) Chicken flock counting method based on deep learning and application
CN114463843A (en) Multi-feature fusion fish abnormal behavior detection method based on deep learning
CN115512215A (en) Underwater biological monitoring method and device and storage medium
CN114663769B (en) Fruit identification method based on YOLO v5
CN113076876B (en) Face spoofing detection method and system based on three-dimensional structure supervision and confidence weighting
CN117576616A (en) Deep learning-based fish swimming behavior early warning method, system and device
CN116434066B (en) Deep learning-based soybean pod seed test method, system and device
CN117392668A (en) Wheat scab state evaluation method and system and electronic equipment
CN117333795A (en) River surface flow velocity measurement method and system based on screening post-treatment
CN117237990A (en) Method and device for estimating weight of pig farm, electronic equipment and storage medium
CN113628182B (en) Automatic fish weight estimation method and device, electronic equipment and storage medium
CN117078637A (en) Method and system for counting fish shoal based on context aggregation and self-distillation
CN109034033B (en) Smoke discharge video detection method based on improved VGG16 convolutional network
CN112700425B (en) Determination method for X-ray image quality of power equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant