CN114596625A - Fish school track monitoring system based on edge calculation - Google Patents
Fish school track monitoring system based on edge calculation Download PDFInfo
- Publication number
- CN114596625A CN114596625A CN202210028326.XA CN202210028326A CN114596625A CN 114596625 A CN114596625 A CN 114596625A CN 202210028326 A CN202210028326 A CN 202210028326A CN 114596625 A CN114596625 A CN 114596625A
- Authority
- CN
- China
- Prior art keywords
- fish school
- edge
- fish
- image
- monitoring system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01K—ANIMAL HUSBANDRY; CARE OF BIRDS, FISHES, INSECTS; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- A01K61/00—Culture of aquatic animals
- A01K61/90—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination
- A01K61/95—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination specially adapted for fish
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
- Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
- Y02A40/81—Aquaculture, e.g. of fish
Abstract
The invention discloses a fish school track monitoring system based on edge calculation, which comprises user terminal equipment and at least one edge device, wherein the edge device comprises a camera and an edge calculation main board, the camera is used for collecting a fish school movement video, and the edge calculation main board comprises: the data receiving unit is used for receiving the fish school motion video and extracting images; the detection unit is used for deploying a pre-trained Yolov5 model and detecting an image; the data analysis unit is used for obtaining the barycenter coordinates of the fish school of each frame of image and executing the following operations: calculating the displacement sum of the fish school in the Kth period; judging whether the difference of the fish school displacement sum of two adjacent periods is greater than a preset threshold value or not; and the user terminal equipment is used for receiving the early warning information sent by each edge computing mainboard. The system realizes the abnormal behavior monitoring of the fish shoal based on edge calculation, saves the operation resources of the server, improves the real-time performance and the accuracy of monitoring, and has the advantages of low labor cost, convenient deployment and wide application range.
Description
Technical Field
The invention belongs to the technical field of aquaculture, and particularly relates to a fish school track monitoring system based on edge calculation.
Background
In the process of fish aquaculture, water quality, temperature, health conditions of fish schools and the like are very important parts, the movement tracks of the fish schools are generally stable under normal conditions, and the fish school tracks are messy under the conditions that the environment changes or the health conditions of the fish schools are poor, namely, abnormal conditions are considered to occur, and need to be found and processed in time so as to avoid great loss.
In recent years, the technology of fish aquaculture is developing in an intelligent and automatic direction. However, the existing enterprises often use the traditional machine learning algorithm combined with the digital image processing algorithm to monitor the fish track, the accuracy of fish swarm track monitoring is low, and a large amount of calculation data needs to be uploaded to a server for operation, so that the time for fish swarm track monitoring is increased, the real-time requirement is difficult to achieve, and meanwhile, precious server resources are occupied. Or the operation is still carried out by relying on a series of traditional culture modes such as pure manpower or manual sensor adding (such as an infrared sensor, an ultrasonic sensor and the like), the accuracy rate of monitoring the fish school track in the fish aquaculture process is low due to the problems of high personal subjective factors, high sensor failure rate, easy aging and the like, and meanwhile, the culture cost is overhigh due to the fact that the configuration cost of manpower and sensors is higher and higher.
With the continuous progress of the edge computing field, the cost of the edge computing equipment is lower and lower, and the rapid development of deep learning in the computer vision and image processing field is combined, more lightweight and rapid models are provided, and the requirements of the models on the computer performance are reduced. It is possible to perform fish school trajectory monitoring by combining edge calculation and computer vision methods instead of traditional manual work or sensors. Therefore, the application provides a fish school track monitoring system based on edge calculation with low cost, high accuracy and real-time performance.
Disclosure of Invention
The invention aims to provide a fish school track monitoring system based on edge calculation, which can accurately monitor abnormal behavior of a fish school based on edge calculation, reduce labor cost in a culture process, save server operation resources, improve the real-time property of fish school track prediction, and has the advantages of convenient deployment and wide application range.
In order to achieve the purpose, the technical scheme adopted by the invention is as follows:
the invention provides a fish school track monitoring system based on edge calculation, which comprises user terminal equipment and at least one edge device, wherein:
the edge device comprises a camera and an edge calculation main board, the camera is used for collecting fish school movement videos, the edge calculation main board comprises a data receiving unit, a detection unit and a data analysis unit which are connected in sequence, wherein:
the data receiving unit is used for receiving the fish school motion video and extracting images;
the detection unit is used for deploying a pre-trained YOLOv5 network model, detecting an image through the YOLOv5 network model, and obtaining a detection frame and corresponding position information [ x, y, w, h ] of each fish, wherein (x, y) is the center coordinate of the detection frame, namely the centroid coordinate of the fish, w is the width of the detection frame, and h is the height of the detection frame;
the data analysis unit is used for obtaining the barycenter coordinates of the fish school of each frame of image and executing the following operations:
calculating the fish school displacement sum S in the Kth periodKThe following formula is satisfied:
wherein (x)i,yi) The fish school centroid coordinate of the ith frame image in the Kth period is shown, n is the total number of images in the Kth period, and K is 1,2,3 and …;
judging whether the difference between the fish school displacement sum of two adjacent periods is larger than a preset threshold value, if so, determining that the fish school movement track is abnormal, and transmitting early warning information to user terminal equipment, otherwise, determining that the fish school movement track is normal, and not transmitting the early warning information to the user terminal equipment;
and the user terminal equipment is used for receiving the early warning information sent by the data analysis unit of each edge computing mainboard.
Preferably, the user terminal device is a server.
Preferably, the coordinates of the centroid of the fish school are obtained by a weighted average method according to the position information of each fish in each frame of image.
Preferably, the data receiving unit is further configured to perform preprocessing on the extracted image, where the preprocessing includes sequentially performing compression processing on the image and removing a background based on a gaussian mixture model.
Preferably, the preset threshold is 25%.
Preferably, the edge computing motherboard is a jetson nano motherboard.
Compared with the prior art, the invention has the beneficial effects that: the system adopts the camera and the edge computing mainboard to complete the deployment of edge equipment, the edge computing mainboard detects the fish school by deploying the lightweight YOLOv5 model, and simulates and judges whether abnormal behaviors exist in the fish school track according to the change of the barycenter of the fish school, and preprocesses the image before detection, thereby removing the influence of image background change factors on the model prediction accuracy, solving the problem that the traditional algorithm is difficult to accurately monitor the fish school track, realizing more accurate track monitoring effect, reducing the labor cost in the culture process, simultaneously avoiding uploading a large amount of data to a server for model reasoning operation, saving valuable server resources, improving the real-time performance of fish school track prediction in the actual culture process, and being convenient to deploy and wide in application range.
Drawings
FIG. 1 is a schematic structural diagram of a fish school trajectory monitoring system based on edge calculation according to the present invention;
FIG. 2 is a flow chart of the fish school trajectory monitoring system based on edge calculation according to the present invention;
FIG. 3 is an original image extracted from a fish motion video by the data receiving unit of the present invention;
FIG. 4 is a diagram of the output result of the detecting unit according to the present invention;
FIG. 5 is a diagram of the movement locus of the fish in the normal condition of the present invention;
fig. 6 is a diagram of the movement locus of the fish school under the abnormal condition of the invention.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It is to be noted that, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the present application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.
As shown in fig. 1 to 6, a fish school trajectory monitoring system based on edge calculation includes a user terminal device and at least one edge device, wherein:
the edge device comprises a camera and an edge calculation main board, the camera is used for collecting fish school movement videos, the edge calculation main board comprises a data receiving unit, a detection unit and a data analysis unit which are connected in sequence, wherein:
the data receiving unit is used for receiving the fish school motion video and extracting images;
the detection unit is used for deploying a pre-trained YOLOv5 network model, detecting images through the YOLOv5 network model, and obtaining a detection frame of each fish and corresponding position information [ x, y, w, h ], wherein (x, y) are central coordinates of the detection frame, namely centroid coordinates of the fish, w is the width of the detection frame, and h is the height of the detection frame;
the data analysis unit is used for obtaining the barycenter coordinates of the fish school of each frame of image and executing the following operations:
calculating the fish school displacement sum S in the Kth periodKThe following formula is satisfied:
wherein (x)i,yi) The fish school centroid coordinate of the ith frame image in the Kth period is shown, n is the total number of images in the Kth period, and K is 1,2,3 and …;
judging whether the difference between the fish school displacement sum of two adjacent periods is larger than a preset threshold value, if so, determining that the fish school movement track is abnormal, and transmitting early warning information to user terminal equipment, otherwise, determining that the fish school movement track is normal, and not transmitting the early warning information to the user terminal equipment;
and the user terminal equipment is used for receiving the early warning information sent by the data analysis unit of each edge computing mainboard.
The detection unit is used for deploying a pre-trained Yolov5 network model, and for training the Yolov5 network model, an image set extracted from a fish school activity video shot by a camera can be used as a data set, for example, the data set is represented by 8: the scale of 2 is divided into a training set and a test set. The training set is labeled and then trained. And testing the trained model by using the test set, if the recognition accuracy reaches more than the preset accuracy (such as 95%), considering that the model training is finished, saving the model parameter file, stopping training, and obtaining the pre-trained Yolov5 network model, otherwise, continuing training until the recognition accuracy reaches more than 95%. And a pre-trained YOLOv5 network model is directly deployed on the edge computing mainboard, so that the fish school activity track is monitored in real time, and the deployment is convenient.
And then judging the fish school movement track through a data analysis unit, when the difference of the fish school displacement sum of two adjacent periods is smaller than or equal to a preset threshold value, continuously monitoring the fish school movement track in real time without issuing early warning information by user terminal equipment, and when the difference of the fish school displacement sum of two adjacent periods is larger than the preset threshold value, considering that the fish school movement track is abnormal, and issuing early warning information to the user terminal equipment. The preset threshold value can be designed according to actual requirements. The user terminal equipment can be connected with the data analysis units of the plurality of edge computing main boards at the same time, and if the number of the edge equipment comprises m, the user terminal equipment is used for receiving early warning information sent by the m edge equipment.
Specifically, the fish shoal displacement sum for each cycle is calculated as follows: in this embodiment, each period (e.g. 100S) includes 100 frames of images, the position distance between two adjacent frames is calculated according to the fish school centroid position of each frame of image, and the sum S of fish school displacement of 100 frames is obtained after additionKRepeating the operation for the next period to obtain a second fish school displacement sum SK+1When S isKAnd SK+1And if the difference does not exceed the preset threshold value, the fish shoal is considered to be in a normal state, otherwise, the fish shoal is considered to be in an abnormal state. Fish school displacement sum SKCalculating the formula:
connecting the fish school mass centers of each frame of image in each period to obtain a fish school motion trail graph, wherein the fish school mass center coordinate (x) of each frame of imagei,yi) The width and height of the image are multiplied by the pixel coordinates, and then the pixel coordinates are converted into pixel coordinates, as shown in fig. 5 and 6, fig. 5 is a diagram of the fish school movement locus under normal behavior, and fig. 6 is a diagram of the fish school movement locus under abnormal behavior.
The system adopts the camera and the edge computing mainboard to complete the deployment of the edge equipment, the edge computing mainboard detects the fish school by deploying the lightweight YOLOv5 model, and simulates and judges whether abnormal behaviors exist in the fish school track according to the change of the barycenter of the fish school, the problem that the traditional algorithm is difficult to accurately monitor the fish school track is solved, more accurate track monitoring effect is realized, the labor cost in the culture process is reduced, meanwhile, local data are directly calculated and processed and then transmitted to the user terminal equipment, the phenomenon that a large amount of data are uploaded to a server to carry out model reasoning operation is avoided, server resources are saved, many unnecessary expenses are avoided, the real-time performance of fish school track prediction in the actual culture process is improved, the deployment is convenient, the culture cost is reduced, and the application range is wide.
In one embodiment, the user terminal device is a server. Or the edge computing main board and the user terminal device can be in wireless connection (such as WIFI) or wired connection, and the details are not repeated herein for the prior art. After the early warning information is checked by a user, the abnormal conditions of the fish school can be further processed in time (such as feeding, oxygenation, water changing and the like), the loss caused by untimely abnormal discovery is avoided to a great extent, the culture quality and the survival rate are improved, and compared with the traditional method, the method is higher in efficiency and lower in cost, and is particularly suitable for large-scale deployment of large-scale fish aquaculture enterprises.
In one embodiment, the fish school centroid coordinates are obtained by a weighted average method according to the position information of each fish in each frame of image. Or may be calculated by a mathematical average method or the like.
In an embodiment, the data receiving unit is further configured to perform preprocessing on the extracted image, where the preprocessing includes sequentially performing compression processing on the image and removing a background based on a gaussian mixture model. And compressing the image according to a preset proportion, for example, compressing the image width to 0-1 to obtain a corresponding compressed image. Detecting the image through a Yolov5 network model to obtain a detection frame and corresponding position information [ x, y, w, h ] of each fish, wherein (x, y) is the center coordinate of the detection frame after image compression, w is the width of the detection frame divided by the width after image compression, and h is the height of the detection frame divided by the height after image compression. In the prior art, the requirement for detecting the real underwater fish school target is difficult to meet under the condition of changing the background light of the picture, the foreground and the background are distinguished according to the change rate of the pixel points through a background removing method based on a Gaussian mixture model, preprocessing is carried out before the image is input into a YOLOv5 model, the influence of the image background change factor on the model prediction accuracy is removed, the accuracy of fish school target detection is improved, and a more accurate track monitoring result is obtained. Background removal methods based on the Gaussian mixture model are common means in the prior art and are not described herein again.
In one embodiment, the predetermined threshold is 25%.
In one embodiment, the edge computing motherboard is a jetson nano motherboard. The edge computing mainboard adopts a jetson nano mainboard, has higher computing speed, or can also adopt a mainboard in the prior art.
The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.
The above-mentioned embodiments only express the more specific and detailed embodiments described in the present application, but not be construed as limiting the claims. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, and these are all within the scope of protection of the present application. Therefore, the protection scope of the present patent shall be subject to the appended claims.
Claims (6)
1. The utility model provides a fish school orbit monitoring system based on edge calculation which characterized in that: the fish school track monitoring system based on edge calculation comprises user terminal equipment and at least one edge device, wherein:
the edge device comprises a camera and an edge calculation main board, the camera is used for collecting fish school movement videos, the edge calculation main board comprises a data receiving unit, a detection unit and a data analysis unit which are connected in sequence, and the edge calculation main board comprises:
the data receiving unit is used for receiving the fish school motion video and extracting images;
the detection unit is used for deploying a pre-trained YOLOv5 network model, detecting an image through the YOLOv5 network model, and obtaining a detection frame and corresponding position information [ x, y, w, h ] of each fish, wherein (x, y) is the center coordinate of the detection frame, namely the centroid coordinate of the fish, w is the width of the detection frame, and h is the height of the detection frame;
the data analysis unit is used for obtaining the fish school centroid coordinates of each frame of image and executing the following operations:
calculating the fish school displacement sum S in the Kth periodKThe following formula is satisfied:
wherein (x)i,yi) The fish school centroid coordinate of the ith frame image in the Kth period is shown, n is the total number of images in the Kth period, and K is 1,2,3 and …;
judging whether the difference between the fish school displacement sum of two adjacent periods is larger than a preset threshold value, if so, determining that the fish school movement track is abnormal, and issuing early warning information to the user terminal equipment, otherwise, determining that the fish school movement track is normal, and not issuing the early warning information to the user terminal equipment;
and the user terminal equipment is used for receiving the early warning information sent by the data analysis unit of each edge computing main board.
2. The edge-computing-based fish school trajectory monitoring system of claim 1 wherein: the user terminal equipment is a server.
3. The edge-computing-based fish school trajectory monitoring system of claim 1 wherein: and the fish school centroid coordinates are obtained by adopting a weighted average method according to the position information of each fish in each frame of image.
4. The edge-computing-based fish school trajectory monitoring system of claim 1 wherein: the data receiving unit is further used for preprocessing the extracted image, and the preprocessing comprises the steps of sequentially compressing the image and removing the background based on a Gaussian mixture model.
5. The edge-computing-based fish school trajectory monitoring system of claim 1 wherein: the preset threshold is 25%.
6. The edge-computing-based fish school trajectory monitoring system of claim 1 wherein: the edge computing mainboard is a jetson nano mainboard.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210028326.XA CN114596625A (en) | 2022-01-11 | 2022-01-11 | Fish school track monitoring system based on edge calculation |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210028326.XA CN114596625A (en) | 2022-01-11 | 2022-01-11 | Fish school track monitoring system based on edge calculation |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114596625A true CN114596625A (en) | 2022-06-07 |
Family
ID=81803484
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210028326.XA Pending CN114596625A (en) | 2022-01-11 | 2022-01-11 | Fish school track monitoring system based on edge calculation |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114596625A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115690570A (en) * | 2023-01-05 | 2023-02-03 | 中国水产科学研究院黄海水产研究所 | Fish shoal feeding intensity prediction method based on ST-GCN |
-
2022
- 2022-01-11 CN CN202210028326.XA patent/CN114596625A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115690570A (en) * | 2023-01-05 | 2023-02-03 | 中国水产科学研究院黄海水产研究所 | Fish shoal feeding intensity prediction method based on ST-GCN |
| CN115690570B (en) * | 2023-01-05 | 2023-03-28 | 中国水产科学研究院黄海水产研究所 | Fish shoal feeding intensity prediction method based on ST-GCN |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11410035B2 (en) | Real-time target detection method deployed on platform with limited computing resources | |
| CN109934805B (en) | Water pollution detection method based on low-illumination image and neural network | |
| CN108805083A (en) | The video behavior detection method of single phase | |
| CN109376589A (en) | ROV deformation target and Small object recognition methods based on convolution kernel screening SSD network | |
| CN109583355B (en) | People flow counting device and method based on boundary selection | |
| CN206281972U (en) | Maritime radar monitoring system | |
| CN109492618A (en) | Object detection method and device based on grouping expansion convolutional neural networks model | |
| CN111008994A (en) | Moving target real-time detection and tracking system and method based on MPSoC | |
| CN112131927A (en) | Sow delivery time prediction system based on posture transformation characteristics in later gestation period | |
| CN110110627A (en) | A method of the real-time target detection towards computing resource limitation Platform deployment | |
| Chen et al. | An intelligent sewer defect detection method based on convolutional neural network | |
| CN112349057A (en) | Deep learning-based indoor smoke and fire detection method | |
| CN115937251A (en) | Multi-target tracking method for shrimps | |
| CN114596625A (en) | Fish school track monitoring system based on edge calculation | |
| CN111860483B (en) | Target detection method based on Haisi platform | |
| CN116016869A (en) | Campus safety monitoring system based on artificial intelligence and Internet of things | |
| CN114565942A (en) | Live pig face detection method based on compressed YOLOv5 | |
| CN116402769A (en) | High-precision intelligent detection method for textile flaws considering size targets | |
| CN114863554A (en) | Sign language recognition system and method based on deep learning model | |
| CN112883755A (en) | Smoking and calling detection method based on deep learning and behavior prior | |
| CN115810163B (en) | Teaching evaluation method and system based on AI classroom behavior recognition | |
| CN110930436A (en) | Target tracking method and device | |
| Liu et al. | Cloud-edge-end cooperative detection of wind turbine blade surface damage based on lightweight deep learning network | |
| CN110555384A (en) | Beef marbling automatic grading system and method based on image data | |
| CN116189076A (en) | Observation and identification system and method for bird observation station |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |