CN117197123A - Fish shoal counting method based on dynamic visual image multi-mode fusion - Google Patents

Fish shoal counting method based on dynamic visual image multi-mode fusion Download PDF

Info

Publication number
CN117197123A
CN117197123A CN202311305616.5A CN202311305616A CN117197123A CN 117197123 A CN117197123 A CN 117197123A CN 202311305616 A CN202311305616 A CN 202311305616A CN 117197123 A CN117197123 A CN 117197123A
Authority
CN
China
Prior art keywords
fish
zebra fish
video
mode
counting
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202311305616.5A
Other languages
Chinese (zh)
Inventor
黎明
陈强华
王彗瑜
赵建鼎
赵莹
李靖超
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Dianji University
Original Assignee
Shanghai Dianji University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Dianji University filed Critical Shanghai Dianji University
Priority to CN202311305616.5A priority Critical patent/CN117197123A/en
Publication of CN117197123A publication Critical patent/CN117197123A/en
Pending legal-status Critical Current

Links

Classifications

    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A40/00Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
    • Y02A40/80Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
    • Y02A40/81Aquaculture, e.g. of fish

Landscapes

  • Image Processing (AREA)

Abstract

The invention discloses a fish swarm counting method based on dynamic visual image multi-mode fusion, which belongs to the technical field of computer vision and comprises the steps of zebra fish data acquisition; processing the collected zebra fish video; inputting the processed zebra fish video into a YOLOv5 network for counting, and outputting and storing the zebra fish quantity value; processing and outputting the saved zebra fish quantity value by using a data processing mechanism; outputting the counted zebra fish quantity. According to the fish shoal counting method based on the dynamic visual image multi-mode fusion, the dynamic visual sensor and the computer are erected outside the culture cylinder, the internal environment of the culture cylinder does not need to be changed when the number of the zebra fish is counted, the number change of the zebra fish can be well mastered, the detection speed is high, and the accuracy is high.

Description

Fish shoal counting method based on dynamic visual image multi-mode fusion
Technical Field
The invention relates to the technical field of computer vision, in particular to a fish swarm counting method based on multi-mode fusion of dynamic visual images.
Background
Patent publication number CN115294067a proposes a method for detecting the number of fries based on volume. The method designs a fish fry counting container, the volume of a single fish fry is calculated in advance in the counting process, and a part of water is added into the container through a hole to enable the liquid level to reach a position with scales, and the scale value at the position is recorded at the moment and is used as a first scale value; then, the fries are poured into the container from the open position at the top of the cylindrical long tube, the liquid level rises, the scale value at the moment is recorded and used as a second scale value, and then the fries number= (second scale value-first scale value)/the volume of single fries. This approach requires that the individual fish volumes be known in advance and that the volumes of each fish be relatively even. However, the volume of fish in the culture tank changes with time, so that the volume of fish in the tank cannot be known in advance. And the growth state of each fish is different, and the volume of each fish cannot be ensured to be approximately equal. Therefore, the method of calculating the number of fish shoals by volume is not applicable to the aquarium fish shoals count.
Patent publication number CN110692574a proposes a fry counting method based on an optical detection device. The detection device comprises a fry water tank to be detected, a water path straight pipe, two optical detection devices and a central computing device, wherein the fry water tank to be detected is used for placing water and a plurality of fries, the water path straight pipe is used for enabling the fries to flow out of the fry water tank to be detected, the two optical detection devices are arranged outside the water path straight pipe front and back, and the central computing device is connected with the two optical detection devices. During detection, the fish fry to be detected enters the straight pipe of the waterway. The first optical detection device continuously projects a first optical signal towards the waterway straight pipe and receives a first reflection signal, and the second optical detection device continuously projects a second optical signal towards the waterway straight pipe and receives a second reflection signal. The central computing device receives the first reflected signal and the second reflected signal, detects the first signal and the second signal which are respectively shielded by the fries and attenuate the signal intensity, and judges the fries to pass through the waterway straight pipe and records the quantity of the fries according to the time difference between the first signal and the second signal. The method needs to enable the fish fry to be tested to pass through the straight pipe of the waterway. The number of the fish shoals in the culture cylinder needs to be counted for many times to master the number change of the fish shoals in a period of time, if all the fishes need to be driven to the straight pipe of the waterway during each counting, the workload is large, and the fish shoals are easy to damage during the process of transferring many times. Therefore, the culture tank does not have the conditions for carrying out the optical detection method.
Patent publication number CN106204626a puts the fry that needs the count into the count case, and the top of count case is equipped with camera device, camera device is connected with the computer. The bottom plate of the counting box is provided with a plurality of water inlets, the side wall of the counting box is provided with at least one overflow port, and a plurality of water inlets form a plurality of vertical upward water flows in the counting box. Under the action of water flow, fries can be evenly spread on a layer of water surface, and the number of fries in the counting box can be accurately counted by matching an image processing-based fries counter with proper algorithm compensation. However, the counting condition needs to be artificially laid on the water surface without overlapping the fish shoals to be measured, and a water flow control device is arranged in the fish tank to realize the condition. When the quantity of the culture cylinders is large, if additional devices are arranged for each cylinder, the required cost is high, and the original culture space can be occupied.
Patent publication number CN207940236U proposes a method for detecting the number of fish shoal based on infrared detection. The infrared emission receivers are arranged at the four right-angle corners and the water inlet positions in the fish tank, when fish passes through the positions of the infrared rays, the infrared rays are shielded, the infrared signals cannot be received by the infrared receiving circuit, the amplifying circuit outputs high level signals and sends the level signals to the first singlechip for controlling and counting, and then the fish quantity display displays the quantity of the fish. However, this method can only roughly count the number of fish shoal near the infrared emitter, and although it can ensure that each fish is detected at an indefinite time, it cannot detect the number of fish shoal in the whole cylinder at the same time. The detection of fish shoal in a culture tank requires knowledge of the total number of fish shoals in the tank, and thus a method that only roughly estimates the number of fish shoals around the detector is not applicable.
Patent publication number CN116569875a proposes a double-layer high-flux fish counting device and method based on machine vision. The device comprises a front channel, a double-layer counting channel and a double-layer fish outlet. The double-layer counting channel is provided with an image acquisition module, and the counting method is to divide a foreground region, extract characteristics, determine the quantity of fish and position according to the difference between an image and a background model. When counting, the fish shoal to be measured enters the double-layer counting channel from the front channel to finish counting, and the fish shoal after counting exits from the double-layer fish outlet. And during counting, the foreground region is segmented and the characteristics are extracted according to the difference between the image and the background model, so that the quantity of fishes is determined. The counting method needs to enable the fish shoal to be detected to pass through a specific device, is suitable for calculating a large amount of fishes at one time, and is not suitable for continuously monitoring the quantity change of the fish shoal. If the method is used for continuously monitoring the number of the zebra fish in the culture cylinder, the detection workload is large each time, and the zebra fish can be damaged in the detection process.
Paper automatic counting System for zebra fish in fish pond uses pure blue background plate, and under the condition of light supplementing, a camera is used for shooting zebra fish in a fish tank. After shooting is completed, zebra fish in the picture are extracted from the background by using a Gaussian mixture model (Gaussian Mixture Model, GMM) and a Blob counting mode, and each zebra fish is segmented to realize counting. The scheme separates the photographed zebra fish from the background and counts each fish after segmentation. However, the method needs a solid background, and light supplementing is adopted in experiments in papers; meanwhile, the edge of a shot picture needs to be aligned with the edge of the fish tank, and if other contents are counted, the fish is mistakenly identified. The requirement of the solid background and the light supplementing lamp limits the use scene of counting, and the requirement of the edge of the shot picture to be aligned with the edge of the fish tank increases the shooting difficulty during counting.
Disclosure of Invention
The invention aims to provide a fish swarm counting method based on dynamic visual image multi-mode fusion, which uses a dynamic visual sensor to collect images and fuse image information under multiple modes of the dynamic visual sensor, so that the utilization rate of collected information is improved, the internal environment of a culture cylinder is not required to be changed when the number of zebra fishes is counted, the number change of the zebra fishes can be better mastered, the detection speed is high, and the accuracy is high.
In order to achieve the above purpose, the invention provides a fish swarm counting method based on dynamic visual image multi-mode fusion, comprising the following steps:
s1, zebra fish data acquisition is carried out;
s2, processing the collected zebra fish video;
s3, inputting the processed zebra fish video into a YOLOv5 network for counting, and outputting and storing the zebra fish quantity value;
s4, processing and outputting the saved zebra fish quantity value by using a data processing mechanism;
s5, outputting the counted zebra fish quantity.
Preferably, the step S1 includes:
s11, a dynamic vision sensor collects the picture of the whole culture cylinder;
s12, after the acquisition is finished, the dynamic vision sensor generates a bin file containing event stream information;
s13, converting the bin file into a video of Binary, count, gray, accumulated modes by the dynamic vision sensor, wherein Binary images of the dynamic vision sensor are acquired by the Binary mode, statistical images of the dynamic vision sensor are acquired by the Count mode, gray images of the dynamic vision sensor are acquired by the Gray mode, and Gray Accumulated images of the dynamic vision sensor are acquired by the Accumulated mode.
Preferably, the step S2 includes:
s21, changing white (rgb= (255, 255)) in the Binary mode video information to green (rgb= (0,255,0));
s22, converting the transparency value of a black (RGB= (0, 0)) part in videos of the three modes of the Binary, the Count and the Gray acquired by the dynamic vision sensor into 0, and overlapping the transparency value on the Accumulated mode to generate a mixed video mixing information of the four modes.
Preferably, the black part (rgb= (0, 0)) is a part without information in the zebra fish video of the three modes of the Binary, the Count and the Gray.
Preferably, the step S3 includes:
s31, converting the zebra fish video into images, wherein the images are training sets;
s32, labeling zebra fish in the training set by using labelme;
s33, putting the training set into a YOLOv5 network for training, and obtaining a pre-training model capable of identifying the zebra fish after training is completed;
s34, randomly selecting a zebra fish video of 1 second as a video to be detected, wherein the 1 second is 30 frames, inputting the video to be detected into a trained zebra fish identification model to count, and outputting and storing the number value of each frame of zebra fish.
Preferably, the step S4 includes:
s41, averaging the saved 30-frame zebra fish quantity values;
s42, carrying out upward rounding treatment on the average value;
s43, outputting the processed numerical value.
Therefore, the fish swarm counting method based on the dynamic visual image multi-mode fusion has the following beneficial effects:
(1) The method uses the dynamic vision sensor to collect images and fuse the image information under multiple modes of the dynamic vision sensor, the dynamic vision sensor records the change of pixel points, and compared with a traditional camera for recording the whole picture, the dynamic vision sensor is more sensitive to moving objects in the picture, and can better capture moving fishes; the imaging modes of the dynamic vision sensor are fused, so that more information can be acquired, and the detection and counting accuracy can be improved;
(2) The shooting mode is to shoot the whole culture tank from the side surface, the operation of foreground extraction is not needed, the fish shoals can be counted in an environment without a solid background, the shooting dead angle is reduced, and the original state of the culture tank is not needed to be changed;
(3) According to the method, the characteristics of the zebra fish are learned through model training, after the model training is completed, the zebra fish identification capability is provided, the interference caused by other sundries can be effectively reduced, in addition, during the training, zebra fish samples with different sizes are considered, the model can identify the zebra fish with different sizes in various body shapes, and the application range is wide;
(4) The dynamic vision sensor and the computer are erected outside the culture cylinder, equipment does not need to be added in the culture cylinder, the original space in the culture cylinder does not need to be occupied, the original living environment of the zebra fish cannot be changed, the zebra fish cannot be damaged, the culture cylinder is convenient to count for many times, and the quantity change of the zebra fish is monitored for a long time.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a flow chart of a method for counting fish shoal based on multi-mode fusion of dynamic visual images according to an embodiment of the invention;
fig. 2 is a zebra fish image collected by an embodiment of a fish swarm counting method based on multi-mode fusion of dynamic visual images, a is a collected Binary pattern diagram, b is a collected Count pattern diagram, c is a collected Gray pattern diagram, and d is a collected Accumulated pattern diagram;
FIG. 3 is a video processing flow chart of an embodiment of a fish school counting method based on multi-mode fusion of dynamic visual images according to the invention;
FIG. 4 is a diagram showing a mixed video effect of an embodiment of a fish school counting method based on multi-mode fusion of dynamic visual images;
FIG. 5 is a flow chart of a data processing mechanism of an embodiment of a fish school counting method based on multi-mode fusion of dynamic visual images;
fig. 6 is a diagram of a result of detection of an embodiment of a fish school counting method based on multi-mode fusion of dynamic visual images.
Detailed Description
Examples
The invention provides a fish swarm counting method based on dynamic visual image multi-mode fusion, which comprises the following steps with reference to fig. 1-6:
and S1, performing zebra fish data acquisition. When data are collected, the dynamic vision sensor collects the whole culture cylinder picture, and the information collected by the dynamic vision sensor is illumination change of each pixel instead of the whole image. The dynamic vision sensor takes pixels as units, if the brightness change of the pixels exceeds a set threshold value, the dynamic vision sensor records an event, the brightness is increased to an on event, the brightness is reduced to an off event, and the dynamic vision sensor can record gray information. After the acquisition is finished, the dynamic vision sensor generates a bin file containing event stream information; the dynamic vision sensor converts the bin file into a video of Binary, count, gray, accumulated in four modes, and Binary images of the dynamic vision sensor are acquired in a Binary mode, namely the gray value of a triggered pixel position is 255, and the gray value of an untriggered pixel position is 0; the Count mode collects statistical images of the dynamic vision sensor, namely the gray value of the triggered pixel position is the number of times the pixel is triggered, and the gray value of the non-triggered pixel position is 0; the Gray mode acquires a Gray image of the dynamic vision sensor, namely the Gray value of the triggered pixel position is the Gray value returned by the dynamic vision sensor, and the Gray value of the non-triggered pixel position is 0. The Accumulated mode collects the gray level Accumulated image of the dynamic vision sensor, that is, only the gray level value of the triggered pixel position is updated each time, and the gray level value of the non-triggered pixel position is kept unchanged.
S2, processing the acquired zebra fish video, wherein the video contains various morphological characteristics of the zebra fish, and the step of processing the video comprises the following steps:
s21, only two Gray values of 0 and 255 are adopted in the Binary mode, and in order to more prominently embody the information of the Binary mode in the mixed video, the Gray values of the Binary mode and the Count mode are distinguished at the same time, and the original white (RGB= (255, 255)) of the information in the Binary mode video is changed into green (RGB= (0,255,0));
s22, in the video in the Binary, count, gray three modes acquired by the dynamic vision sensor, the part without information defaults to black (RGB= (0, 0)). The transparency value of the black (rgb= (0, 0)) portion of the three modes is converted into 0, and superimposed on the Accumulated mode, a mixed video in which four mode information is mixed is generated.
Step S3, inputting the processed zebra fish video into a YOLOv5 network for counting, and outputting and storing the zebra fish quantity value, wherein the method specifically comprises the following steps of:
s31, converting the zebra fish video into images, wherein the images are training sets;
s32, labeling zebra fish in the training set by using labelme;
s33, putting the training set into a YOLOv5 network for training, and obtaining a pre-training model capable of identifying the zebra fish after training is completed;
s34, randomly selecting a zebra fish video of 1 second (fps=30) as a video to be detected, inputting the video to be detected into a trained zebra fish identification model to count, and outputting and storing the number value of each frame of zebra fish.
Step S4, processing and outputting the saved zebra fish quantity value by using a self-defined data processing mechanism, wherein the processing steps comprise:
s41, averaging the saved 30-frame zebra fish quantity values;
s42, carrying out upward rounding treatment on the average value;
s43, outputting the processed numerical value.
And S5, outputting the counted number of the zebra fish.
The scheme provides a data processing mechanism. When the model is identified and counted, a plurality of false identification conditions exist, for example, the fishes are mutually blocked, and when the blocked fishes are not identified, the result of the counting is smaller than a true value; when fish moves close to the water surface and the reflection is mistakenly identified as fish, the counting result is larger than the true value. The method of continuously collecting counting results for 1 second (fps=30) to average and rounding the average upwards can effectively reduce interference caused by false recognition and effectively improve counting accuracy.
Example 1
Randomly selecting 20 zebra fish, placing the zebra fish into a culture jar with the size of 22cm x 16cm x 17cm, and randomly acquiring 100 seconds (fps=30) of video by using a dynamic vision sensor. The 100 second video was duplicated for correction. The acquired videos are counted by using the trained model and the data processing mechanism of the scheme, so that 100 counting results are obtained, and the average accuracy of the 100 results is 97%.
When the scheme is used for counting, only the dynamic visual sensor and the computer are required to be erected outside the culture cylinder, and the original state of the culture cylinder cannot be changed. Meanwhile, the method can be applied to other small fish counting scenes.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting it, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that: the technical scheme of the invention can be modified or replaced by the same, and the modified technical scheme cannot deviate from the spirit and scope of the technical scheme of the invention.

Claims (6)

1. A fish swarm counting method based on dynamic visual image multi-mode fusion is characterized by comprising the following steps:
s1, zebra fish data acquisition is carried out;
s2, processing the collected zebra fish video;
s3, inputting the processed zebra fish video into a YOLOv5 network for counting, and outputting and storing the zebra fish quantity value;
s4, processing and outputting the saved zebra fish quantity value by using a data processing mechanism;
s5, outputting the counted zebra fish quantity.
2. The method for fish school counting based on multi-mode fusion of dynamic visual images according to claim 1, wherein said step S1 comprises:
s11, a dynamic vision sensor acquires the picture of the whole zebra fish culture tank;
s12, after the acquisition is finished, the dynamic vision sensor generates a bin file containing event stream information;
s13, converting the bin file into a video of Binary, count, gray, accumulated modes by the dynamic vision sensor, wherein Binary images of the dynamic vision sensor are acquired by the Binary mode, statistical images of the dynamic vision sensor are acquired by the Count mode, gray images of the dynamic vision sensor are acquired by the Gray mode, and Gray Accumulated images of the dynamic vision sensor are acquired by the Accumulated mode.
3. The method for fish school counting based on multi-mode fusion of dynamic visual images according to claim 1, wherein said step S2 comprises:
s21, changing white in the Binary mode video information into green;
s22, converting the transparency value of the black part in the videos of the three modes of the Binary, the Count and the Gray acquired by the dynamic vision sensor into 0, and overlapping the transparency value on the Accumulated mode to generate a mixed video with mixed information of the four modes.
4. A method for fish school count based on multi-mode fusion of dynamic visual images as defined in claim 3, wherein: the black part is a part without information in the zebra fish video in three modes of the Binary, the Count and the Gray.
5. The method for fish school counting based on multi-mode fusion of dynamic visual images according to claim 1, wherein said step S3 comprises:
s31, converting the zebra fish video into images, wherein the images are training sets;
s32, labeling zebra fish in the training set by using labelme;
s33, putting the training set into a YOLOv5 network for training, and obtaining a pre-training model capable of identifying the zebra fish after training is completed;
s34, randomly selecting a zebra fish video of 1 second as a video to be detected, wherein the 1 second is 30 frames, inputting the video to be detected into a trained zebra fish identification model for counting, and outputting and storing the number value of each frame of zebra fish.
6. The method for fish school counting based on multi-mode fusion of dynamic visual images according to claim 1, wherein said step S4 comprises:
s41, averaging the stored number values of 30 frames of zebra fish;
s42, carrying out upward rounding treatment on the average value;
s43, outputting the processed numerical value.
CN202311305616.5A 2023-10-10 2023-10-10 Fish shoal counting method based on dynamic visual image multi-mode fusion Pending CN117197123A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202311305616.5A CN117197123A (en) 2023-10-10 2023-10-10 Fish shoal counting method based on dynamic visual image multi-mode fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202311305616.5A CN117197123A (en) 2023-10-10 2023-10-10 Fish shoal counting method based on dynamic visual image multi-mode fusion

Publications (1)

Publication Number Publication Date
CN117197123A true CN117197123A (en) 2023-12-08

Family

ID=88988784

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202311305616.5A Pending CN117197123A (en) 2023-10-10 2023-10-10 Fish shoal counting method based on dynamic visual image multi-mode fusion

Country Status (1)

Country Link
CN (1) CN117197123A (en)

Similar Documents

Publication Publication Date Title
CN101944267B (en) Smoke and fire detection device based on videos
CN110691193B (en) Camera switching method and device, storage medium and electronic equipment
CN110414334A (en) A kind of smart water quality recognition methods maked an inspection tour based on unmanned plane
CN102196182A (en) Backlight detection equipment and method
CN102547079B (en) Night scene light source detection device and method
CN109726796A (en) A kind of farm's management method neural network based
CN108921099A (en) Moving ship object detection method in a kind of navigation channel based on deep learning
US11200681B2 (en) Motion detection method and motion detection system with low computational complexity and high detection accuracy
CN112508033B (en) Detection method, storage medium, and electronic apparatus
CN108182682A (en) A kind of detection method based on Computer Image Processing and pattern-recognition
CN111539311A (en) Living body distinguishing method, device and system based on IR and RGB double photographing
CN114170549A (en) Water surface floater detection method based on deep learning
CN105405145B (en) A kind of granule number grain method based on image Segmentation Technology
CN113225461A (en) System and method for detecting video monitoring scene switching
CN108710852A (en) It is a kind of to limit the size distribution image recognition methods for shooting depth and system
CN111521245A (en) Liquid level control method based on visual analysis
CN117197123A (en) Fish shoal counting method based on dynamic visual image multi-mode fusion
CN210028839U (en) Article taking and placing detection system
CN106546196A (en) A kind of optical axis real-time calibration method and system
CN115801942A (en) Automatic detection system of cell-phone
CN114782368A (en) Shrimp larva automatic counting method and system based on visual detection and fusion of multiple dynamic thresholds
CN111027560B (en) Text detection method and related device
CN208370199U (en) A kind of metering detection system based on computer vision
CN114757963A (en) Moving target extraction method under complex illumination change at night
CN109919863B (en) Full-automatic colony counter, system and colony counting method thereof

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination