CN116267753A

CN116267753A - Fry classifying and counting device and method based on vision and touch sense

Info

Publication number: CN116267753A
Application number: CN202310290442.3A
Authority: CN
Inventors: 李磊; 时国胜; 许家伟
Original assignee: Jiangsu University of Science and Technology
Current assignee: Jiangsu University of Science and Technology
Priority date: 2023-03-23
Filing date: 2023-03-23
Publication date: 2023-06-23
Anticipated expiration: 2043-03-23
Also published as: CN116267753B

Abstract

The invention discloses a fish fry classifying and counting device and method based on vision and touch, wherein 3 types of fish fries are distributed in a fry pond in a layered manner, an image acquisition camera is arranged in the classifying and counting pond, three unidirectional channel pipes corresponding to the 3 types of fish fries in a layered manner are arranged in the classifying and counting pond, the three types of fish fries enter a feeding pond through the three unidirectional channel pipes respectively, a touch counting device is arranged on the unidirectional channel pipes, and a real-time detection camera is arranged in the classifying and counting pond; the controller trains according to the images and video data acquired by the image acquisition camera to obtain a network model, and classifies and identifies three unidirectional channel intra-tube video data acquired by real-time detection and shooting to obtain the number of other two types of fish fries which are mistakenly input into each unidirectional channel; the controller collects the trigger signal of each touch counting device and records the corresponding touch counting value; and calculating according to the touch count value and the false entry value to obtain the total number of various fries. The invention combines visual counting and tactile counting, and improves the detection efficiency and the result accuracy.

Description

Fry classifying and counting device and method based on vision and touch sense

Technical Field

The invention relates to a variety of fry counting technologies which are applicable to intelligent cultivation of deep sea cultivation workers and boats and are oriented to the field of deep sea cultivation, in particular to a fry classifying and counting device and method based on vision and touch.

Background

With the development of deep sea aquaculture, the deep sea aquaculture industry in China is focused on, and the deep sea aquaculture is not dependent on manual management, so that the development of mechanical and intelligent fishery aquaculture is more advanced. However, in the aspect of counting the fish fries cultivated in deep sea, the traditional counting method mostly adopts manual counting, which is time-consuming and labor-consuming, has large errors, and is used for deep sea-oriented cultivation, and the fish fries are firstly classified and counted in a huge quantity and possibly in 2 to 3 or more types, and are unlikely to be realized only by manpower.

At present, a plurality of fry counting methods based on photoelectric counting, vision counting and image segmentation are mainly used, and the methods are not very mature and have some difficulties to be improved in fry counting application, on one hand, when the former two methods are used for counting, because the quantity of fries is large, the problems of stacking up and down, overlapping front and back, jamming of a fish inlet, repeated counting and the like easily occur when the fries pass through a counting area, so that the counting result of a detection device is greatly influenced, and the latter method is used for carrying out area calculation on the image segmentation process, and if the overlapped fries are encountered, the precise counting can not be carried out well; on the other hand, deep sea cultivation is conducted on fries of 2 to 3 types or more, the number of fries of each type is difficult to obtain, and in many technical devices and methods, the number of fries is counted for the whole fish shoal, and some structural devices and methods for classifying and counting different fishes are not formed.

Disclosure of Invention

The invention aims to: it is an object of the present invention to provide a visual and tactile based fry sorting and counting device which is suitable for sorting and counting of 3 fry types.

The invention further aims to provide a fish fry classifying and counting method based on vision and touch, which utilizes the YOLOv5 algorithm in vision and touch to jointly count, so that the purposes of improving the detection efficiency and the result accuracy and also achieving good classifying and counting of fish fries of different types are achieved.

The technical scheme is as follows: the invention relates to a fish fry classifying and counting device based on vision and touch, which comprises a seedling raising pool, a classifying and counting pool, a feeding pool, a fish fry stagnation acceleration prevention device and a controller, wherein I-class fish fry, II-class fish fry and III-class fish fry are respectively distributed from the upper layer to the lower layer in the seedling raising pool, and an image acquisition camera is arranged in the seedling raising pool; the classifying and counting pool is separated from the seedling raising pool and the feeding pool through a first-stage separator and a second-stage separator respectively, three unidirectional channel pipes are arranged between the first-stage separator and the second-stage separator in the classifying and counting pool, the three unidirectional channel pipes correspond to habit distribution of the fries in each layer, the seedling raising pool and the feeding pool are communicated through the three unidirectional channel pipes, a first-stage self-adaptive net mouth device and a second-stage self-adaptive net mouth device are sequentially arranged in the unidirectional channel pipes from the seedling raising pool to the feeding pool, and a touch counting device is arranged on the unidirectional channel pipes above the second-stage self-adaptive net mouth device; the fry stagnation acceleration prevention device comprises a reservoir, three hoses and three control valves, wherein the reservoir is arranged at the top end of the first-stage partition plate and is respectively communicated with three one-way channel pipes through the three hoses, the communication position of the hoses and the one-way channel pipes is positioned above the outlet of the second-stage self-adaptive net mouth device, and the control valves are arranged on the hoses; the classification counting pool is also internally provided with a real-time detection camera which is used for collecting video data in three unidirectional channel pipes in real time;

The controller comprises a data processing module, a model training module, a classification counting module and a control module, wherein an image acquisition camera acquires images and video data of three types of fish fries in a seedling raising pool and uploads the images and the video data to the data processing module, the data processing module intercepts the video data into image data frame by frame, marks the three types of fish fries in all the image data respectively to form a data set, and the model training module adopts the data set training to obtain a YOLOv5 network model; the three types of fries trigger corresponding touch counting devices in the process of entering the feeding pool through three unidirectional channel pipes respectively, and the touch counting devices upload signals to the classification counting module to perform initial classification counting; the real-time detection camera transmits video data of three types of fries in three unidirectional channel pipes acquired in real time to the controller, and the number of the fries in each type entering the other two unidirectional channel pipes by mistake is obtained through classification and identification of a YOLOv5 network model, and the classification counting module calculates a final classification counting result of the fries according to the initial classification counting result and the false entering number result; when the touch counting device is triggered, the control module controls the corresponding control valve to be opened, and when the touch counting device is not triggered, the control valve is closed, and the control module is also used for controlling each touch counting device, the image acquisition camera and the real-time detection camera.

Preferably, the upper part of the seedling raising pool is also provided with a water outlet.

Preferably, the touch counting device comprises a touch sensor, a telescopic contact device and a fixed sleeve, the fixed sleeve is fixedly communicated with the unidirectional channel pipe and is positioned above the secondary self-adaptive net mouth device, the telescopic contact device is embedded into the fixed sleeve, the lower end of the telescopic contact device stretches into the unidirectional channel pipe, the fry touches the telescopic contact device when passing through the secondary self-adaptive net mouth device, the telescopic contact device rises to contact the touch sensor, and the classification counting module receives the touched information of the touch sensor and records the touch times.

Preferably, the first-stage self-adaptive net mouth device and the second-stage self-adaptive net mouth device are larger in net mouth, smaller in net mouth, and close to the seedling raising pool, are net mouth, and close to the feeding pool, are net mouth.

Preferably, the upper part of the feeding pool is further provided with a feed feeding device, the feed feeding device comprises three groups of drawing plates, a feed bin, a material channel and a discharge port, the drawing plates isolate the feed bin from the material channel, the discharge port is communicated with the material channel, and feed in the feed bin of the drawing plates enters the feeding pool through the material channel through the discharge port.

Preferably, the bottom of the reservoir is also provided with a support plate, and the reservoir is further fixed with the first-stage baffle plate.

In another embodiment of the invention, a fish fry classifying and counting method based on vision and touch comprises the following steps:

s1, an image acquisition camera in a seedling raising pool acquires images and video data of three types of fish fries in the seedling raising pool, the data are uploaded to a controller, the controller intercepts the video data frame by frame to obtain image data, the acquired image data and the intercepted image data are preprocessed and marked, the marked image data are divided into a training set and a verification set, the training set and corresponding labels thereof are used for manufacturing the training data set according to a set format and a set sequence, and the verification set and corresponding labels thereof are used for manufacturing the verification data set according to the set format and the set sequence;

s2, inputting a training data set into the YOLOv5 network model for training to obtain a trained YOLOv5 network model, inputting a verification data set into the trained YOLOv5 network model for verification, if the verification is qualified, storing the trained YOLOv5 network model, if the verification is unqualified, modifying model training parameters on the basis of previous training to continue training until the network model meets the accuracy and speed of real-time detection, and storing the trained YOLOv5 network model;

S3, the touch counting device transmits the collected touch information to the controller, and the controller records the initial quantity of fish fries passing through the corresponding unidirectional channel tube according to the touch information of the touch counting device, namely a touch classification count value; meanwhile, the real-time detection camera transmits the real-time collected video data of the fries in the unidirectional channel pipes to the controller, and the trained YOLOv5 network model stored in the step S2 is utilized for classification and identification to obtain the number of other two types of fries wrongly entering each unidirectional channel pipe, namely a vision auxiliary classification count value; the touch sense classification count value corresponding to each channel tube subtracts the number of other two types of fries which enter the unidirectional channel tube by mistake and the number of the other two unidirectional channel tubes by mistake, so that the total number of the fries of each type is finally obtained.

Further, the method for preprocessing and labeling the image data in the step S1 is as follows:

screening all image data one by one, selecting image data with obvious characteristics, removing useless image data, performing dark channel defogging treatment on the image data to obtain a high-quality image, marking three fish fry targets on each piece of image data on labelimg marking software, storing the marked image data and exporting the marked image data into a standard VOC (volatile organic compound) format label, and further converting the VOC format label into a YOLO format label by using a python script program; and finally, dividing all marked image data and the corresponding label files into a training data set and a verification data set according to a proportion.

Further, in step S2, the YOLOv5 network model can frame the location of the fish in real time in the image data and display the fish category label at the upper left corner of the generated frame; when the I-class fish fry is detected, a target frame is generated on the I-class fish fry target in real time, and a label 'I' is displayed at the upper left corner of the target frame; when the II-class fish fry is detected, a target frame is generated on the II-class fish fry target in real time, and a label II is displayed at the upper left corner of the target frame; when the III fish fry is detected, a target frame is generated on the III fish fry target in real time, and a label III is displayed at the upper left corner of the target frame; and detecting that the colors of the fish fry generation target frames of each category are different, wherein the number of the generation frames is the number of the fish fries.

Further, the step S3 specifically includes:

s31, I fries, II fries and III fries enter corresponding first one-way channel pipes, second one-way channel pipes and third one-way channel pipes respectively, then enter a first-stage self-adaptive net mouth device to separate a large number of gathered fries, then enter a second-stage self-adaptive net mouth device to trigger respective touch sensors, and a controller records the triggering times A, B, C of three one-way channel in-pipe touch sensors respectively, namely the touch classification count value of the I fries in the first one-way channel pipe is A, the touch classification count value of the II fries in the second one-way channel pipe is B, and the touch classification count value of the III fries in the third one-way channel pipe is C;

S32, simultaneously, the real-time detection camera uploads the video data of the fries in the three unidirectional channel pipes detected in real time to the controller, and the controller processes the video data to obtain vision auxiliary count values of various fries; specifically, the quantities of class II fries and class III fries which are wrongly entered in the first unidirectional passage pipe are X and Y respectively, the quantities of class I fries and class III fries which are wrongly entered in the second unidirectional passage pipe are M and N, and the quantities of class I fries and class II fries which are wrongly entered in the third unidirectional passage pipe are P and Q;

s33, the controller calculates the total number of various fries according to the touch sense classification count value and the vision auxiliary classification count value, and specifically comprises the following steps: the total number of class I fries alpha= (A-X-Y) +M+P, the total number of class II fries beta= (B-M-N) +X+Q, and the total number of class III fries gamma= (C-P-Q) +Y+N.

The beneficial effects are that: compared with the prior art, the invention has the remarkable technical effects that:

(1) In the prior art, a specific method for attracting fries from one pool to another pool is not described, the fry classification and counting device of the invention attracts fries from a seedling raising pool to a feeding pool in a bait feeding mode, and a feed feeding device capable of simultaneously throwing various baits is designed, so that three fries which like different baits can be attracted to the feeding pool as much as possible, and the feasibility of the whole system is ensured.

(2) Aiming at the characteristic that fishes with different habits are distributed in a seedling raising pool in a layered manner, the three unidirectional channel pipes are respectively arranged in the counting pool from top to bottom and correspond to the distribution characteristic of each type of fishes, so that the three types of fishes can reach the feeding pool, and the reliability of the whole device is ensured.

(3) The self-adaptive net mouth device designed by the invention can only pass one fish at a time, and due to the self-adaptive advantage, the fish can be reduced to the original size after passing through the net mouth, the fish fry can not flow back and recoil after passing through the net mouth, the first-stage self-adaptive net mouth device avoids the phenomenon that a large number of fish fry are gathered and jammed before reaching a counting area, and can effectively disperse fish shoals to relieve the subsequent counting pressure, the second-stage self-adaptive net mouth device avoids the phenomenon that the fish fries are stacked up and down or overlapped front and back in the counting area to cause missing counting and repeated counting,

(4) The touch counting device designed by the invention has the advantages of strong practicality and adaptability, low cost and high counting efficiency.

(5) The fry stagnation acceleration prevention device can effectively prevent fries from stagnating and blocking in the channel, and improves the counting efficiency.

(6) The invention can detect the false entering information of the fish fry in real time by adopting a visual detection algorithm, has higher detection precision and detection speed, and can carry out classified counting on fishes with different habits by adopting a method for counting together with the touch counting device.

(7) According to the invention, aiming at the characteristic of low underwater lighting rate, the LED lamp is additionally arranged on the camera, so that the underwater light is enhanced, the quality of acquired images and video data is improved, and the real-time counting efficiency and the recognition accuracy can be improved.

Drawings

FIG. 1 is a schematic diagram of the structure of the device of the present invention;

FIG. 2 is a schematic diagram of a classification counting cell and its structure;

FIG. 3 is a schematic diagram of a self-adaptive net mouth, wherein (a) is a schematic diagram of a first net mouth of a fry, (b) is a schematic diagram of the fry supporting the net mouth to a maximum size, (c) is a schematic diagram of the fry beginning to retract from the net mouth, and (d) is a schematic diagram of the fry completely exiting from the net mouth;

FIG. 4 is a schematic diagram of a tactile sensation counting apparatus;

fig. 5 is a schematic diagram of a counting principle of the touch counting device, wherein (a) is a schematic diagram of a retractable contact device which is not touched by a fry completely entering a net opening, (b) is a schematic diagram of a retractable contact device touched by a fry exiting net opening, and (c) is a schematic diagram of a retractable contact device descending after the fry exits the net opening;

FIG. 6 is a schematic structural view of an anti-fry stagnation acceleration device;

FIG. 7 is a schematic diagram of the principle of preventing fry stagnation acceleration;

FIG. 8 is a schematic view of the feed feeding apparatus;

FIG. 9 is a schematic view of a drain opening;

FIG. 10 is a schematic diagram of the image acquisition camera and the real-time detection camera;

FIG. 11 is a flow chart of the method of the present invention;

FIG. 12 is a data set making flow chart;

FIG. 13 is a model training flow diagram;

FIG. 14 is a flow chart of a fish fry classification and counting method;

in the figure: 1. a seedling raising pool; 10. class I fries; 11. class II fries; 12. class III fish fries; 13. an image acquisition camera; 14. a water outlet; 2. classifying and counting the pool; 21. a first-stage separator; 22. a second-stage separator; 23. i a unidirectional channel pipe; 24. II is a unidirectional channel pipe; 25. III, a unidirectional channel tube; 26. a first-level self-adaptive network port device; 27. a second-level self-adaptive network port device; 28. a haptic counting device; 29. detecting a camera in real time; 131. a standard connector; 132. a housing; 133. a camera; an led lamp; 281. a tactile sensor; 282. a retractable contact arrangement; 283. a fixed sleeve; 3. feeding a pool; 4. an accelerating device for preventing fish fry stagnation; 41. a reservoir; 42. a first hose; 43. a second hose; 44. a third hose; 45. a first control valve; 46. a second control valve; 47. a third control valve; 48. a support plate; 5. a controller; 51. a control line; 6. a feed feeding device; 61. drawing a plate; 62. a storage bin; 63. a material channel; 64. and a discharge port.

Detailed Description

The invention will now be described in detail with reference to the drawings and specific examples.

The invention designs a fish fry counting device for deep sea cultivation, which can be applied to intelligent cultivation of deep sea cultivation workers and ships, is applicable to the classified counting of 3 fish fry types, and provides a counting method based on vision and touch, which can effectively reduce the problems of counting omission and repeated counting, can also improve the detection efficiency and the result accuracy, and can well perform classified counting on fishes with different habits.

As shown in fig. 1, the fish fry classifying and counting device based on vision and touch sense comprises a seedling raising pond 1, a classifying and counting pond 2, a feeding pond 3, an anti-fish fry stagnation accelerating device 4 and a controller 5, wherein I-class fish fries 10, II-class fish fries 11 and III-class fish fries 12 are respectively distributed from the upper layer to the lower layer in the seedling raising pond 1, an image acquisition camera 13 is further arranged in the seedling raising pond 1 and used for shooting image data and video data of each layer of fish fries in the fish fry pond 1, and a horizontal water outlet 14 is formed in the upper part of the side wall of the seedling raising pond; the seedling raising pool 1 and the feeding pool 3 are separated by a first-stage partition plate 21 and a second-stage partition plate 22, the area formed between the first-stage partition plate 21 and the second-stage partition plate 22 is a classification counting pool 2, the classification counting pool 2 is separated from the seedling raising pool 1 and the feeding pool 3 by the first-stage partition plate 21 and the second-stage partition plate 22 respectively, and three one-way channel pipes 23, 24 and 25 are arranged between the first-stage partition plate 21 and the second-stage partition plate 22 in the classification counting pool 2; the seedling raising pool 1 and the feeding pool 3 are communicated through three one-way channel pipes 23, 24 and 25, namely a No. I one-way channel pipe 23, a No. II one-way channel pipe 24 and a No. III one-way channel pipe 25 in sequence from top to bottom, wherein the three one-

way channel pipes

23, 24 and 25 correspond to habit distribution of each layer of fries, the No. I one-way channel pipe 23 corresponds to the upper layer of the seedling raising pool in which the type I fries 10 are distributed, the No. II one-way channel pipe 24 corresponds to the middle layer of the seedling raising pool in which the type II fries 11 are distributed, and the No. III one-way channel pipe 25 corresponds to the lower layer of the seedling raising pool in which the type III fries 12 are distributed; the top end of the first-stage partition plate 21 is provided with a fry stagnation acceleration prevention device 4, the fry stagnation acceleration prevention device 4 is communicated with three one-way channel pipes through three hoses, a first-stage self-adaptive net mouth device 26 and a second-stage self-adaptive net mouth device 27 are arranged in the three one-way channel pipes, and a touch counting device 28 is arranged on the one-way channel pipe above the second-stage self-adaptive net mouth device 27; the classifying and counting pool 2 is also internally provided with a real-time detection camera 29, and the real-time detection camera 29 can detect video images of various fish fries in three unidirectional channel pipes in the classifying and counting pool in real time; a feed feeding device 6 is arranged above the feeding pool 3; the controller 5 controls the fry stagnation acceleration preventing device 4, the tactile counting device 28, the image capturing camera 13 and the real-time detecting camera 29 through the control line 51.

The controller 5 comprises a data processing module, a model training module, a classification counting module and a control module, wherein the image acquisition camera 13 acquires images and video data of three types of fish fries in the seedling raising pond 1, the images and the video data are uploaded to the data processing module, the data processing module intercepts the video data into image data frame by frame, the three types of fish fries in all the image data are respectively marked to form a data set, and the model training module adopts data set training to obtain a YOLOv5 network model; the three types of fries trigger the corresponding touch counting device 28 in the process of entering the feeding pool through the three unidirectional channel pipes respectively, and the touch counting device 28 uploads signals to the classification counting module to perform initial classification counting; the real-time detection camera 29 transmits video data of three types of fries in three unidirectional channel pipes acquired in real time to the controller, and the video data are classified and identified through a YOLOv5 network model to obtain the number of the fries in each type of the other two unidirectional channel pipes by mistake, and the classification counting module calculates to obtain the final classification counting result of the fries according to the initial classification counting result and the false-in number result; when the touch counting device 28 is triggered, the control module controls the corresponding control valve to be opened, and when the touch counting device 28 is not triggered, the control valve is closed, and the control module is also used for controlling each touch counting device 28, the image acquisition camera 13 and the real-time detection camera 29.

If only two tanks (the classifying and counting tank is combined with the feeding tank) of the seedling raising tank and the feeding tank are designed, the fish fry can move back and forth in the feeding tank to influence the counting result, and in order to avoid the phenomenon, the classifying and counting tank is added between the seedling raising tank and the feeding tank for unidirectional diversion.

As shown in fig. 2, the classification counting pool 2 is formed by a first-stage partition plate 21 and a second-stage partition plate 22, a first-stage unidirectional channel pipe 23, a second-stage unidirectional channel pipe 24 and a third-stage unidirectional channel pipe 25 are sequentially arranged between the first-stage partition plate 21 and the second-stage partition plate 22 in the classification counting pool 2 from top to bottom, and a first-stage self-adaptive net mouth device 26 and a second-stage self-adaptive net mouth device 27 are sequentially arranged in each unidirectional channel pipe from the seedling raising pool 1 to the feeding pool 3.

Aiming at the characteristic that different fish fries are distributed in the upper, middle and lower three layers in different fish fry like ponds, three one-way channel pipes are designed at the first-stage partition plate and the second-stage partition plate of the classification counting pond, when baits are thrown into the feeding pond, three types of fish fries respectively enter the corresponding I-type one-way channel pipe 23, the corresponding II-type one-way channel pipe 24 and the corresponding III-type one-way channel pipe 25 to enter the classification counting pond, then enter the first-stage self-adaptive net mouth device to separate and disperse a large number of gathered fish fries, so that the phenomenon that a large number of fish fries gather and are jammed before reaching a counting area is avoided, the subsequent counting pressure can be effectively relieved, then the fish fries enter the second-stage self-adaptive net mouth device, the phenomenon that the fish fries are stacked up and down or overlapped front and back in the counting area to cause counting omission is avoided, and the fish fries reach the feeding pond after counting.

As shown in fig. 3, the entrance of the self-adaptive net mouth device is larger, the exit is smaller, and only one fish can pass through each time, and due to the self-adaptive advantage, the sizes of the fish fries can be reduced to the original sizes after the fish fries pass through the net mouth, the fish fries can not flow back after the fish fries pass through the net mouth, and the phenomenon that the fish fries are stacked up and down or overlapped front and back in a counting area to cause counting omission and repeated counting is avoided.

As shown in fig. 4 and 5, the tactile counting device 28 includes a tactile sensor 281, a retractable contact device 282 and a fixed sleeve 283, the fixed sleeve 283 is fixedly communicated with the unidirectional channel pipe and is located above the secondary adaptive net mouth device 27, the retractable contact device 282 is embedded into the fixed sleeve 283, the lower end of the retractable contact device 282 extends into the unidirectional channel pipe, when a fry passes through the secondary adaptive net mouth device 27, the fry touches the retractable contact device 282 to rise to contact the tactile sensor 281, and the classification counting module of the controller 5 receives the touched information of the tactile sensor and records the touching times, namely the tactile classification counting value.

In fig. 5, (a), (b) and (c) represent the entry of a fry into the secondary adaptive portal device, the fry triggering of the tactile counting device, and the exit of the fry from the secondary adaptive portal device, respectively. When the fry passes through the two-stage self-adaptive net mouth device, the size of the net mouth is enlarged, the fry touches the telescopic contact device 282 embedded in the fixed sleeve 283, the telescopic contact device 282 touches the touch sensor 281 fixed at the hose, the three touch sensors are connected to the controller 5 through the control wire 51, the controller 5 receives the touched information of the touch sensors and records the touching times, and when the fry exits the net mouth, the telescopic contact device also retracts slowly to the original position due to the self-adaptive principle of the self-adaptive net mouth device.

As shown in fig. 6 and 7, the fry stagnation acceleration device 4 comprises a reservoir 41, three hoses 42, 43, 44 and three control valves 45, 46, 47, wherein the reservoir 41 is additionally arranged above the first-stage partition plate 21 and is supported by a support plate 48, the upper end of the first hose 42 is communicated with the reservoir 41, the lower end of the first hose 42 is communicated with the first unidirectional passage pipe 23, and the first hose 42 is provided with the first control valve 45; the upper end of the second hose 43 is communicated with the reservoir 41, the lower end of the second hose 43 is communicated with the second unidirectional passage pipe 24, and a second control valve 46 is arranged on the second hose 43; the upper end of the third hose 44 is communicated with the reservoir 41, the lower end of the third hose 44 is communicated with the third unidirectional passage pipe 25, and a third control valve 47 is arranged on the third hose 44; the controller 5 is connected to the first control valve 45, the second control valve 46 and the third control valve 47 via control lines 51, respectively; the lower ends of the three hoses are respectively communicated with the three unidirectional channel pipes and led to the positions right above the fish outlets of the two-stage self-adaptive net mouth devices, when the controller 5 receives the information triggered by any one touch sensor, the controller controls the corresponding control valve to be opened, water flow is ejected by utilizing the gravity principle, fish fries are flushed forward, the fish fries leave the unidirectional channel pipes as soon as possible and enter the feeding pool, and the control valve is closed after the fish fries leave. The fry stagnation acceleration preventing device 4 increases the fluidity of the fry in the pipeline. The fish fry is prevented from staying in the unidirectional pipeline after the fish fry is discharged from the secondary self-adaptive net mouth device in the counting pool, so that the next fish fry outlet net mouth is influenced, and the phenomenon of congestion is caused.

As shown in fig. 8, the feed feeding device 6 is composed of three groups of drawing plates 61, a bin 62, a feeding channel 63 and a discharge port 64, wherein each group of drawing plates isolates the bin 62 from the feeding channel 63, the discharge port 64 is communicated with the feeding channel 63, and after the drawing plates 61 are drawn, feed in the bin 62 enters the feeding pool 3 through the feeding channel 63 by the discharge port 64. The fodder feeding device can adopt the mode of artifical material loading to put into three feed bins with multiple bait, when throwing something and feeding, can take out the board that takes out in the middle of the device, take out the distance that the board took out through the control and take out, control the speed of feeding.

According to the invention, the fish fries are attracted from the seedling raising pool to the feeding pool in a bait feeding mode, and are counted by the counting device when passing through the classified counting pool. And there are multiple fish in the pond of growing seedlings, the device can throw in multiple bait simultaneously, can attract the three kinds of fries that like different bait to the pond of throwing something and feeding as much as possible.

As shown in fig. 9, in order to prevent the water level of the seedling raising pool and the feeding pool from rising due to the operation of the stagnation preventing accelerating device, a water outlet 14 is formed on the right side of the seedling raising pool and at the same level with the horizontal plane for controlling the water level balance in the seedling raising pool.

As shown in fig. 10, the camera includes a standard connector 131, a housing 132, and a camera 133, the standard connector 131 being disposed on the housing 132 and connected to the controller through a control line 51, the camera 133 being mounted on the housing 132; aiming at the characteristic of low underwater lighting rate, LED lamps 134 are additionally arranged on the two cameras of the image acquisition camera 13 and the real-time detection camera 29, so that underwater light is enhanced, the quality of images and video data acquired by the image acquisition camera is improved, and the counting real-time efficiency and the recognition accuracy of the real-time detection camera are improved.

In the embodiment of the invention, a controller is connected with three control valves, three touch sensors, an image acquisition camera and a real-time detection camera through control lines.

As shown in fig. 11, the fish fry classifying and counting method based on vision and touch of the invention comprises the following steps:

s1, an image acquisition camera in a seedling raising pool acquires images and video data of three types of fish fries in the seedling raising pool, the data are uploaded to a controller, the controller intercepts the video data frame by frame to obtain image data, the acquired image data and the intercepted image data are preprocessed and marked, the marked image data are divided into a training set and a verification set, the training set and corresponding labels thereof are used for manufacturing the training data set according to a set format and a set sequence, and the verification set and corresponding labels thereof are used for manufacturing the verification data set according to the set format and the set sequence; as shown in fig. 12, specifically:

the image acquisition camera shoots images and video data of three types of fish fries in the fry rearing pond, the acquired video data are intercepted into frame-by-frame image data, then all the image data are screened one by one, the images with obvious characteristics are selected to reject useless images, and the images are subjected to dark channel defogging treatment to achieve the effect of enhancing and freshening the images. After the high-quality images are obtained, labeling three types of fish fry targets on each image on labelimg labeling software, and storing and exporting the labeled images into a standard VOC file format. To facilitate subsequent model training, the VOC formatted tags are further converted to YOLO formatted tags (txt format) using the python script program. And finally, marking all marked images and corresponding label files according to 8:2 is divided into a training data set and a verification data set according to a certain format and sequence, and the training data set and the verification data set are placed under a folder for subsequent training of the YOLOv5 network model, and the whole data set file is manufactured.

S2, inputting a training data set into the YOLOv5 network model for training to obtain a trained YOLOv5 network model, inputting a verification data set into the trained YOLOv5 network model for verification, if the verification is qualified, storing the trained YOLOv5 network model, if the verification is unqualified, modifying model training parameters on the basis of previous training to continue training until the network model meets the accuracy and speed of real-time detection, and storing the trained YOLOv5 network model; as shown in fig. 13, specifically:

the data set file is placed in a YOLOv5 model catalog, and corresponding data set path parameters ("-data") are modified in the model project file to read the fish image data set for training. In order to avoid slow convergence of model training, a training method of pre-training weights is adopted to improve the convergence speed of model training. In order to obtain a model with better performance, the following super parameters are set before model training: the training file (train. Py) was set to 400 iterations ("-epochs"), 16 images per input ("-batch-size"), 0.005 weight decay, 0.9 momentum, and 0.001 initial value of learning rate. Obtaining a preliminary network model after training is finished, modifying corresponding parameters in a file (val.py) for verifying the performance of the model to carry out precision test on a verification data set, and if the precision is unqualified, adjusting and optimizing super parameters on the basis of the last training to continue training, so as to improve the detection precision of the model until the training precision of the model meets the requirement; and if the model meets the performance requirements of the accuracy and the speed of the real-time detection, stopping training to obtain a final model.

In order to enable the YOLOv5 network model to realize target detection and identification by docking the real-time detection camera. In the real-time detection file (detect. Py), the weight parameter ("-weights") is set as the best weight obtained after model training. Then, the device attribute hyper parameter ("-source) is set as the path of the real-time detection camera (source default value is set to 2 in this embodiment), thereby completing the docking of the YOLOv5 network model with the real-time detection camera. The real-time detection file (detect. Py) is run to realize the real-time detection and identification of three types of fries, and the real-time picture is displayed on the controller. ( And (3) injection: the default value of the source parameter is 0, which corresponds to the camera path of the controller. If the controller is externally connected with a plurality of cameras, the source parameter value system defaults to correspond to 1, 2 and 3 … … respectively. Because the device is externally connected with an image acquisition camera and a real-time detection camera, the default value of the source parameter is set to be 2 so as to correspond to the real-time detection camera. )

The YOLOv5 network model has high real-time detection efficiency, can frame the position of the fish in real time and display the fish class label at the upper left corner of the generation frame, if the I-class fish fry is detected, a target frame is generated on the I-class fish fry target in real time, and the label 'I' is displayed at the upper left corner of the target frame; if the II-class fry is detected, a target frame is generated on the II-class fry target in real time, and a label II is displayed at the upper left corner of the target frame; if the III fish fry is detected, a target frame is generated on the III fish fry target in real time, and the label 'III' is displayed on the upper left corner of the target frame. And the color of the fish fry generation target frame of each category is detected to be different, so that the fish fry generation target frame is easy to distinguish. The number of the generating frames is the number of the fries, and according to the number of the fries, the real-time detection camera can detect the false entry number of each type of fries in the three unidirectional channel pipes on the controller in real time.

S3, the touch counting device (28) transmits the collected touch information to the controller, and the controller records the initial number of fish fries passing through the corresponding unidirectional channel tube, namely a touch classification count value, according to the touch information of the touch counting device (28); meanwhile, the real-time detection camera transmits the real-time collected video data of the fries in the unidirectional channel pipes to the controller, and the trained YOLOv5 network model stored in the step S2 is utilized for classification and identification to obtain the number of other two types of fries wrongly entering each unidirectional channel pipe, namely a vision auxiliary classification count value; the touch sense classification count value corresponding to each channel tube subtracts the number of other two types of fries which enter the unidirectional channel tube by mistake and the number of the other two unidirectional channel tubes by mistake, so that the total number of the fries of each type is finally obtained. As shown in fig. 14, specifically:

three layers of fish fries with different habit distribution: the method comprises the steps that I-class fries 10, II-class fries 11 and III-class fries 12 enter corresponding I-class unidirectional channel pipes, II-class unidirectional channel pipes and III-class unidirectional channel pipes respectively, then enter a first-stage self-adaptive net mouth device to separate a large amount of gathered fries, then enter a second-stage self-adaptive net mouth device, after entering the second-stage self-adaptive net mouth device, net mouths are enlarged, a telescopic contact device is touched, the telescopic contact device touches a touch sensor, the telescopic contact device is sleeved on a fixed sleeve and can stretch up and down, the touch sensor is connected with a controller through a control wire 51, the controller obtains touched sensor information, the touch times of the touch sensors in the three unidirectional channel pipes are respectively A, B, C, namely the touch classification count value of the I-class fries in the first unidirectional channel pipe is A, the touch classification count value of the II-class fries in the second unidirectional channel pipe is B, and the touch classification count value of the III-class fries in the third unidirectional channel pipe is C;

Meanwhile, the real-time detection camera detects that the number of the I-type unidirectional channel tubes which are wrongly entered by the II-type fries 11 and the III-type fries 12 is X and Y respectively, the number of the II-type unidirectional channel tubes which are wrongly entered by the I-type fries 10 and the III-type fries 12 is M and N respectively, the number of the P and Q respectively, and the numerical value of A, B, C, X, Y, M, N, P, Q is counted on the controller.

The initial number of the class I fries is the number of triggering times of the touch counting device minus the number of the class II fries 11 and the class III fries 12 which are wrongly fed into the class I unidirectional channel pipe, namely (A-X-Y), and the total number of the class I fries is the initial number plus the number of the class I fries wrongly fed into the class II unidirectional channel pipe and the class III unidirectional channel pipe, namely alpha= (A-X-Y) +M+P; the initial number of the II fries is the number of triggering times of the touch counting device minus the number of the I fries and the III fries which are wrongly fed into the II unidirectional channel tubes, namely (B-M-N), and the total number of the II fries is the initial number plus the number of the II fries which are wrongly fed into the I unidirectional channel tubes and the III unidirectional channel tubes, namely beta= (B-M-N) +X+Q; the initial number of the III fries is the trigger number of the touch counting device minus the number of the I fries and the II fries which are wrongly fed into the III unidirectional channel pipe, namely (C-P-Q), and the total number of the III fries is the initial number plus the number of the III fries which are wrongly fed into the I unidirectional channel pipe and the II unidirectional channel pipe, namely gamma= (C-P-Q) +Y+N, so that the total number of the I fries 10, the II fries 11 and the III fries 12 is respectively obtained.

The invention discloses a device and a method for classifying and counting 3 fish fries for intelligent cultivation of deep sea cultivation industry boats, wherein 3 fish fries to be detected are placed in a seedling raising pond, an image acquisition camera is placed in the seedling raising pond, a feed feeding device is arranged above a feeding pond, the feeding pond and the seedling raising pond are separated by a first-stage partition plate and a second-stage partition plate and are connected through three unidirectional channel pipes to form a classifying and counting pond, a hose is connected above the three unidirectional channel pipes of the classifying and counting pond, a water reservoir above the classifying and counting pond is connected with a control valve on the hose to form a combined counting module, a touch counting device arranged above an interface of the unidirectional channel pipe and the hose and a real-time detection camera placed in the classifying and counting pond are used for assisting in false feeding and counting of fish fries, and the combination of the touch counting device and the real-time detection camera is used for classifying and counting the fish fries of different habit categories. In addition, the control part of the whole device is controlled by a controller through a control line, and the fish attracting effect is achieved by adopting a feed feeding mode, so that the fish attracting effect reaches the feeding pool from the seedling pool. The first-level and second-level self-adaptive net mouth device arranged in the unidirectional channel pipe of the classification counting pool avoids the phenomenon that a great deal of fries are gathered to cause congestion, stacked up and down and overlapped front and back, and the fry stagnation acceleration prevention device arranged above the classification counting pool increases the fluidity of the fries and avoids the phenomenon that the fries are blocked in the pipeline.

Claims

1. The fish fry classifying and counting device based on the vision and the touch is characterized by comprising a seedling raising pool (1), a classifying and counting pool (2), a feeding pool (3), an anti-fish fry stagnation accelerating device (4) and a controller (5), wherein I-class fish fries (10), II-class fish fries (11) and III-class fish fries (12) are respectively distributed from the upper layer to the lower layer in the seedling raising pool (1), and an image acquisition camera (13) is further arranged in the seedling raising pool (1); the classification counting pool (2) is separated from the seedling raising pool (1) and the feeding pool (3) through a first-stage partition plate (21) and a second-stage partition plate (22), three one-way channel pipes (23, 24 and 25) are arranged between the first-stage partition plate (21) and the second-stage partition plate (22) in the classification counting pool (2), the three one-way channel pipes (23, 24 and 25) correspond to habit distribution of each layer of fries, the seedling raising pool (1) and the feeding pool (3) are communicated through the three one-way channel pipes, a first-stage self-adaptive net mouth device (26) and a second-stage self-adaptive net mouth device (27) are sequentially arranged in the one-way channel pipes from the seedling raising pool (1) to the feeding pool (3), and a touch counting device (28) is arranged on the one-way channel pipe above the second-stage self-adaptive net mouth device (27); the fry stagnation acceleration prevention device (4) comprises a reservoir (41), three hoses (42, 43, 44) and three control valves (45, 46, 47), wherein the reservoir (41) is arranged at the top end of the primary partition plate (21) and is respectively communicated with three one-way channel pipes (23, 24, 25) through the three hoses (42, 43, 44), the communication position of the hoses and the one-way channel pipes is positioned above the outlet of the secondary self-adaptive net mouth device (27), and the control valves are arranged on the hoses; a real-time detection camera (29) is also arranged in the classification counting pool (2) and is used for collecting video data in three unidirectional channel tubes in real time;

The controller (5) comprises a data processing module, a model training module, a classification counting module and a control module, wherein an image acquisition camera (13) acquires images and video data of three types of fries in the seedling raising pool (1) and uploads the images and the video data to the data processing module, the data processing module intercepts the video data into image data frame by frame, marks the three types of fries in all the image data respectively to form a data set, and the model training module adopts data set training to obtain a YOLOv5 network model; the three types of fries trigger corresponding touch counting devices (28) in the process of entering the feeding pool through three unidirectional channel pipes respectively, and the touch counting devices (28) upload signals to the classification counting module to perform initial classification counting; the real-time detection camera (29) transmits video data of three types of fries in three unidirectional channel pipes acquired in real time to the controller, and the number of the fries in each type entering the other two unidirectional channel pipes by mistake is obtained through classification and identification of a YOLOv5 network model, and the classification counting module calculates the final classification counting result of the fries according to the initial classification counting result and the false entering number result; when the touch counting device (28) is triggered, the control module controls the corresponding control valve to be opened, and when the touch counting device (28) is not triggered, the control valve is closed, and the control module is also used for controlling each touch counting device (28), the image acquisition camera (13) and the real-time detection camera (29).

2. The fish fry classifying and counting device based on vision and touch as claimed in claim 1, wherein the upper part of the fry rearing pond (1) is further provided with a water outlet (14).

3. The fish fry classifying and counting device based on vision and touch sense according to claim 1, wherein the touch sense counting device (28) comprises a touch sense sensor (281), a telescopic contact device (282) and a fixed sleeve (283), the fixed sleeve (283) is fixedly communicated with the unidirectional channel pipe and is positioned above the secondary self-adaptive net mouth device (27), the telescopic contact device (282) is embedded into the fixed sleeve (283), the lower end of the telescopic contact device extends into the unidirectional channel pipe, the fish fry touches the telescopic contact device (282) when passing through the secondary self-adaptive net mouth device (27) and rises to contact the touch sense sensor (281), and the classifying and counting module receives the touched information of the touch sense sensor and records the touch times.

4. The fish fry classifying and counting device based on vision and touch as claimed in claim 1, wherein the first stage self-adapting net mouth device (26) and the second stage self-adapting net mouth device (27) are larger at net mouth and smaller at net mouth, and are close to the seedling raising pool and are close to the feeding pool.

5. The fish fry classifying and counting device based on vision and touch as claimed in claim 1, wherein a feed feeding device (6) is further arranged above the feeding pool (3), the feed feeding device (6) comprises three groups of drawing plates (61), a bin (62), a material channel (63) and a discharge port (64), the drawing plates isolate the bin (62) from the material channel (63), the discharge port is communicated with the material channel (63), and feed in the drawing plates enters the feeding pool from the discharge port through the material channel.

6. The fish fry classifying and counting device based on vision and touch as claimed in claim 1, wherein the bottom of the water reservoir (41) is further provided with a support plate for further fixing the water reservoir to the first stage partition plate.

7. A fish fry classifying and counting method based on vision and touch is characterized by comprising the following steps:

S3, the touch counting device (28) transmits the collected touch information to the controller, and the controller records the initial number of fish fries passing through the corresponding unidirectional channel tube, namely a touch classification count value, according to the touch information of the touch counting device (28); meanwhile, the real-time detection camera transmits the real-time collected video data of the fries in the unidirectional channel pipes to the controller, and the trained YOLOv5 network model stored in the step S2 is utilized for classification and identification to obtain the number of other two types of fries wrongly entering each unidirectional channel pipe, namely a vision auxiliary classification count value; the touch sense classification count value corresponding to each channel tube subtracts the number of other two types of fries which enter the unidirectional channel tube by mistake and the number of the other two unidirectional channel tubes by mistake, so that the total number of the fries of each type is finally obtained.

8. The visual and tactile-based fry classifying and counting method of claim 7, wherein the preprocessing and labeling of the image data in step S1 includes:

9. The visual and tactile based fry classifying and counting method of claim 7, wherein in step S2, the YOLOv5 network model is capable of framing the fish' S position in real time in the image data and displaying the fish class label in the upper left corner of the frame; when the I-class fish fry is detected, a target frame is generated on the I-class fish fry target in real time, and a label 'I' is displayed at the upper left corner of the target frame; when the II-class fish fry is detected, a target frame is generated on the II-class fish fry target in real time, and a label II is displayed at the upper left corner of the target frame; when the III fish fry is detected, a target frame is generated on the III fish fry target in real time, and a label III is displayed at the upper left corner of the target frame; and detecting that the colors of the fish fry generation target frames of each category are different, wherein the number of the generation frames is the number of the fish fries.

10. The fish fry classifying and counting method based on the vision and the touch as claimed in claim 7, wherein the step S3 is specifically:

s31, I fries (10), II fries (11) and III fries (12) respectively enter corresponding first one-way channel pipes, second one-way channel pipes and third one-way channel pipes, then enter a first-stage self-adaptive net mouth device to separate a large number of gathered fries, then enter a second-stage self-adaptive net mouth device to trigger respective touch sensors, and a controller respectively records the triggering times A, B, C of the three touch sensors in the first one-way channel pipe, namely the touch classification count value of the I fries in the first one-way channel pipe is A, the touch classification count value of the II fries in the second one-way channel pipe is B, and the touch classification count value of the III fries in the third one-way channel pipe is C;