TWI778762B - Systems and methods for intelligent aquaculture estimation of the number of fish - Google Patents

Abstract

A technology for estimating the total number of fish in the fish farm is disclosed, in which a plurality of training examples of consecutive frames of images of the fish school are annotated with strong, weak or no labels, a training criterion is set to train a neural network to be a neural network having a target function capable of converting fish school images to fish density maps, and a beta-binomial mixture model set based on prior information regarding the fish farm and the species of fish is used to estimate the total number of fish in the fish farm.

Description

Method and system for estimating the number of intelligently farmed fish

The present invention relates to artificial intelligence aquaculture fish quantity estimation, in particular, training a neural network to become a target probability function neural network capable of generating a density map for estimating fish quantity, and automatically estimating aquaculture in aquaculture farms. Method and system for total number of fish.

As the global population increases and the number of wild aquatic products plummets, the supply of aquatic products is facing severe challenges. Therefore, the development of aquaculture is one of the necessary means to meet the growing demand for aquatic products, and it is also an important means to protect wild aquatic products from extinction. However, the impact of environmental pollution caused by aquaculture is also worrying. Therefore, the application of smart technology to monitor water quality and aquaculture target behavior, for aquaculture, can not only monitor the aquaculture environment, monitor the growth and health of fish, but also reduce aquaculture manpower and feed waste. In this way, in addition to saving feed, reducing aquatic mortality and other breeding costs, it can also reduce water pollution, reduce the risk of fish disease, and protect the water environment because no excess feed is fed.

Therefore, the effective application of intelligent technologies such as artificial intelligence to solve various problems faced by the aquaculture industry is a very important topic for modern aquaculture. In many applications, fish monitoring technology is the key technology for constructing a smart breeding system. The non-contact monitoring method of such monitors or photographic equipment can interfere or damage the breeding target to a very low degree, and is very suitable for the aquaculture industry. Therefore, it is necessary to combine monitoring systems and artificial intelligence to accurately analyze and count the total number of farmed aquatic products, analyze the behavior, appetite, etc.

Some techniques using artificial intelligence to monitor fish are known, but they have problems such as high cost and low accuracy. For example, when marking images for training, the center pixel occupied by each fish must be marked , Do not allow fish to be missed or mislabeled, and it is difficult to deal with overlapping fish. Alternatively, cut each fish image in blocks and then mark the number of fish in each block, but there will be A lot of fish are cut to the detriment of accuracy or increased marking work, etc., which can cause high cost problems.

Therefore, more accurate and lower cost AI technology is needed for applications such as fish monitoring or fish farming.

In view of the above, the present invention provides a low-cost, highly accurate method and system for estimating the number of farmed fish with artificial intelligence.

According to an aspect of the present invention, a neural network training method is provided, which is used to generate a density map for estimating the number of farmed fish in a farm area, and use at least two imaging devices to shoot and obtain many frames of fish images as many learning samples. , the method includes the following steps: a labeling step, in the form of strong labeling, weak labeling, and no labeling, label each of the learning samples to distinguish the strong pixel labeling area with one fish or no fish at all, There are at least the weak pixel marked area and the unmarked pixel area of the two-tailed fish; and, in the training step, the target probability function is generated according to the probability relationship between the strong pixel marked area and the corresponding learning sample, and the weak pixel marked area is based on. And the unmarked pixel area is consistent with the calculation results corresponding to the respective probability relationships of the learning samples and the target probability function, as a training criterion, to teach the neural network to become the target probability function neural network, so as to generate a neural network that can be used for Density map for estimating fish numbers.

According to another aspect of the present invention, there is provided a method for estimating the number of farmed fish using artificial intelligence, for estimating the total number of fish farmed in a farm, the method comprising the following steps: using at least two imaging devices to obtain multiple frames of fish school Image; use the trained target probability function neural network to convert multiple frames of fish school images into multiple corresponding density maps; according to the configuration of at least two imaging devices, process each density map obtained by each imaging device at the same time as if from Obtained from the same viewing angle; using the connected object marking method, mark the connected areas of each pixel whose probability value is greater than the predetermined threshold in each density map obtained by the processed imaging devices at the same time; Taking a×b pixels as the unit, it is divided into a predetermined number of blocks and sorted. The larger number of connected areas in the blocks with the same order is the representative number, and the total number of representatives is obtained by adding up the representative numbers. Both a and b are is a natural number; and, based on the binomial distribution, according to the fish species and the conditions of the breeding field, a mixed binomial distribution model is designed, and the representative total number obtained at different times and the predetermined parameter values representing each condition are used, Estimate the total number of farmed fish.

According to another aspect of the present invention, an artificial intelligence breeding fish quantity estimation system is provided for estimating the total number of fish cultured in a breeding field. The system includes: at least two imaging modules for photographing aquaculture fish; a neural network A road module, including a target probability function neural network for completing training; a processing unit; and a communication unit, wherein the neural network module and the processing unit work together to convert the photographed images of aquaculture fish into a fish density map, And, using the fish population estimation method according to the present invention, the total number of farmed fish is estimated.

Preferably, the imaging device is a sonar imaging device. In addition, the neural network module And at least one of the processing units is integrated with at least one imaging module. At least one of the neural network module and the processing unit can be set in the cloud, the Internet of Things, or in aquaculture equipment.

According to the artificial intelligence fish quantity estimation method and system of the present invention, the learning samples can be marked accurately at low cost, the cost of artificial intelligence application can be significantly reduced, and when artificial intelligence is used to cultivate aquatic products, the breeding efficiency and efficiency can be significantly improved. Reduce farming costs.

100: Box net farming field

101a: Sonar Imaging Devices

101b: Sonar Imaging Devices

200: Neural Network Training Methods

201: Labeling Steps

202: Training steps

302: Individual fish

304: Background Outline

306: Fixed Facility Outline

308: Weak marked area

310: Weakly marked area

312: Weak marked area

500: Method for Estimating Number of Fish

501: Connected area count step

502: Estimating the total number of fish using mixed binomial steps

800: Artificial Intelligence Fish Estimation System

801: Imaging module

802: Neural Network Module

803: Processing Unit

804: Communication unit

The features, aspects, and advantages of the present invention will be better understood by reading the detailed description with reference to the accompanying drawings, in which the drawings are not necessarily to scale, but generally emphasize various aspects of the present disclosure 1 is a view showing the configuration of an aquaculture field sonar imaging device according to an embodiment of the present invention; FIG. 2 is a block diagram showing a neural network training process according to an embodiment of the present invention; FIG. 3A Fig. 3B is a fish school image corresponding to the completed marking of Fig. 3A; Fig. 4 is a fish school density map; Fig. 5 is a block diagram for illustrating the estimation of the number of fish according to an embodiment of the present invention method; FIG. 6 is a diagram illustrating a graph segmentation method used in accordance with an embodiment of the present invention; FIGS. 7A, B, C, and D are graphs showing beta probability distributions in different situations, respectively; and FIG. 8 is a block diagram , to illustrate the fundamental invention of the artificial intelligence fish appetite judgment system.

Aquaculture is carried out in specific fields such as ponds, box nets, etc., and the aquaculture targets can be a variety of fish, shrimp, shellfish and so on. In this manual, cage-net cultured Take fish as an example. Effective and accurate monitoring of farmed fish is a key factor to achieve efficient farming. Generally speaking, fish population monitoring includes at least: (1) growth monitoring, such as body length and body width measurement; (2) quantity measurement, automatically counting and breeding The number or distribution of fish swimming freely in the field; (3) Behavior monitoring, automatically monitoring, analyzing and identifying the feeding behavior, appetite, disturbance, health status and so on of the fish. In these monitoring applications, the monitoring of behaviors such as quantity measurement and appetite is difficult to achieve accurate measurement and accurate analysis, and it is also a key factor for the efficiency and cost reduction of aquaculture. According to the present invention completed by the inventor's long-term research, the measurement accuracy of the number of fish can be significantly improved, and the behaviors such as feeding and appetite of the fish can be accurately analyzed and identified, thereby greatly improving the breeding efficiency and reducing the breeding cost. In the following description, the embodiments of the present invention will be described in detail with reference to the drawings, but it should be understood that the following descriptions of the present invention are only by way of example, so as to help understand the spirit and principle of the present invention, but not to limit the present invention. .

Generally speaking, when applying artificial intelligence, artificial intelligence training must be performed with a predetermined number of samples first. After the training achieves the predetermined goal, data or techniques such as parameters obtained from the training results are used to generate artificial intelligence for specific applications. Smart technology. The samples are usually images or graphics of the target to be processed, and the techniques commonly used for training are neural networks and so on. According to the present invention, since artificial intelligence is applied to aquaculture, it is necessary to obtain clear images in turbid or dimly lit waters for study when estimating the number or distribution of fish, monitoring fish appetite and other behaviors, etc. Training to achieve automatic and accurate estimation or judgment. Typical optical cameras or video cameras usually cannot obtain clear images for analysis in this application, so equipment that can capture clear images in water must be used.

According to the embodiment of the present invention, it is preferable to use a sonar imaging device to capture the dynamics of fish in the farm area, but any device that can clearly capture images in water can be used, Not limited to sonar imaging equipment. When using sonar imaging equipment to monitor fish, the horizontal angle range, vertical angle range, effective distance resolution, etc. of the fan-shaped sound beam will be considered to obtain the desired fish image. Since fish schools move dynamically in the farm area, even if the same sonar imaging device is placed in different locations, the imaging results will be different due to the above-mentioned inherent limitations or the dynamic movement of fish schools. According to an embodiment of the present invention, at least two sonar imaging devices are preferably deployed. For example, as shown in FIG. 1 , one sonar imaging device is arranged every 180 degrees in the cage and net breeding field 100 , with a total of Two sonar imaging devices 101a and 101b are deployed. Of course, one sonar imaging device may be deployed every 120 degrees or other predetermined angles.

Due to the movement of a large number of fish in the farm, the influence of inherent facilities such as sonar imaging equipment and box net facilities, there are often fish overlapping, shadowing, swimming in and out of the sonar detection range, and coming from the facility. Interference with echoes of bubbles, fan-shaped viewing angles or inherent limitations of different sonar imaging devices, noise, etc., result in errors such as omissions, misjudgments, etc., in the acquired sonar images. Therefore, when using these images as samples for learning and training, the images must be pre-processed to effectively eliminate or significantly reduce these factors that adversely affect the discrimination or training. Learning the training results, it can automatically and accurately analyze the length of the fish, the number and behavior of the fish, and so on. It is worth mentioning that, according to an embodiment of the present invention, a fish school image is obtained using a multi-imaging sonar viewing angle.

As mentioned above, when multiple frames of fish school images are used as artificial intelligence learning samples, the above-mentioned problems such as overlapping, shading, and instantaneous swimming in and out of fish will cause difficulties in artificial intelligence learning. Therefore, according to the present invention, each frame of fish school is The method of marking and classifying the image to divide the whole frame of image into a plurality of marked regions can overcome these difficulties and improve the accuracy and efficiency of learning. According to the present invention, the neural network is taught with classified samples to set evaluation criteria, for example, set A loss function is calculated to teach the neural network to achieve consistency in the learning responses of different labeled regions. Generally speaking, the learning methods of neural networks can be divided into unsupervised learning methods that learn from areas that are not marked at all, and semi-supervised learning methods that can learn from unmarked or marked areas. Learning, and supervised learning for learning in marked areas. In artificial intelligence learning and training, it is usually based on the original performance of the learning sample in terms of statistics or probability, and the output performance of the neural network for the same sample as input, whether the two meet the expected similarity or consistency as training. Completion assessment. Hereinafter, the embodiments according to the present invention will be described in detail to help better understand the spirit and principle of the present invention.

When estimating the number of fish, it is first necessary to be able to accurately estimate the distribution of the fish population and to generate the corresponding density map. FIG. 2 is a block diagram showing a neural network training process 200 in an artificial intelligence training phase according to an embodiment of the present invention. As shown in FIG. 2 , the training process 200 mainly includes a marking step 201 and a training step 202 . Below, each step will be explained in detail.

In the labeling step 201 , each image in the multi-frame fish images serving as a learning sample is classified and labeled so that each pixel belongs to a different labeling category. According to an embodiment of the present invention, each frame of images is marked with three types of markings: no marking, weak marking, and strong marking, wherein the strong marking means that a single point is used to mark the fish when it is determined that a fish exists. And the area that is sure to be free of fish is marked with a polygonal circle surrounding the area; weak marking is to determine the area where at least n ( n≧ 2) and at most cn ( c >1) fish appear but the location of some fish cannot be determined. , it is marked with a rectangle enclosing the region; and, pixels or regions that are not marked are considered uncertain or that have been marked with a considerable number of similar markings before are not marked. In the following, the area marked by the strong marking method is called the strong pixel marking area, the area marked by the weak marking method is called the weak pixel marking area, and the unmarked area is called the unmarked pixel area. In this way, according to the embodiment of the present invention, a marked image frame can be divided into a strong pixel marked area, a weak pixel marked area, and an unmarked pixel area.

3A and 3B, the marking step 201 according to the present embodiment will be described in detail. FIG. 3A is an image of a school of fish acquired by the sonar imaging device 101 , and FIG. 3B is an image corresponding to the completion mark of FIG. 3A . First, the strong labeling is explained. In the fish school image in FIG. 3A , the images of individual fish bodies that can be clearly identified are marked with a single dot mark, and are classified into the label category f1 . For example, all the individual fish bodies 302 identified in FIG. 3A are marked with dots and classified into category f1. In this way, each marked fish will be represented by a dot and given the tag class f1, in this way, the marking as shown in FIG. 3B is generated. In order to distinguish the fish school image from the background image, the location in FIG. 3A that can be clearly determined as the background or the fixed facility is marked with a multi-point enclosing outline to obtain the background outline 304 and the fixed facility outline 306 as shown in FIG. 3B , They are classified into label categories B and F respectively. Label category B represents background and F represents fixed facilities, which is also a strong label. In this way, although the overall strong pixel marking area is also strongly marked, it is still possible to distinguish areas with fish and no fish due to different label categories, and the total number of fish in the overall strong pixel marking area can also be clearly and easily calculated. With regard to weak labeling, it refers to the area marked by a rectangular box with at least n(n≧ 2) and at most cn(c>1) fishes present, and it is clearly known that at least a certain number of fish are present in this area. The number of fish may be, for example, at least 2, 5 or 10, classify them in class f2, f5 or f10 respectively, if c is set to 2 label class f2 for 2 to 4 fish and f5 for 5 to 10 fish, and so on. In this way, weakly marked regions 308, 310 and 312 as in Figure 3B are obtained. In addition, for those pixels or regions whose label category cannot be determined, or similar pixels or regions that have been labeled many times before, it is not necessary to give strong labels to avoid labeling errors. According to the embodiment of the present invention, in addition to the strong marking and non-marking methods that can save the cost of marking, the weak marking method does not need to mark individual fish, but uses the outline of the area including the fish within a predetermined number range as marking, which is more obvious. Save on labeling costs.

As described above, each pixel in the image frame acquired by the sonar devices 101a and 101b belongs to the strong pixel marked area, the weak pixel marked area, and the unmarked pixel area, respectively. Although the above description specifies the number of label categories and the meaning of the label categories, it is only for the convenience of description, not to limit the meaning of the label categories, and the label categories can be changed according to conditions. According to the above method, a tag file is generated to record the address, label and associated tag type of each pixel. For example, in the markup file of an image with a resolution of 480×640, record the m×nth pixel as a strong pixel and the label type f1, the k×mth pixel as a weak pixel and the label type f2, and the (k+2)× (m+2) pixels are unmarked, and so on. In addition, in the following, for the sake of brevity, when referring to words such as strong labeling, weak labeling, and no labeling, in addition to specifying the labeling method, it also means giving the corresponding label category.

Before explaining the training step 202. First refer to the fish density map shown in Figure 4 to briefly explain the meaning of the fish density map. The fish school density map (hereinafter referred to as the density map) is a fish school distribution map corresponding to the frame image generated according to the probability distribution of fish appearing in each pixel of a frame of fish school image. From a probability point of view, a density map is equivalent to a probability distribution map in which the total number of cells is equal to the total number of pixels (resolution) of the fish image, and the numbers in the cells represent the probability of fish appearing.

In the training step 202, training criteria are set to train the neural network to achieve results that most closely approximate the real application. Taking each frame of fish school images as learning samples, the following formula (1) is used to generate the corresponding fish school density map as a learning reference. Density map:

其中，D(p)是區域F中的密度圖，p表示位置，p_k表示第k個位置。

where D(p) is the density map in region F , p denotes the position, and p _k denotes the kth position.

According to the embodiment of the present invention, the following formula (2) is designed to generate fish corresponding to each frame Density map of the group image map:

其中，Y=代表整幀影像的所有像素y_i，＊表示卷積運算，

(p；0,Σ)代表期望值與共變異數矩陣分別為0與Σ的高斯密度函數，此處，P(Y=1|X,Θ)係表整幀影像有魚的機率。

Among them, Y = represents all pixels _yi of the whole frame of image, * represents the convolution operation,

(p; 0, Σ) represents the Gaussian density function whose expected value and covariance matrix are 0 and Σ , respectively, where P(Y=1| X , Θ) represents the probability of fish in the whole frame image.

The fish density map generated using Equation (1) is used as a comparison reference during training to be compared with the density map generated by Equation (2) for evaluating training. As mentioned above, the fish in the strong pixel-marked area in a frame of image is the most definite and countable, so its probability distribution is the most reliable, and can be used to evaluate the probability distribution of fish in other pixel areas. Therefore, when setting the training criteria, it will be considered whether the probability relationship between the strong pixel marked area of each learning sample relative to the density map generated by training and the probability relationship relative to the comparison reference density map reaches a predetermined degree of consistency or consistency (in In other words, from the viewpoint of probability comparison consistency, the other marked pixel areas must be the same as the strong pixel marked areas. match or within the predetermined range. In addition, according to the probability relationship between the strong pixel marked area and the comparison reference density map, a probability function that can best reflect the probability distribution of fish in the fish school image frame is designed to train the neural network, according to formula (2) Generate a fish density map. Below, it will be further described in detail.

According to embodiments of the present invention, the neural network used for training may be, for example, a neural network of a U-Net architecture, a P-Net architecture, or other architectures. Preferably, the probability function P _{( yi} | X , Θ) is used as the learning target of the neural network, and it is trained as a probability function P( yi | X , Θ) neural network, where _P ( _yi | X ,Θ)≧0, X represents a single frame of fish school image, y _i represents the ith pixel in the whole frame image, Θ represents the neural network parameter, P( y _i =1| X ,Θ) represents the pixel i is Indicates the probability that a fish exists, P( y _i =0| X ,Θ) represents the probability that pixel i is marked with a fish not existing, that is, P( y _i =1| X ,Θ)+P( y _i =0| X ,Θ)=1.

Then, according to the spirit of the training criteria, define the loss function, and use the minimization of the loss function to indicate that the training is completed and meets the expected goal. According to an embodiment of the present invention, the loss function L (Θ|Θ ' ) is set as follows:

其中，S代表所有強標示的像素集合，W代表弱標示的像素集合，Ω代表一張影像裡所有像素的集合，Θ'為前一次疊代神經網路參數，Θ為目前的神經網路參數，α>0與β>0為調整這三項權重的參數。

Among them, S represents the set of all strongly marked pixels, W represents the set of weakly marked pixels, Ω represents the set of all pixels in an image, Θ ' is the previous iteration neural network parameter, Θ is the current neural network parameter , α>0 and β>0 are the parameters to adjust the three weights.

(...)、α×

(...)、及β×

(...)，其中，3-1項會計算強像素標示區之參考密度圖D(P)與魚群密度圖P(y _i |X,Θ)的一致性，3-2項係對非強標示的像素集合進行半監督式學習以達到與強標示像素一致的學習反應，3-3項係對弱標示像素區的訓練結果要滿足該弱標示範圍裡至少有n(n≧2)與至多cn(c>1)尾魚的限制，其中，α及β係權重參數。 In order to simplify the description, the above loss function (3) is mainly divided into three items: 3-1, 3-2 and 3-3, in order:

(...), α ×

(...), and β ×

(...), among which, items 3-1 will calculate the consistency between the reference density map D(P) of the strong pixel marked area and the fish density map P _{( yi} | X ,Θ), and items 3-2 are for non- Semi-supervised learning is performed on the strongly marked pixel set to achieve the same learning response as the strongly marked pixels. Items 3-3 are the training results for the weakly marked pixel area to satisfy at least n(n≧2) and The limit of at most cn(c>1) fish, where α and β are weight parameters.

In the training step 202 , during training, the neural network is taught to calculate the loss function in an iterative and minimization manner. The iterative calculation will calculate the probability that each non-strongly marked pixel is a fish and the probability that it is not a fish based on the currently learned fish post-event probability function (that is, the currently trained neural network). As a result, the previous iteration considered that It is a fish, the result of the next iteration will be more inclined to be a fish, and the result of the previous iteration will be more inclined to not be a fish. It is worth noting that the neural network does not even match the strongly labeled pixels at the beginning of the iteration. During training, the original image and the file generated by the labeling process are used as input, and the neural network performs the learning calculation according to the loss function as described above until the loss function (3) is minimized, and the obtained at this time is calculated. The weight parameters α, β and the learning parameter Θ are stored as the target weight parameter and the target neural parameter respectively, and the neural network after the training can be a function P _{( yi} | X ,Θ) neural network. In other words, any untrained identical neural network with target neural parameters Θ and target weight parameters α, β can be a function P( y _i | X ,Θ) neural network.

In the above, it has been clearly explained how to design and use the loss function and the objective function to train the neural network according to the training criteria, so that the learning of each marked pixel area can achieve the expected probability more consistent, and achieve the training goal. Neural network for application. In this way, according to the results of the completed training, using the same type of neural network as the trained neural network, and inputting relevant parameters, such as the neural parameter Θ and the target weight parameters α, β, the function P( y _i | X , Θ) Neural network to automatically convert any frame of fish image into a corresponding density map for estimating the total number of fish.

Next, referring to FIG. 5 , a method 500 for estimating the number of fishes according to an embodiment of the present invention will be described, which is used for estimating the total number of fishes cultured in a farm. As shown in FIG. 5 , the method 500 for estimating the number of fish includes a step 501 of counting connected regions and a step 502 of estimating the total number of fish by using a mixed binomial method.

In step 501, the fish school images obtained by each imaging device are input into the function P( y _i | X ,Θ) neural network that completes the training line as described above, so as to generate a two-dimensional fish school density map corresponding to the input fish school images . According to the embodiment of the present invention, two sonar imaging devices 101 arranged at 180 degrees in the field are used to capture images in the field, so at the same time, there will be images obtained by the sonar imaging device 101a and the sonar imaging device 101a. For the image obtained by the imaging device 101b, two density maps corresponding to the two frame images are generated. In order to make these two density maps generated from the same viewing angle, one of them is rotated according to the deployment method of the sonar imaging device 101 , and the other is not rotated at all. For example, in this embodiment, the One of the density maps is rotated 180 degrees. Then, using methods such as connected component labeling, etc., to find the connected regions of pixels whose fish-only response intensity (probability value) is greater than a predetermined threshold in each of the two density maps. Regarding the specific description of the connected area, for example, taking the pixel whose probability value is greater than the predetermined threshold as the center, first find out the 8 adjacent pixels of its upper, lower, left, right, upper left, lower left, upper right, and lower right. Whether there is a pixel greater than the predetermined threshold, if so, do not include the adjacent pixel in its connected area, if not, continue to find whether there is a pixel greater than the predetermined threshold in the adjacent pixels of the adjacent pixel, according to According to this rule, the search is continued until there is no pixel greater than the predetermined threshold value and no more pixels can be found. In this way, the central pixel and these found pixels that are not greater than the predetermined threshold value are a connected area. It can be seen from the above description that a connected area can clearly represent at least one fish in the area. According to this embodiment, if n connected regions are found in a density map, it is considered that there are n fishes in the density map. Therefore, according to the present invention, it can be said that one connected area has one fish as the criterion to find out how many connected areas in a density map to represent how many fish.

Then, for the aquaculture field area in the image, with a × b pixels as a basic sub-unit, the two simultaneously obtained density maps of the aquaculture field area are divided into K × L pixels as shown in Figure 6. (K/a)×(L/b) block basic subunit. For example, if the field is 400 pixels wide and 400 pixels high, and is divided according to 80x80 pixels as a basic subunit, then the width direction 5(=400/80)×height direction 5(=400/80)=25 Block basic subunit. In this way, block 1 with a width of 0~79 pixels x 0~79 pixels high, block 2 with a width of 80~159 pixels x 0~79 pixels high... and block 25 with a width of 320~399 pixels x 320~399 pixels high, so Divide to get 25 blocks of 80 pixels wide x 80 pixels High density subunits. The blocks thus divided are numbered sequentially from left to right and from top to bottom. Then, add up the larger number of connected areas in the corresponding blocks in these two pictures to obtain the total number of connected areas counted representing the same time, the same field and the same viewing angle. For example, the density Figure A and density map B are the density maps obtained at the same time and one of them has been rotated. The density maps A and B are respectively divided into a1, a2, ... a25 and b1, b2, ... b25 blocks, in a1 There are 3 connected areas, and there are 5 connected areas in b1, then the 5 connected areas of b1 are represented. In this way, 25 representative values are found and added up, and the total number obtained represents the same time, The total number of connected areas counted from the same field and the same viewing angle, that is, the total number of fish.

Next, step 502 is described, in which the mixed beta-(Beta-) binomial distribution model processing method is used to more accurately estimate the total number of farmed fish in the farm area. Due to the problems of scanning dead corners, overlapping, and occlusion in the sonar image, not every fish can be easily observed. Assuming that there are m fish in the culture pond, if the observed event for each fish is a Bernoulli trial with probability p , then the probability of observing n fish can be assumed to be the binomial of m and p distributed:

但由於聲納影像存在不同程度魚遮蔽等問題，導致每尾魚被觀察到的事件在不同遮蔽等問題情況下會是不相同的白努利試驗，此外，在不同場域或不同魚種時，這些問題情況又會不同。因此，用密度圖估計之魚隻數量雖可由二項分布所組成，但須與能顧及這些因素的其它分布相混合，以更精確地估算魚隻數量。也就是說以上述二項式分布為基礎，考量不同狀況c(c

1)下之白努利試驗，設計出混合Beta-二項分布模型，搭配EM演算法(最大期望演算法Expectation-Maximization Algorithm)，透過最大似然估計法則(Maximum Likelihood Estimation Principle)，根據N筆使用密度圖估計出的魚隻數量n _i ,i=1,...,N，來更準確估計出養殖池內的魚隻數量m。估計混合貝他-(Beta-)二項分布模型未知數之完整似然函數(Complete Likelihood Function)可以定義如下：

However, due to the problems of different degrees of fish shadowing in sonar images, the observed events of each fish will be different Bernoulli tests under different shadowing and other problems. In addition, in different fields or different fish species , the situation will be different. Therefore, the population of fish estimated by the density map can be composed of a binomial distribution, but must be mixed with other distributions that take into account these factors to more accurately estimate the number of fish. That is to say, based on the above binomial distribution, considering different conditions c ( c

1) In the Bernoulli experiment below, a mixed Beta-binomial distribution model is designed, with the EM algorithm (Expectation-Maximization Algorithm), through the Maximum Likelihood Estimation Principle, according to N strokes Use the estimated number of fish n _i , i =1,..., N from the density map to more accurately estimate the number of fish m in the culture pond. The Complete Likelihood Function for estimating the unknowns of a mixed beta-(Beta-) binomial distribution model can be defined as follows:

，z _ik 是用來指示第i筆資料是在c種狀況中哪一個 狀況的潛在變數(Latent Variable)。另外，在上述似然函數L中N,c,n _i ,α _k ,β _k 為是觀察到的值或者是設定的參數，皆是已知值，其中N為根據不同時刻成像設備取得的魚群影像之資料筆數，c為魚被觀察到的狀況數，n _i 為第i筆使用密度圖估計出的魚隻數量，α _k ,β _k 為有關p _k 的先驗機率(Prior Probability)之Beta分布的α，β參數。透過EM演算法，最大化似然函數L可求得所有未知數。根據養殖場域、魚種等條件，透過設定Beta分布的α _k ,β _k 參數來設定為第k種情況下魚被觀測到的機率p _k 的先驗機率。舉例而言，在本實施例中，由於本實施例是雙聲納系統，因此我們設定聲納探測魚群會有以下4(c=4)種情況：兩支聲納都觀測到很多魚，在此情況下，整體而言每隻魚被觀察到的事件皆是機率為p ₁的白努利試驗；第一視角聲納觀測到很多魚並且第二視角聲納觀測到很少魚，在此情況下，整體而言每隻魚被觀察到的事件皆是機率為p ₂的白努利試驗；第一視角聲納觀測到很少魚 並且第二視角聲納觀測到很多魚，在此情況下，整體而言每隻魚被觀察到的事件皆是機率為p ₃的白努利試驗；以及兩支聲納都觀測到很少魚，在此情況下，整體而言每隻魚魚被觀察到的事件皆是機率為p ₄的白努利試驗。因此，舉例而言，在本實施例中，我們將有關p ₁,p ₂,p ₃,p ₄的先驗機率貝他(Beta)分布之α，β參數分別設定為(2,2),(2,4),(2,4),(2,7)，其背後意義分別代表p ₁先驗機率為0.5附近較高，其先驗機率圖如圖7A所示，p ₂與p ₃先驗機率為0.25附近較高，其先驗機率圖如圖7B和7C所示，p ₄先驗機率0.125附近較高，其先驗機率圖如圖7D所示。將步驟501得到之連通區物件計數結果代入混合雙項分布模型進行計算，可以得到最終的魚隻數量估測結果。具體而言，魚群的種類、及養殖場域的條件，都會影響到密度圖中魚存在的機率，因此，除了單純的二項分布之外，還需考慮這些因素，因此，根據本發明會以二項分布為基礎，並與考慮這些因素的其它方程式相合併，產生混合二項分布式以更精準地估算魚隻數量。 In the above likelihood function L , m , p _k , z _ik are unknowns, where m is the total number of fish in the fish pond, p _k is the probability of the fish being observed in the kth case, and the value of z _ik is 0 or 1, k=1,...,c and

, zik is a latent variable used to indicate _which of the c conditions the i -th data is. In addition, in the above likelihood function L , N , c , n _i , α _k , β _k are the observed values or the set parameters, which are all known values, where N is the fish school obtained by the imaging device at different times The number of image data, c is the number of fish observed, n _i is the number of fish estimated by the i -th density map, α _k , β _k is the prior probability (Prior Probability) of p _k The alpha, beta parameters of the beta distribution. Through the EM algorithm, all unknowns can be obtained by maximizing the likelihood function L. According to the conditions of the breeding field, fish species, etc., by setting the α _k , β _k parameters of the Beta distribution, the prior probability of the observed probability p _k of the fish in the kth case is set. For example, in this embodiment, since this embodiment is a dual sonar system, we set the following 4 (c=4) situations for the sonar to detect fish schools: both sonars observe a lot of fish. In this case, the events observed for each fish as a whole are Bernoulli trials with probability p ₁ ; many fish are observed by the first-view sonar and few fish are observed by the second-view sonar, where In this case, the events observed by each fish as a whole are Bernoulli trials with probability p ₂ ; few fish are observed by the first-view sonar and many fish are observed by the second-view sonar, in this case , the events observed by each fish as a whole are Bernoulli trials with probability p ₃ ; and both sonars observed few fish, in which case each fish was observed as a whole The observed events are all Bernoulli trials with probability p ₄ . Therefore, for example, in this embodiment, we set the α and β parameters of the prior probability Beta distribution of p ₁ , p ₂ , p ₃ , and p ₄ to be (2, 2), respectively, (2,4), (2,4), (2,7), the meaning behind them respectively represents that the prior probability of p ₁ is relatively high around 0.5, and the prior probability diagram is shown in Figure 7A, p ₂ and p ₃ The prior probability is higher around 0.25, and its prior probability maps are shown in Figures 7B and 7C, and the prior probability of p ₄ is higher around 0.125, whose prior probability maps are shown in Figure 7D. Substitute the count result of objects in the connected area obtained in step 501 into the mixed binomial distribution model for calculation, and the final fish quantity estimation result can be obtained. Specifically, the type of fish and the conditions of the breeding field will affect the probability of fish in the density map. Therefore, in addition to the simple binomial distribution, these factors also need to be considered. Therefore, according to the present invention, the Based on the binomial distribution and combined with other equations that take into account these factors, a hybrid binomial distribution is produced to more accurately estimate fish populations.

Specifically, according to the present invention, the types of fish and the conditions of the breeding area are considered, and these considerations are covered by the various conditions of the fish images obtained by the sonar imaging device. Therefore, according to the Bernoulli test taking into account these different conditions, according to the present invention, a mixed beta binomial distribution model is designed, and the complete likelihood function of the unknowns of the mixed beta-binomial distribution model is generated accordingly, so as to be more Further accurate estimates of fish populations.

FIG. 8 is a block diagram illustrating an artificial intelligence fish estimation system 800 of the present invention. As shown in FIG. 8 , the system 800 includes an imaging module 801 , a neural network module 802 , a processing unit 803 , and a communication unit 804 . For example, the imaging module 801 can be a sonar imaging device, which is used to capture fish school images and transmit them to the neural network module 802 . The neural network module 802 is provided with a functional neural network with a trained neural network parameter Θ value obtained through artificial intelligence training, and can build a 2-dimensional fish density map generation function, for example, the 2-dimensional fish density The image generation function can be the formula (2) shown above: P(Y=1| X , Θ)* g (p; 0 _, Σ), the neural network module 802 will automatically generate the image sent by the imaging device 801 is converted into a representative density map and sent to the processing unit 803 . The processing unit 803 may use the fish quantity estimation method according to the present invention to estimate the appetite of the farmed fish population. The communication unit 804 can communicate with an external unit according to various communication protocols such as Lora, Wifi, GMS, etc., and the external unit can be a cloud network, the Internet of Things, aquaculture equipment, and the like.

Preferably, the units or modules in the system 800 according to the present invention can be formed together in the same device, or located at different places individually or in any combination with each other. For example, the imaging module 801, the neural network module 802, and the communication unit 804 can be integrated into a fish imaging device with artificial intelligence, and the processing unit 803 can be installed in the cloud, Internet of Things, water vehicles or other In computer equipment with communication functions. Alternatively, the imaging module 801 and the communication unit 804 are integrally provided in the farm domain, and the neural network module 802 and the processing unit 803 are provided in the cloud, Internet of Things, water vehicle or other computer equipment with communication functions.

In the above, the neural network training method, the artificial intelligence fish quantity estimation method, and the artificial intelligence fish swarm appetite judgment training system according to the present invention are described by way of example. From the above description, those skilled in the art should understand that the present invention has significant advantages and effects compared with the prior art. For example, according to the present invention, it is possible to effectively decide whether to change the breeding environment, change the feeding amount or formula, and take measures to promote the health of the fish according to the result of automatic estimation without disturbing the fish.

Although the preferred embodiments of the present invention have been described above, these are for illustrative purposes only and should not be construed as limiting the scope of the present invention. Without departing from the spirit of the present invention, Many modifications can be implemented by those skilled in the art, and the appended claims cover all such modifications that fall within the scope and spirit of the invention.

500: Method for Estimating Number of Fish

501: Connected area count step

502: Estimating the total number of fish using mixed binomial steps

Claims (11)

A neural network training method is used to generate a density map for estimating the number of farmed fish in a farm area, and use at least two imaging devices to capture many frames of fish images as many learning samples. The method includes the following steps: In the marking step, the learning samples are marked in the form of strong marking, weak marking, and no marking, so as to distinguish into strong pixel marking areas with one fish or no fish at all, and weak pixels with at least two fishes The marked area and the unmarked pixel area; the training step is to generate a target probability function according to the probability relationship between the strong pixel marked area and the corresponding learning sample, and to learn the corresponding learning sample according to the weak pixel marked area and the unmarked pixel area The respective probability relationships of the samples and the corresponding calculation results of the target probability function are relatively consistent with the expected probability, which is used as a training criterion to teach the neural network to become the target probability function neural network, so as to generate a system that can be used to estimate fish populations. Quantity density map. According to the training method of claim 1, the target probability function is P( y _i | X ,Θ), X represents a single frame of image, y _i represents the ith pixel in the whole frame of image, and Θ represents the neural network parameter , P( y _i =1| X ,Θ) is the probability that pixel i is marked with a fish, and the loss function is designed according to the training criterion as L (Θ|Θ ' ):

Among them, S represents all strong pixel marked areas, W represents the set formed by weak pixel marked areas, Ω represents the set of all pixels in an image, Θ is the current neural network parameter, Θ ' is the previous iteration of the neural network parameters, D ( p _i ) is the reference density map function, P _g ( y _i =1| X ,Θ) is the target density map function, α>0 and β>0 are the weight parameters; Among them, the loss function is iteratively calculated until the loss function is minimized, and the neural parameters Θ, and the weight parameters α and β obtained during the minimization are used as the target neural parameters and weight parameters, so that the neural network becomes a The target probability function neural network. The training method of claim 1, wherein the at least two imaging devices are two sonar imaging devices arranged at 180-degree intervals. A method for estimating the number of farmed fish using artificial intelligence to estimate the total number of fish farmed in a farm, the method includes the following: Use at least two imaging devices to obtain multiple frames of fish images; Use the training method of request item 1 to complete the training target probability function neural network, and convert the multi-frame fish images into a plurality of corresponding density maps; According to the configuration of the at least two imaging devices, each density map obtained by each imaging device at the same time is processed as if obtained from the same viewing angle; Using the connected object marking method, mark the connected area of each pixel whose probability value is greater than a predetermined threshold value in each density map obtained by each imaging device that has undergone the processing at the same time; Each density map obtained at the same time as the marked connected area is divided into a predetermined number of blocks with a × b pixels as the unit and sorted, and the number of the larger connected area in the blocks with the same order is the representative number, The number of representatives is added to obtain the total number of representatives, a and b are both natural numbers; Based on the binomial distribution, according to the fish species and the conditions of the breeding field, a mixed binomial distribution model is designed, and the total number of the farmed fish is estimated by using the representative total number obtained at different times and the predetermined parameter values representing each situation. . According to the estimation method of claim 4, the binomial distribution model is a mixed beta-binomial distribution model, and the complete likelihood function of the mixed beta-(Beta-) binomial distribution model shown below can be maximized. Find the total number of farmed fish: L ( m , p _k , z _ik | n _i , α _k , β _k )=

Among them, m represents the total number of farmed fish in the farm to be estimated, p _k is the probability of the fish being observed under the condition of the k - _{th fish farm, zik} is 0 or 1, k=1, ...,c,c is the total number of farm conditions, and

, z _ik is a latent variable used to indicate which of the c conditions the i-th data is in, N , c , n _i , α _k , β _k are all known values, where N is the imaging device at different times The number of fish images obtained, n _i is the number of fish estimated by the i -th density map using the density map, α _k , β _k are the α and β parameters of the beta distribution of the prior probability of p _k . The estimation method of claim 4, wherein the at least two imaging devices are two sonar imaging devices arranged at intervals of 180 degrees. The estimation method of claim 6, wherein the total number of conditions c of the farm is equal to 4. An artificial intelligence breeding fish quantity estimation system for estimating the total number of fish cultured in a breeding field, the system comprising: at least two imaging modules for photographing the fish population in the breeding field; a neural network module, including: The training method of claim 1 completes the training target probability function neural network; a processing unit; and a communication unit, wherein the neural network module and the processing unit cooperate with each other to convert the photographed image of the farmed fish school into a fish school density map, and, using the estimation method as in any one of claims 4 to 7, an estimate of the total number of farmed fish. The system of claim 8, wherein the at least two imaging modules comprise sonar imaging equipment. The system of claim 8, wherein at least one of the neural network module and the processing unit is set in the cloud, the Internet of Things or a breeding equipment. The system of claim 8, wherein at least one of the neural network module and the processing unit is integrated with the at least one imaging module.

Images