AU2020102433A4

AU2020102433A4 - Machine learning based fish monitoring machine and method thereof

Info

Publication number: AU2020102433A4
Application number: AU2020102433A
Authority: AU
Inventors: Akash Kumar Bhoi; Sailaja G.; S. Jagadeesh; Siva Nagireddy Kalli; Arshad Mohammed; Bibi Maryam; Raju Muthyala; B. Narendra Kumar; G. Prasanna Kumar; K.C Ravikumar; P. Satish Kumar; Mahaboob Shaik; R. Srinu Naik; Korrapati Tulasiram
Original assignee: G Sailaja Dr; Jagadeesh S Dr; Kalli Siva Nagireddy Dr; Maryam Bibi Mrs; Muthyala Raju Dr; Narendra Kumar B Dr; Prasanna Kumar G Dr; Ravikumar K C Dr; Satish Kumar P Dr; Srinu Naik R Dr
Current assignee: G Sailaja Dr; Jagadeesh S Dr; Kalli Siva Nagireddy Dr; Maryam Bibi Mrs; Muthyala Raju Dr; Narendra Kumar B Dr; Prasanna Kumar G Dr; Ravikumar KC Dr; Satish Kumar P Dr; Srinu Naik R Dr
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2020-11-05
Anticipated expiration: 2028-09-25

Abstract

"MACHINE LEARNING BASED FISH MONITORING MACHINE AND METHOD THEREOF" Exemplary aspects of the present disclosure are directed towards the MACHINE LEARNING BASED FISH MONITORING MACHINE AND METHOD THEREOF consisting of Thermal Camera 101 and Camera-module 102 placed in each chute 107, Microcontroller 103, the plurality of RFID reading/writing devices 105 and plurality of RFID tags 106 implanted in fishes. The Microcontroller 103 capable of collecting both thermal and visual images and process them using trained Machine Learning Algorithms 104 to monitor, count and determine health conditions of fishes in a farm pond or cage culture. When fish enter the chute, Microcontroller 103 determines, the fish size, temperature, weight and any anomaly with the help of images and relevant Machine Learning Algorithms and data-set available. If found any abnormality, the fish is trapped in the chute and intimated to the user else updated data along with previous data with timestamp updated in RFID tags 106 and user interface. 1/6 103 101 102 105 107 107 109 FIG I THERMAL IMAGING BASED FISH MONITORING MACHINE

Description

1/6

103

101

102

105 107

107 109

FIG I THERMAL IMAGING BASED FISH MONITORING MACHINE I TITLE MACHINE LEARNING BASED FISH MONITORING MACHINE AND METHOD THEREOF PREAMBLE TO THE DESCRIPTION

The following specification particularly describes the invention and the manner in which it is to be performed.

DESCRIPTION TECHNICAL FIELD

[0001] The present disclosure generally relates to the fish monitoring system, more particularly about the implementation of machine learning, thermal imaging and RFID systems for monitoring the fishes in closed precision farming.

BACKGROUND

[0002] Without limiting the scope of the invention, its background is described in connection with devices, programs, and methods relating to fish counting or monitoring devices as an example.

[0003] Aquatic culture taking a drastic transformation not only due to technological advancement but the increase of demand also. To coup up with the market, especially in rare and precious fisheries wing started cage culture and captive breeding. This lead to the use of advanced technologies in every aspect of cultivating fishes in a closed environment. The present disclosure focuses on overcoming the disadvantages of invention and techniques available in the market, as discussed below.

[0004] In a prior art U.S. Patent Publication No. 2004/0249860, an apparatus for collecting, storing and transmitting fishing information is described, and in its Abstract states, a system for collecting, storing, processing and transmitting fishing or other sport information includes a data logging and processing module and connected sensors for automatically collecting data during participation in the sport. A data communication module collects data input by the user. Data from both modules is input into a personal computer connected to a web site that uses the data from the modules and retrieving related data to provide virtual guide service.

[0005] Similarly, in U.S. Pat. No. 7161872, a fishing data display device, is described. Its Abstract states a fish depth monitor is a device that can display first water depth data that indicates the location of terminal tackle that is engaged on fishing line reeled out from a spool, and second water depth data that shows the area of the bottom of a fishing location transmitted from a fish finder, and includes a case, first and second reception portions, a display unit, and a control unit. The first and second reception portions receive the first water depth data from an electric driven reel and the second water depth data from the fish finder. The control unit graphically displays the received first and second water depth data in positions that correspond to the water depth on the display unit. It is possible to intuitively determine the positional relationship between the location of the terminal tackle and the area of the bottom.

[0006] In U.S. Pat. No. 7236426, integrated mapping and audio systems are described. Abstract states an integrated position mapping system and an integrated sonar mapping system both permit recording, storage and playback of audio data. Audio data is provided to the integrated sonar mapping system or the integrated position mapping system so that it may be correlated to position data or echo data. The integrated sonar mapping system includes a sonar transducer for emitting and receiving sonar signals that may be subsequently be processed to provide echo data from objects in the water which reflect sonar signals. The integrated position mapping system includes a position receiver for providing position data. A controller not only processes the data for storage, but also correlates the audio data to echo data or position data. The correlated data may, therefore, be retrieved for playback of the audio data and display of the position data or echo data.

[0007] In the prior U.S. Pat. No. 6222449. A remote fish logging unit is described. It explains a portable recording device, namely a remote logging unit, for electronically recording relevant information related to fishing conditions, and the like. The recording device has sensors for detecting environmental conditions and/or for measuring physical data on a specimen caught. The device also includes input mechanisms, such as a touch screen, for manually entering information, and a display for reviewing information stored in the device's memory. The recording device may be connectable to a personal computer for creating a personal log of the user's activities and/or for loading additional information into the device. The device may transfer recorded data to a central repository, for example using a transmitter/receiver for sending a data signal to a network server which maintains a database of information related to fishing conditions at a number of locations. The network server may receive and compile information from a number of remote units at various locations, thereby providing a system for sharing such information. A remote unit may contact the server from a remote location and request information on fishing conditions for a selected location.

[0008] Another Prior art TW1577961B which issued in 2012, with title Method for measuring size, electronic apparatus with a camera. - the explained invention relates the to an electronic device with the camera such as the digital camera, a mobile phone, a mobile device, and the like, and an application of the camera-equipped electronic device. In recent years, with the spread of digital cameras and camera-equipped mobile devices, various types of captured objects can be easily photographed and recorded. And, all smartphones can perform various application operations by downloading programs. Patent Document 1 relates to a screen photographed by a video recorder using water, and a telescopic free positioning 21 is provided for the video recorder. The distance between the video recorder and the object to be photographed, the distance from which the image is taken, and the pixel of the captured image are displayed, and the measurement size is shown.

[0009] The present invention provides significant and rapid video imaging, both standard and thermal for ascertaining the fish size, length, weight and health conditions. The temperature measurements enabled with machine-learning helps in establishing the health offishes. The colour pattern observation helps in determining skin/scales related problems in the fish. These aspects help the user for understanding health conditions and predicting deceases.

[0010] The present invention addresses the shortcomings mentioned above of the prior art.

[0011] All publications herein are incorporated by reference to the same extent as if each publication or patent application were specifically and individually indicated to be incorporated by reference. Where a definition or use of a term in an incorporated reference is inconsistent or contrary to the definition of that term provided herein, the definition of that term provided herein applies, and the definition of that term in the reference does not apply.

SUMMARY

[0012] The following presents a simplified summary of the disclosure in order to provide a basic understanding of the reader. This summary is not an extensive overview of the disclosure, and it does not identify key/critical elements of the invention or delineate the scope of the invention. Its sole purpose is to present some concepts disclosed herein in a simplified form as a prelude to the more detailed description that is presented later.

[0013] Exemplary embodiments of the present disclosure are directed towards the Machine Learning-Based Fish Monitoring Machine And Method Thereof.

[0014] An exemplary object of the present disclosure is directed towards a system that monitors and detects the fish health and determines the dimensions. Also, to identify the anomaly in the fish temperature and scale colour for predicting health condition.

[0015] An exemplary aspect of the present subject matter is directed towards integrating Thermal camera 101 and Normal camera with microcontroller 103 to form a Fish Monitoring Machine(FMM) 100. Microcontroller 103 executes the relevant Machine Learning Algorithms (MLA) to identify the fish, its type, its dimensions, its colour, temperature.

[0016] Another exemplary object of the present disclosure is directed towards integrating the FMM 100, and RFID Read/Write device for reading and writing the data of the fish that is, its type, its dimensions, its colour, and the temperature on to the RFID tags implanted in the fish.

[0017] Another exemplary object of the present disclosure is directed towards integrating Thermal camera 101 with microcontroller 103 along with relevant MLA to identify the temperature of the fish and find any anomaly in the temperature reading.

[0018] An exemplary aspect of the present subject matter is directed towards the use of Video camera 102, microcontroller 103 along with relevant MLA for identifying the colour on the scales of the fish. Wherein the irrelevant coloured patches on the scales of fish denotes prevailed disease condition.

[0019] An exemplary aspect of the present subject matter is directed towards the use of Video camera 102, microcontroller 103 along with relevant MLA for identifying the type offish by identifying shape and size of scale, fin, tail and head portion of thefish.

[0020] Another exemplary aspect of the present disclosure is directed towards notification and sharing of the fish data and anomaly data with the user on the mobile app.

[0021] Another exemplary aspect of the present disclosure is directed towards the post anomaly detection process wherein the fish will be trapped in the chute once an anomaly is detected.

Brief Description of the Drawings

[0022] In the following, numerous specific details are set forth to provide a thorough description of various embodiments. Certain embodiments may be practised without these specific details or with some variations in detail. In some instances, certain features are described in less detail so as not to obscure other aspects. The level of detail associated with each of the elements or features should not be construed to qualify the novelty or importance of one feature over the others.

[0023] FIG.1 is a diagram depicting the Machine Learning-Based Fish Monitoring Machine And Method Thereof, according to an exemplary embodiment of the present disclosure.

[0024] FIG. 2 is a 100 Component Architecture of Farm Monitoring Machine, according to an exemplary embodiment of the present disclosure.

[0025] FIG. 3 is a representation of 300, the process executed in Fish Monitoring Machine, according to an exemplary embodiment of the present disclosure.

[0026] FIG. 4 is a representation of 400 Machine Learning Algorithm 1 (MLA 1) to detect the type of fish and dimensions of the fish, according to an exemplary embodiment of the present disclosure.

[0027] FIG. 5 is a diagram 500 Machine Learning Algorithm 2 (MLA 2) to detect the abnormality in heat signature, according to an exemplary embodiment of the present disclosure.

[0028] FIG. 6 is a diagram depicting 500 Machine Learning Algorithm 3 (MLA 3) to detect the abnormality in colour, according to an exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0029] It is to be understood that the present disclosure is not limited in its application to the details of construction and the arrangement of components outlined in the following description or illustrated in the drawings. The present disclosure is capable of other embodiments and of being practised or of being carried out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting.

[0030] The use of "including," "comprising," or "having" and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. The terms "a" and "an" herein do not denote a limitation of quantity, but rather denote the presence of at least one of the referenced item. Further, the use of terms "first," "second," and "third," and the like, herein do not denote any order, quantity, or importance, but rather are used to distinguish one element from another.

[0031] Referring to FIG. 1 is a diagram depicting the Thermal Imaging-Based Fish Monitoring Machine consisting of Fish Monitoring Machine (FMM) 100 placed in a chute 107, made of silicon rubber based material forming a tunnel type structure to pass the fish through. Wherein FMM 100 mainly consists of Thermal Camera 101, Video Camera 102, a plurality of RFID (Radio Frequency identification) Read/ Write Device 105 integrated with a microcontroller 103.

[0032] Further to it, when a fish enters the chute 107, the microcontroller 103 executes the steps 301 to 303 of the process 300 to read the fish data if already present in the RFID tags implanted in the fish through RFID (Radio Frequency identification) Read/ Write Device 105. Simultaneously if no data is available in the fish RFID tag then the microcontroller 103 executes the step 304 of process 300 to detect the fish type and other physical parameters. By executing the remaining steps if any anomaly is found in the fish then the microcontroller 103 closes the latch 109 of the chute 107 and inform the user through Mobile App using GPRS module 103a and trap the fish inside till the user arrives.

[0033] In accordance with a non-limiting exemplary embodiment of the present subject matter, FIG. 2 is a Component Architecture of Farm Monitoring Machine (FMM) 100. The microcontroller 103 is typically a JETSON NANO capable of executing Al & ML algorithms and can to parallel computing. The inheriting capabilities made this microcontroller 103 best suited for this application. The major disadvantage of this microcontroller 103 is lack of communication network and hence to overcome this, connected a GPRS module 103a. The cameras both thermal 101 and standard 102 are connected to the microcontroller 103 to acquire the live feeds of the fish images entering inside the chute. The thermal camera acts as a motion sensor and detects the fish based on the heat signature.

[0034] Referring to FIG 3 is a diagram depicting a Process 300 executed in Fish Monitoring Device. Once the microcontroller 103 detects heat signature through Thermal camera 101 and ascertains that fish entered the chute 107 is starts the process 300. The process begins at step 301, to acquire the live thermal images of the fish arriving inside the chute 107 through thermal camera 101. In step 302, the microcontroller 103 to obtain the live video feeds of the fish. In step 303, the microcontroller 103 read the fish data from RFID tag 106 implanted in fish and retrieve the Fish ID and other parameters. Step 304 gets executed by microcontroller 103 to detect the type of fish and dimensions of the fish by running Machine Learning Algorithm 1 ( MLA-1 ).

[0035] Further to it, in step 305, the microcontroller 103 identify any abnormal heat signatures of the fish by executing Machine Learning Algorithm 2 (MLA-2). Similarly, in step 306, the microcontroller 103 identify any irregular skin colour pattern of the fish using Machine Learning Algorithm 3 (MLA-3). In Step 307, microcontroller 103 refers the data pertaining to steps 303 to 306 and if no abnormality found then write the acquired data from steps 303 to 306 ( Fish Type, Time &date, Signature ID, temperature and measurements of fish) on RFID concerned fish. If anomaly detected then in step 308, microcontroller 103 intimate the user with the data (Fish-Type, Time &date, Signature ID, temperature and measurements of fish) and write same on RFID of concerned fish and trap the fish inside the chute 107 by closing the gate latch 109 till user releases the latch.

[0036] In accordance with a non-limiting exemplary embodiment of the present subject matter, FIG. 4 depicting the 400 Machine Learning Algorithm 1 ( MLA 1) to detect the type of fish and dimensions of the fish. Starting at step 401, microcontroller 103 acquire Video Images of the fish from Video camera 102. In successive step 402, microcontroller 103 Search and Match the images in databases for identifying the shape of Head, Gills Shape, Fin Shape and Scales shape to predict the type of fish. In step 403, microcontroller 103 process the image and based on pixel strength and other image processing methods expect the dimensions of the fish, that is length and height. Further, Microcontroller 103 in step 404, Based on measurements and values relevant in the weight table ascertain the weight of the fish and update the information by tagging the fish ID in the database.

[0037] Following is a non-limiting exemplary embodiment of the present subject matter, as shown in FIG. 5, which is a 500 Machine Learning Algorithm 2 (MLA 2) to detect the abnormality in heat signature of the fish. The process starts at step 501, wherein microcontroller 103, Acquire Thermal Images of the fish from Thermal camera 101. In step 502, it searches and Matches the thermal signatures in databases for identifying any abnormality in fish temperature. The databases are kept ready with the training set for the random forest search algorithms having a knowledge base of fish types and their body temperature. In successive step 503, microcontroller 103, calculate the percentage change in temperature of the fish, if the change is more than +/- 10% then flag the fish for abnormality. Else in step 504, it signals that the difference is in limits of +/- 10% and states fish as healthy.

[0038] Following is a non-limiting exemplary embodiment of the present subject matter, as shown in FIG 6 is the 600 Machine Learning Algorithm 3 ( MLA 3) to detect the abnormality in colour. In step 601, microcontroller 103 Acquire Video Images of the fish from Video camera 102. In step 602, it Searches and Match the scales shape and size of the fish and identify the fish type and predict approximate age. In step 603, microcontroller 103, Compare the database with the colour of fish scales and confirm that there is no colour change. If it found an unwanted colour pattern, then in step 604, microcontroller 103 identifies the colour of the scale and match the colour in diseases table (DB-1) to determine exact abnormality. The abnormality is flagged and sent to the user as a notification in the Mobile App. Subsequently, microcontroller traps the fish in the chute by closing the Gate 109.

Claims

I STATEMENT OF CLAIMS We Claim:

1. Thermal Imaging based Fish Monitoring Machine and Method Thereof: consisting of Thermal-Camera 101 and Camera-module 102 placed in each chute 107, Microcontroller 103, the plurality of RFID reading/writing devices 105 and plurality of RFID tags 106 implanted in fishes; and Microcontroller 103 determines, the fish type, size, temperature, dimensions, weight, scale colour and any anomaly with the help of images and relevant Machine Learning Algorithms and data-set available; and If Microcontroller 103 found any abnormality, the fish is trapped in the chute 107 and intimated to the user else updated data along with previous data with timestamp updated in RFID tags 106 and Mobile App;

2. The device, as claimed in claim 1, by using thermal camera 101, the microcontroller 103 by executing the relevant machine learning algorithm predicts the anomaly in heat signature of the fish.

3. The device as claimed in claim 1, wherein Microcontroller 103 uses video feeds from by using video camera 102 and by executing the relevant machine learning algorithm, determines scale colour and thereby ascertains the decease contracted by the fish.

4. The device of claim 1, wherein microcontroller 103 automatically reads the data from RFID tag 106 when fish enters the chute 107 using RFID reader 105 and write the updated data on to the RFID tag 106 when fish leaves the chute 107.

5. The device of claim 1, wherein microcontroller uses video feeds from by using video camera 102 and by executing the relevant machine learning algorithm, determines the type of fish by matching fish scale, fin, tail, gill and head shape.