Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
  • G01B11/00—Measuring arrangements characterised by the use of optical techniques
  • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
  • G01B11/25—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures by projecting a pattern, e.g. one or more lines, moiré fringes on the object
• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
  • A01K61/00—Culture of aquatic animals
  • A01K61/90—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination
  • A01K61/95—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination specially adapted for fish
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V10/00—Arrangements for image or video recognition or understanding
  • G06V10/10—Image acquisition
  • G06V10/12—Details of acquisition arrangements; Constructional details thereof
  • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
  • G06V10/145—Illumination specially adapted for pattern recognition, e.g. using gratings
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V20/00—Scenes; Scene-specific elements
  • G06V20/60—Type of objects
  • G06V20/64—Three-dimensional objects
  • G06V20/653—Three-dimensional objects by matching three-dimensional models, e.g. conformal mapping of Riemann surfaces
• • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
  • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
  • Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
  • Y02A40/00—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production
  • Y02A40/80—Adaptation technologies in agriculture, forestry, livestock or agroalimentary production in fisheries management
  • Y02A40/81—Aquaculture, e.g. of fish

Definitions

• the present invention concerns a method for calculating physical dimensions, such as one or more of: 3D model, size, weight and volume, for free movable objects in water, in accordance with the preamble of claim 1.
• the present invention concerns a system for calculation of physical dimensions, such as one or more of: 3D model size, weight and volume, for freely movable objects in water, in accordance with the preamble of claim 18.
• the present invention is directed to calculation of one or more of: 3D model, size, weight and volume of fish in fish farming net cages, fish hatcheries where tubs are used, and similar applications.
• Reliable automatic systems for monitoring of fish in fish farms are desirable to optimize the operation and reduce the costs.
• the size of the fish, average weight, size distribution in the net cage, total biomass, including the condition of the fish, behavior and actual feed consumption are factors which are desirable to control.
• NO 332103 concerns a system and a method for calculation of size of marine organisms in water.
• the system utilizes a distance meter to measure distance to fish of which a camera is going to take an image of.
• the camera is arranged to shoot images down into the water, and the images are taken when the distance meter has detected an object within a metering range.
• the image and the information from the distance meter are transferred to an image analysis tool to determine the size and weight of the fish.
• NO 330423 describes a device and a method for counting fish or determination of biomass.
• a device for determination of the volume or mass of an object suspended in a medium is described.
• the volume or mass is calculated by using a 3D camera in combination with a grayscale 2D camera.
• NO 330863 describes a device and a method for measuring average weight and appetite feeding in fish farms, where a method and a system for recording substantially freely movable objects in a fish farming net cage. Images from a given number of cameras, from respective angles, are transferred to a data processing unit for processing. The data processing unit detects whether there is fish, pellet, faeces or other foreign elements in the image. Information of weight and volume is connected with data about the fish feeding. From EP 1 217 328 a method is known for providing a 3D image by projecting a known pattern on an object at a certain distance, and take an image of the object having the projected pattern.
• the peculiar with EP 1 217 328 is that the given pattern is formed by arranging alternating areas of local maximum and minimum light density.
• WO 2010/098954 describes a method for estimation of a physical dimension of an object, where a given light pattern is projected onto the object, whereupon the reflected light is detected and the collected data is processed in a computer to provide a three-dimensional presentation of the object, and the physical dimension of the object can then be calculated on basis on the three- dimensional structure.
• EP 1 659 857 Al describes a method for recording and estimating the weight of fish.
• a number of cameras, particularly CCD cameras record images of fish moving by the cameras in a transfer pipe. The fish is illuminated from different sides in the transfer pipe, and images of different parts of the fish are recorded by a sequence controller in such a manner that a combined image recording is produced, which is being used as a basis in estimation of the fish weight.
• a unit for performing measurements on fish moving in a transfer pipe is described, having at least two light sources on the wall of the transfer pipe for illumination of fish, including two or more cameras, particularly CCD cameras, arranged uniformly in a cross-plane around the circumference, to record reflections from the fish or shadow images of fish.
• EP 2 425 215 describes a contact-less system and a method for estimation of the mass or weight of a target object.
• the target object is depicted and a spatial illustration of the target object (the animal) is derived from the images.
• a virtual spatial model is provided from a characteristic object from a class of objects to which the target object belongs.
• the virtual spatial model is transformed for optimally matching the spatial presentation of the single animal.
• the mass or weight of the target object is estimated as a function of shape variables characterizing the transformed virtual object.
• NO 20101736 describes a system and a method for calculation of size of marine organisms in water, such as fish, where at least one camera is used, and image analysis tool to analyze images taken by the camera.
• the system comprises use of a distance meter to measure distance to fish of which the camera is to take images of, where said camera is arranged to take images down into the water of single fish at reception of a signal from the distance meter, and that images of the fish including information about the distance to the fish, is arranged to be transferred to the image analysis tool for determination of the size and weight of the fish.
• An obvious disadvantage of this is that the distance to the object is required to be able to calculate the volume of the object.
• the distance to the object is used, and from this, the size of the object is calculated by counting number of pixels with fish in the image and distance. Since the distance only is measured in one point, the distance measurement is quite inaccurate and dependent on how the object is moving or is positioned. It should also be noted that this solution only is arranged to take images of single fish based upon the distance meter, something which results in that fish to be evaluated must be within the metering range of the distance meter, resulting in a limited data basis for statistical analysis.
• US 2004/0008259 Al describes a system which is recording physical dimensions of an object, consisting of a light source which illuminates the object, and a camera taking images of the object.
• the light source is illuminating the object by light having known wavelength and pattern, and the camera has filter for filtering out light not originating from the light source, so that light from the environments not interfere with the measurements.
• a processor calculates the physical size of the object from the images.
• This publication is arranged for use in air and not for use in water, and is not able to handle back-scattering from particles, algae or similar in water.
• Particles, algae, high fish density and similar substantially affects the light conditions in a net cage. This makes the use of computer-aided imaging challenging, since the system has to take these varying conditions into consideration, which the present systems fail to conduct satisfactorily.
• the main object of the present invention is to provide a method and a system solving the disadvantages of the prior art mentioned above.
• An object of the present invention is to provide a system and a method which is capable of estimating physical dimensions for freely movable objects in water, also when the light conditions and visibility in general are bad and/or varying.
• the present invention provides a system and a method for measuring physical dimensions for freely movable objects in water, particularly of freely movable fish in fish farming net cages, fish hatcheries where tubs are being used, or similar applications by using at least one light source arranged to illuminate an object by emitting light or structured light at a given wavelength, including at least one recording means in the form of a 2D camera for recording still images or video of the illuminated objects.
• the system comprises at least one light source arranged to project a predefined or pre-selected light pattern at a selected wavelength on freely movable objects to be observed, including at least one recording means in the form of a 2D camera arranged to depict (still image or video) the freely movable object being observed having the light pattern from the light source projected thereon.
• the system comprises at least one light source being arranged to illuminate freely movable objects to be observed by light having a selected wavelength, including at least two recording means in the form of at least two 2D cameras arranged to provide a stereoscopic vision system.
• the system comprises a control unit arranged to control the light source to emit light having given properties, such as selection of wavelength (colour), intensity, frequency, etc.
• the control unit will be arranged to select actual light pattern when a light pattern is to be projected onto the object.
• the system may comprise a separate image processing unit provided with means and/or software arranged to utilize the information from the illuminated object or objects, on which the light pattern is projected, recorded by the recording means to generate a 3D model of the freely movable object.
• the image processing unit or its functions may be arranged in the control unit or in an external unit.
• a light pattern is preferably used that covers as much of the object as possible. Speed calculation is difficult due to the object may have varying speed, e.g. fish will have varying speed due to it is using the tail when swimming. By projecting the object with a pattern, only one image is required to calculate the 3D model.
• the 3D model can be used to estimate volume and weight on basis of a function of shape variables characterizing the depicted object. Measured parameters in the 3D model, such as length and height, are inserted into a calculation formula for estimation of weight.
• a substantial advantage of the present invention is that it enables calculation of weight of one or more fish at a time in every image depending on the fish density.
• the system comprises static optical or programmable filters arranged to the recording means, said filters being arranged accept selected wavelengths, thus obtaining homogenous quality of the recorded images.
• selected wavelength will mainly be the wavelength which is sent from the light source to illuminate the object or to project the light pattern onto the freely movable object, which e.g. will be light in the infrared wavelength range.
• the object is advantageously illuminated/projected by the light source at a defined angle in relation to the recording means.
• the recording means are provided with static optical or programmable filters this results in that unwanted reflection is avoided, so that the recording means collect correct light/light pattern and produces images which can be used in automatic image processing in the image processing unit. Since light absorbs quickly in water, it is particularly important to use filter combinations to obtain images with sufficient quality. Additionally, the conditions in a net cage change rapidly. This makes presently known image processing difficult and this is one of the reasons that there are no automatic optically based biomass meters on the market today.
• filtering and correct angle is important with regard to minimizing reflections from, e.g. the shiny surface of the fish. Too much reflection will result in saturation of image chip due to too much light.
• the water absorption also makes this angle parameter a lot more important than in air, since it is particularly important to obtain as much reflection of the light as possible.
• the anisotropic properties of the fish-scales results in that the light is scattered in different directions, and therefore the energy in the light is scattered and is absorbed faster in the water, resulting in less light reaching the recording means.
• Infrared light is absorbed quickly in water, which means that at a certain depth there is no noise from infrared light from the sun. For that reason, only infrared light from an artificial light source is visible in the images. Homogenous measurement conditions are obtained by filtering away all wavelengths shorter than the artificial light source. In order to ensure good measurements due to the light being absorbed quickly in water, a light source having correct wave length and correct density must be used.
• the contrast in water decreases by increasing distance to the object. Since the contrast is low in underwater images, it is necessary to add information to the images, which in the present invention has been done by projecting a pattern onto the object. Adding information to the image by the projected pattern will also make it possible to distinguish several objects from each other, since 3D models of the objects may be generated irrespective of the distance to them being different, something which enable calculation of volume, weight and size therefrom.
• the system must be calibrated in water to increase accuracy of the measurements. This must be done to take into consideration that the refraction index in air versus water is different.
• the present invention will not suffer at conditions like this, since the light source can be arranged with selected wavelength, intensity, frequency, etc. so that the present invention may take these conditions into consideration and thus provide images of homogenous quality for image processing.
• the present invention is based on an optical system combined with artificial light source, where the light source illuminates objects or projects a light pattern onto the object, such as fish, at a selected wavelength, and filter away all remaining light from the recording means by means of filters, or use an image producing sensor which only collects the selected wavelength.
• a homogenous measuring environment is achieved. For example, by using a wavelength, such as in the infrared wavelength range, which is quickly absorbed in water, low interference is obtained from other light sources (such as sunlight or other artificially supplied light).
• a homogeneous quality of the recorded images for use in image processing is obtained by filtering away/minimizing other wavelengths than the light supplied from the light source or utilize an image producing sensor that collects selected wavelength, i.e. the images are without interference from other light sources.
• the homogeneous image quality makes it possible to perform a robust and accurate image processing and 3D modelling.
• the 3D model is, in accordance with the present invention, calculated by triangulation between light and image, or between images from several cameras. This provides a "depth image" where the pixel value represents distance to the measured object.
• the present invention When the object to be observed is fish, it is an advantage to arrange the present invention close to the feeding area for the fish in a net cage, since all fish is passing to eat during a feeding period and will provide a best possible representative selection of measured fish.
• the present invention may naturally be placed anywhere in the net cage. By placing the system in accordance with the present invention in the vicinity of the feeding area, it is not necessary to move the system around in the net cage to seek out the fish where it resides.
• the goal of fish farmers is that all fish must appear at the feeding area to eat during the day, something which results in that both small and large fish will pass the present invention during a feeding cycle.
• the present invention may also be provided with a winch or similar for measurement through a water column, and for elevation and lowering in the net cage, alternatively also arranged to be moved around within the net cage if desired by means of suitable means for this.
• the present invention is also applicable in estimation of number of fish, by using data about fish density in a model for this.
• Figure 1 shows a principle drawing of a system in accordance with a first embodiment of the present invention using structured light
• Figure 2 shows a principle drawing of a system in accordance with a second embodiment of the present invention which is using a light source for illumination of the object to be observed
• Figure 3 shows a principle drawing of a system in accordance with the present invention located in a net cage
• Figure 4 shows a block diagram of a system in accordance with the present invention.
• Figure 1 shows a principle drawing of a system in accordance with a first embodiment of the present invention, to be arranged in a net cage for observation of fish.
• a system in accordance with the first embodiment of the present invention comprises at least one light source 10 arranged for emitting structured light at a selected wavelength, and at least one 2D camera 11 provided with an optical filter 12.
• the 2D camera 11 may be arranged to take still images only, video only or both video and still images.
• the light source 10 is, to enable emission of structured light, provided with means and/or software for emitting a known or selected light pattern 13 which is to be projected on fish 100 or other objects residing within the illumination range of the light source 10 at a selected wavelength.
• structured light basically only one 2D camera is required for image recognition, but using several cameras may also be an advantage, which will be described further below.
• the light pattern 13 projected by the light source 10 may exhibit any selected pattern.
• the pattern may be formed by one or more of (but not limited to the list): - either horizontal or vertical lines,
• the pattern as a whole should preferably cover the entire object to be observed.
• Figure 2 shows a principle drawing of a system in accordance with a second embodiment of the invention, to be arranged in a net cage for observation of fish.
• a system in accordance with the second embodiment of the present invention comprises at least one light source 10 arranged to illuminate the object at a selected wavelength, and at least two 2D cameras lla-b arranged to provide a stereoscopic vision system.
• at least two 2D cameras are required to enable generation of a 3D model of the object.
• Two or more cameras are required to enable calculation of distance to the object and thereupon generating 3D model.
• the 2D cameras lla-b may be arranged to take still images only, video only, or both video and still images.
• the light source(s) 10 is/are, both in the first and second embodiment, arranged to emit light having desired parameters with regard to wavelength (color), optionally frequency, intensity, etc.
• the light source is preferably of the type light emitting diode (LED), laser or another type arranged for emitting light at selected wavelength, including light sources where only selected wavelength is filtered out.
• the 2D camera(s) is/are, both in the first and second embodiment, provided with optical filters 12, 12a-b arranged to accept light only at selected wavelengths.
• the 2D cameras 11, lla-b are provided with image producing sensors which only collects selected wavelength.
• Use of several 2D cameras lla-b provides more possibilities, optionally also in combination with several light sources or controllable light sources.
• the 2D cameras 11, lla-b may, for example, be provided with different filters 12a-b, i.e. filters having different properties.
• one of the cameras may be provided with an I filter which only accepts IR light.
• the light source 10 emits a light/light pattern with IR light
• this camera will therefore accept the light/light pattern.
• the other camera may, for example, be arranged with a filter which accepts only green light.
• a 3D model may be generated on basis of the two images in combination.
• light sources are being used, for example two light sources. Then, for example, both light sources may emit light with a pattern at different wavelengths and the two cameras 12a-b collect separate patterns, whereupon the images are combined in generation of a 3D model. Moreover, the light source may emit light at different wavelengths to obtain best possible images or video for processing.
• the system advantageously comprise a suspension device 20, e.g. formed of a suspension point 21 and a rod 22 arranged vertically from the suspension point 21, said rod 22 being adapted for attachment of the 2D cameras lla-b and the light source 10.
• the light source 10 is in the example shown in Figure 2 arranged between the two 2D cameras lla-b, at a given distance between the 2D cameras lla-b and the light source 10, and that the light source 10 exhibits an angle in relation to the fish to be observed, whereas it in the first embodiment is located under the 2D camera 11, but it could naturally also have been located above the camera.
• the light source 10 may be located anywhere in relation to the 2D camera(s) depending on the light reflection from the object to be illuminated.
• FIG. 3 shows a principle drawing of a system in accordance with the first embodiment of the invention arranged in a net cage 30 in the vicinity of a feeding station 40.
• the system in accordance with the invention is in this example connected to a winch 50 for elevating and lowering the system in the net cage.
• the system may be arranged to means for moving the system around within the net cage 30, if desirable. It may be seen from the figure that a light pattern 13 is projected onto a fish 100' by the light source 10, while other fishes 100 do not have a light pattern projected thereon.
• the system may be arranged stationary in the feeding area, so that the users don't have to move the system around, as all fish usually visit the feeding area during the course of a day.
• the system comprises an image processing unit 60 provided with means and/or software to utilize the information from the light pattern 13 recorded by the 2D camera 11 to generate a 3D model of the fish 100' being illuminated (for the first embodiment) or provided with means and/or software to utilize the information from the stereoscopic vision created by the two 2D cameras lla-b (for the second embodiment).
• the image processing unit 60 may be a separate unit, a unit integrated in a control unit 70 for the system, or an external unit 80.
• the system comprises a control unit 70 which may comprise the image processing unit 60, and is moreover provided with means and/or software for controlling the light source(s) 10, the recording means 11, lla-b and optionally the filter(s) 12, 12a-b.
• the control unit 70 will provide the filters 12, 12a-b with settings informing which wavelength the light source 10 is emitting light with, so that the filters 12, 12a-b are adjusted to receive light only at this wavelength.
• the system is provided with communication means for communication with an external unit 80, which external unit 80 may also be the control unit of the system.
• the control unit 70, the image processing unit 60 or the external unit 80 are provided with means and/or software for utilizing the generated 3D model for estimation of physical dimensions of the depicted fish, such as size, volume and weight, based on a function of shape variables which are characterizing the depicted object. Measured parameters (for example length and height) from the 3D model are used in a model for estimating weight.
• the weight of singular fish is advantageously stored in a database, for example, in the external unit 80 or the control unit 70, so that the average weight may be calculated when the data basis has become large enough for this.
• the total biomass in the net cage can be calculated from the average weight and the total number of fish.
• the size distribution may be found as well.
• control unit 70 or the external unit 80 is arranged to present the calculated physical dimensions to the user.
• control unit 70 may be arranged to control the winch 50 for elevating and lowering the system in the net cage 30, including any other means for moving the system in accordance with the present invention around within a net cage 30.
• control unit 70 is also arranged to control the feeding station 40 on basis of image information from the recording means.
• the present invention also have another field of use by that the images/videos acquired also may be used to search for, e.g. salmon louse, deformations, injuries, illness and similar.
• the present invention is not limited to net cages but may also be used in fish hatcheries, where tubs are used, and similar applications.

Landscapes

• Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Computer Vision & Pattern Recognition (AREA)
• Life Sciences & Earth Sciences (AREA)
• Multimedia (AREA)
• Software Systems (AREA)
• Zoology (AREA)
• Environmental Sciences (AREA)
• Artificial Intelligence (AREA)
• Marine Sciences & Fisheries (AREA)
• Biodiversity & Conservation Biology (AREA)
• Animal Husbandry (AREA)
• Length Measuring Devices By Optical Means (AREA)
• Length Measuring Devices With Unspecified Measuring Means (AREA)
• Geophysics And Detection Of Objects (AREA)
• Measuring Arrangements Characterized By The Use Of Fluids (AREA)

Priority Applications (2)

Applications Claiming Priority (2)

Publications (1)

Family

ID=50978780

Family Applications (1)

Country Status (5)

Cited By (24)

Families Citing this family (4)

Citations (2)

Family Cites Families (3)

• 2012
  • 2012-12-20 NO NO20121541A patent/NO337305B1/no active IP Right Review Request
• 2013
  • 2013-12-20 WO PCT/NO2013/050231 patent/WO2014098614A1/en active Application Filing
  • 2013-12-20 CA CA2895758A patent/CA2895758A1/en not_active Abandoned
  • 2013-12-20 EP EP13864165.9A patent/EP2936051A4/en not_active Ceased
• 2015
  • 2015-06-18 CL CL2015001722A patent/CL2015001722A1/es unknown

Patent Citations (2)

Non-Patent Citations (2)

Also Published As

Similar Documents

Legal Events