Classifications

• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
  • A01K61/00—Culture of aquatic animals
  • A01K61/90—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination
• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
  • A01K61/00—Culture of aquatic animals
  • A01K61/90—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination
  • A01K61/95—Sorting, grading, counting or marking live aquatic animals, e.g. sex determination specially adapted for fish
• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
  • A01K67/00—Rearing or breeding animals, not otherwise provided for; New or modified breeds of animals
  • A01K67/033—Rearing or breeding invertebrates; New breeds of invertebrates
• • A—HUMAN NECESSITIES
  • A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
  • A01K—ANIMAL HUSBANDRY; AVICULTURE; APICULTURE; PISCICULTURE; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
  • A01K79/00—Methods or means of catching fish in bulk not provided for in groups A01K69/00 - A01K77/00, e.g. fish pumps; Detection of fish; Whale fishery

Definitions

• the invention relates to detection of features, structures and/or organs in organisms like insects, shrimp and fish, more specifically to the imaging-based detection and classification.
• Imaging of objects from two or more directions is highly important for visualizing some features that otherwise may not be seen when viewed from a single direction. It is particularly useful for analyzing developing organs in insect or fish larvae. When viewed from a single direction, as with a microscope, these organs may be obscured or hidden by fat tissue or other organs, making it difficult to determine their presence, color, morphology, size and shape. Single-perspective systems may address this problem by acquiring multiple images of the object along the optical axis. However, this method requires a significant amount of time to collect the multiple images and therefore cannot be readily applied for high throughput. Moreover, this method may further require a complicated and costly setup of adaptive optics or confocal microscopy.
• Flow cytometery systems offer the ability to optically analyze a large number of samples in a rapid and automated fashion. They are typically designed for imaging microscopic objects much smaller than insect or fish larvae, such as single cells. This is usually done from a single direction and with a narrow depth of field (due to the high numerical aperture and magnification of the objectives used).
• a number of methods for addressing the three-dimensionality of objects in flow cytometry have been previously discussed. They are primarily intended for mapping the external 3D shape of small target objects (US 6,049,381 A, US 8,009,189 B2 and US 8,121,388 B2). As to detecting and mapping internal structures, this currently requires that they be fluorescent (US 5,644,388 A).
• the identification and separation of male and female organisms is necessary in various agricultural and industrial aspects.
• industrial fish farming seeks to optimize reproduction for increased yield. This depends, in part, on the sex-ratio of males-females in the breeding tanks.
• some fish play an important role in genetic research, as they have short life cycles and are simple to work with and contain.
• stronger and healthier individual organisms may be required for further stages of research, breeding or farming.
• the identification and separation of male and female fish are required in any genetic study. Large scale sex separation can be beneficial in large scale farming because the sexes have different development times, flavor profiles and price.
• the present invention describes an imaging flow technique (i.e., one or more methods and systems) designed to analyze organisms such as insects, fish, shrimp one-by-one, and optionally- to sort them.
• the organisms may be pre-adult (for example, larvae), or adult.
• a computer implemented method for automated detection of features, structures and/or organs in organisms being insects, shrimp or fish comprising: a) providing a flow-system comprising a fluidic channel with an inlet, a detection region and at least one outlet, an electro-optical unit for monitoring the detection region, a processor; b) ensuring passage of the organisms via a fluidic channel towards the detection region, c) imaging an individual organism in the detection region by one or more sensors of the electro-optical module, thereby acquiring optical data on the organism, d) transmitting the acquired optical data to the processor, e) analyzing the obtained optical data by the processor, for detecting presence of one or more of said features, structures and/or organs of interest.
• a system for automated detection of one or more features, structures and/or organs in organisms being insects, shrimp or fish comprising: a a fluidic channel comprising an inlet, a detection region and at least one outlet, said channel is configured to allow flowing there-though of the organisms suspended in liquid media; b an electro-optical module for monitoring the detection region, c a processor in communication with the electro-optical module, wherein said electro-optical module comprises at least one sensor configured to acquire optical data by imaging an individual organism and to transmit said optical data to the processor, and the processor is configured to receive, process and analyze the optical data acquired by the electro-optical module so as to detect said structures and/or organs of interest in the organism.
• the proposed flow technique may be used both for pre-adult and adult organisms of the mentioned plurality including insects, shrimp and fish. It should be noted however that for insects and shrimp, the proposed technique is mainly applicable to pre-adult organisms.
• each individual pre- adult organism for example larva.
• differences at this stage are usually very subtle, and observable only in a number of structures or precursors of organs, they may be hard to detect even with high-end microscopes.
• the complex interior of the larva causes light to be transmitted, refracted and/or reflected in ways that are highly dependent on the orientation of the larva with respect to any illumination sources and/or image capturing units.
• this invention describes a method and a system to achieve this in a fast, reliable and reproducible manner without requiring 3D reconstruction or fluorescence labeling/ expression.
• the organisms for example, larvae
• the color of the illumination can be chosen according the level of penetration in the larvae analyzed and by how much it emphasizes the structures and/ or organs of interest.
• the recognition of the structures and/ or organs of interest may be improved according to their broad spectrum absorbance profile.
• the acquired images may be further analyzed by a machine learning or artificial intelligence algorithm for the system to alter the flow path or the condition of the larvae downstream of the detection region (e.g., separate them according to sex).
• a method for automated detection of structures and/or organs in organisms comprising the steps of: (a) illuminating the organisms in a detection region, (b) imaging the organisms in the detection region, with an extended depth of field that spans at least one-fourth of their thickness, from at least two different directions, thus acquiring optical data; (c) analyzing the optical data for the presence of one or more structures and/or organs of interest.
• extended depth of field is one of the method advantages and should be understood as follows.
• An extended depth-of-field imaging system makes it possible to clearly capture a subject without adjusting the distance between the subject and system.
• said detection region is at least a portion of a flow-channel.
• flow-channel may be used intermittently with the term "fluidic channel”.
• the detection region may include an electro-optical module for monitoring the organisms flowing in the at least a portion of the fluidic channel.
• the electro-optical module comprises at least one sensor (imaging unit).
• the channel may be transparent or partially transparent.
• the flow-channel may be manufactured as a capillary from glass or plastic.
• the organisms of interest (insects, shrimp or fish) flow through said fluidic channel one at a time. In some embodiments, said organisms in said detection region are aligned parallel to said flow-channel.
• the inner cross-section of the channel may be selected to be slightly greater than the average thickness of the organism of interest, to allow both the organism's flow through the channel, and its appropriate positioning in the channel while being imaged.
• the inner cross -section of the channel may be up to of about 120% of the organism's thickness.
• the cross-sectional shape of said flow-channel is square or rectangular.
• said flow-channel is positioned between image capturing unit(s) to image said insects, shrimp and fish, and light-source(s) for illuminating them.
• image capturing unit or “imaging unit” may be used intermittently with the term “sensor”.
• the number of said observation directions is two, and the angle between them is 90°. In various embodiments, the number of said observation directions may vary between 2 and 4. When using 2 or 4 directions, an angle of 90 degrees may be recommended between them, to contribute to most representative information. When using 3 directions, then an angle of 120 degrees between them may be recommended.
• the depth of field is at least of about 1/4 of the entire thickness of the organism, at each particular observation direction. This parameter is developed due to the fact that the organs of interest may be located close to the surface of the larvae body. In practice, the range of the depth-of-field may be between 1/4 and 1 of the organism's thickness, when the organism's thickness is close to the inner size of the flow channel. However, in case the channel is wider than the organism's body thickness, the range of the depth-of-field may be selected accordingly, for example between 1/2 and 3 or more.
• said at least two observation directions form a plane perpendicular to the flow channel's axis.
• said images (forming the optical data) are pre-processed before being analyzed.
• said images are acquired by the sensors being CCD or CMOS cameras.
• said images taken from different directions are not all simultaneous (for example, when said images are taken at of different locations along the flow-channel).
• said images taken from different directions and/or at different locations are synchronized.
• one said image capturing unit may image said organisms from different directions through a system of mirrors.
• said image capturing unit (and/or the whole electro-optical module) is rotatable around the flow-channel.
• said pre-adult insects, shrimp and fish are rotated about the main axis of the flow channel.
• said imaging is performed with an f-number greater than 4.
• the depth-of-field may be controlled by changing the f-number and other parameters of the system (for example, optical magnification, camera sensor, etc.)
• pulsed (strobed) illumination may be used, thereby reducing heat effect of the illumination and rendering requirements to the camera quality less critical.
• the strobed illumination allows using simpler sensors (cameras), e.g. those without a global shutter.
• Strobed illumination not only replaces the need for a global shutter, but also allows using powerful pulses which are effective for obtaining high quality images and harmless to the organisms.
• Non-strobed illumination of similar power would require a high-power, non-coherent light source in order to prevent interference artifacts such as speckles. This in turn translates into:
• pulsed illumination is more advantageous, especially when the organisms are heat sensitive. Further, the pulse duration is chosen so as to ensure that the images are not smeared because of the organisms flow through the channel.
• dT (N * p * PS) / (v * M) (1)
• N the size of the object of interest (in the flow direction)
• p the % of allowed smearing for the object of interest
• p n/N where n is the length of the smeared region in pixels in the flow direction)
• PS the pixel size of the imaging sensor (e.g., camera)
• v the flow velocity of the object of interest in the channel
• M is the optical magnification of the imaging system.
• the above equation allows determining parameters of the pulsed illumination in order to obtain an image of high quality with a predetermined, modest length of smearing (for example the smearing of about 1-2 pixels only) being invariant to the organism's size.
• a predetermined, modest length of smearing for example the smearing of about 1-2 pixels only
• the inventors have achieved such a quality of the image, that for example, the size of a larva on the picture may be equal to its real size + 1-2 pixels only.
• the captured images may be smeared by less than 0.25% of the full frame.
• light bursts may be used. These are short pulses having duration of less than 50 microseconds.
• relatively low power pulses may be used, if the camera is highly light sensitive. In all cases, the average pixel intensity in the ROI, with no object (organism) present, should be greater than 10% of the saturation value.
• the illumination may be performed by at least one light source which may form part of the mentioned electro-optical module.
• said illumination is white or broad spectrum. In some further embodiments, such illumination is the preferred type of illumination. In addition, short illumination pulses (for example, with a pulse duration of less than 50 microseconds) allow for cost reduction.
• said illumination may be IR or near-IR illumination.
• the color of the illumination may be chosen according to each specific application.
• a monochromatic or quasi-monochromatic illumination may be advantageous in emphasizing structures or organs of interest.
• the object of interest may have strong absorbance at a certain wavelength or spectrum (for example due to high concentration of molecules which absorb light). It is also possible to encounter the other extreme, low absorbance, which causes the object of interest to appear close to transparent.
• the object of interest or its surroundings may be fluorescent. In which case, illuminating them at the correct “excitation” wavelength causes a fluorescent emission at a different color. In case of detecting a strong signal associated with a certain color (for example, in case of selffluorescence or strong absorbance to a specific light color in sex-organs of a specific organism) illumination at such a specific light color will be preferred.
• white or broad spectrum illumination is selected together with a color camera (e.g., 3 RGB channels), since this combination provides more spectral information regarding the illuminated object (organism) compared to monochromatic or quasi-monochromatic) illumination.
• a color camera e.g., 3 RGB channels
• a plurality of pixel rows of said at least one image capturing unit (sensor) are positioned in parallel to the main axis of said flow-channel.
• only part of the image capturing sensor (actually only some rows of the plurality of pixel rows of said image capturing sensor) is used for acquiring images. Such a solution allows for faster imaging.
• said illumination is collimated.
• said images not containing the organisms of interest are used for adjusting and pre-processing said images containing said organisms.
• images may be used for background subtraction, for reference to determine whether an acquired image contains a larva etc., and for testing and aligning the system.
• a blocking structure (for example, an aperture) is associated with each said image capturing unit, such that only light from said detection region can reach it.
• said one or more organs of interest are sex related.
• the sex of said pre-adult insects, shrimp and fish can be determined.
• the presence or absence of a male or female sex organ may be determined by a machine learning (ML) algorithm.
• the ML algorithm needs high quality pictures. For example, the Inventors may use pictures where the relevant organ in Aedes albopictus L3 larvae is -1200 pixels in size. It is still visible at lower resolution (even ⁇ 75pixels), yet the ML code has a harder time detecting it at the same level of precision.
• the above equation (1) may be used.
• the use of pulsed white or broad spectrum illumination from two or more directions in combination with the imaging having an extended depth of field imaging, allows receiving high quality images suitable even for detecting features/organs of pre-adult organisms, and for further sorting of such organisms.
• the sorting may be focused, for example, on separation of strong organisms from weak ones, alive ones from dead ones, fast developing ones from retarded ones etc.
• the sorting may include sex separation of the organisms of interest.
• a controller coupled to the flow-system can alter the flow path or the condition of the organisms such as shrimp, fish or pre-adult insects.
• each one can be directed to the appropriate outlet (e.g., males only, females only, uncertain). This may be done using valves which control the flow path.
• the appropriate outlet e.g., males only, females only, uncertain.
• the technique may comprise choosing to alter the condition of certain larvae (e.g., to destroy them, sterilize them by irradiation) by using the same flow system, and after the sex has been determined.
• a computer-implemented method for automated detection of features, structures and/or organs in organisms being shrimp, fish or pre-adult insects, comprising steps of:
• a suitable flow-system for automated detection of said features, structures and/or organs in organisms being shrimp, fish or pre-adult insects comprising:
• an electro-optical module for monitoring the organisms in the detection region, including: an illumination unit configured for illuminating the organisms in the detection region, an imaging unit comprising one or more sensors configured for acquiring optical data by capturing images of an individual organism in the detection region from at least two different observation directions and with an extended depth of field that spans at least one-fourth of the organism's thickness,
• a processor in communication with the electro-optical module, configured for receiving the optical data from said module, processing and analyzing thereof for the presence of one or more said structures and/or organs of interest in the organism.
• a computer implemented method of sorting organisms being shrimp, fish or pre-adult insects, comprising: a) providing a flow system comprising a fluidic channel with an inlet, a detection region and at least one outlet, an electro-optical unit for monitoring the detection region, a processor and a controller; b) ensuring passage of the organisms suspended in liquid media through the inlet into the fluidic channel towards the detection region; c) acquiring optical data of the individual organism by imaging thereof at the detection region by one or more sensors of the electro-optical module; d) transmitting the optical data of the individual organism to the processor; e) processing the optical data and performing classification of the individual organism based on one or more morphologic features and/or color- related features and/or sex related features (for example, a set of parameters indicative of the sex), extracted from the optical data; f) providing instructions to the controller based on the classification of the individual organism, and g) sorting the organism according to the instructions received by the controller.
• a flow system comprising a
• the sorting of the organisms based on the morphologic or color-related features may be used in order to select strong, and healthy individual organisms.
• some specific, sex-related features/parameters may be used for sex separation of the organisms.
• a suitable flow-system for sorting of organisms being shrimp, fish or pre-adult insects comprising: a fluidic channel comprising an inlet, a detection region and at least one outlet, said channel is configured to allow flowing there-through of the organisms suspended in liquid media; a processor; an electro-optical module for monitoring the detection region, being in communication with the processor, a controller in communication with said processor; wherein the electro-optical module comprises at least one sensor configured to acquire optical data of an individual organism by imaging thereof, and to transmit said optical data to the processor, the processor is configured to receive and process the acquired optical data, to perform classification of said organism based on one or more morphologic features and/or sex-related features (for example, a set of parameters indicative of the sex), and to instruct the controller to sort said organism based on the classification.
• a fluidic channel comprising an inlet, a detection region and at least one outlet, said channel is configured to allow flowing there-through of the organisms suspended in liquid media
• a processor an electro-optical module for
• the present invention provides a computer implemented algorithm comprising code to perform the steps of the method as defined in any of the above-described methods.
• Computer readable non-transitory storage medium is also provided, storing computer-implementable instructions of said code.
• the present invention is a novel, rapid and cost effective technique of analyzing and possible sex-sorting of the mentioned organisms, such as pre-adult insects and shrimp, and fish in any stage (whether in the larva or fry or fingerling or adult developmental stages).
• the process as inter-alia disclosed is automated and preferably does not require user intervention.
• insects refers to fish, shrimp and insects.
• the insects of interest are mainly in the pre-adult shape, to allow their survival in the proposed flow system.
• the shrimp should be understood as a decapoda group comprising various shrimp, prawns and crawfish.
• fish as used herein generally refers to gill-bearing aquatic craniate animals.
• fluidic channel refers to a device, apparatus or a container having an inner space and outer walls, wherein the inner space is contained with and/or allow the passage of a flowable fluid or liquid media or solution, for example water or isotonic aqueous solution.
• a fluidic channel has at least one inlet and at least one outlet and in the context of the present invention is adapted to contain organisms of interest, suspended in a liquid media passing or flowing from the inlet portion of the channel towards the outlet potion of the channel while subjected to automatic/computed analyzing and/or classification process.
• movement in the context of the invention is meant to be understood as voluntary or un-voluntary movement of the organism suspended in liquid media, as it passes through the channel.
• streaming refers to flowing or rapidly flowing of a liquid or fluid or suspension, e.g., a velocity or volume flow in a channel.
• the term “streaming” covers both the case of passive flow, and the case of active flow of liquid media in the fluidic channel.
• liquid media refers, without limitation, to a substance in a physical state in which it does not resist change of shape but does resist change of size.
• passive flow refers to flow control methods which require no auxiliary power.
• the non-limiting list of passive flow control methods includes any passive devices which are steady and require no energy by definition (e.g., derived by gravity forces), such as turbulators, roughness elements or any other passive device influencing the fluid dynamics of the flow.
• active flow refers to flow control methods which require energy expenditure.
• the non-limiting list of active flow control methods includes valves, pumps, pressurizing means, applying a positive or negative (vacuum) pressure difference with respect to parts of the system of the present invention or any other artificially driven actuator which require energy.
• the system and method for sex-separation or sorting of organisms comprises means to drive them out of a container containing unsorted organisms and through the system that may be connected to such a container towards the outlet of such a system.
• These include vacuum, valves, gravity forces and pumps.
• These means may further include a controllable guiding tool configured to guide the organisms to an outlet associated with the acquired classification. Examples of such a guiding tool may comprise one or more valves, a transducer configured to generate acoustic radiation forces, electrodes configured to apply electroosmotic forces to the liquid media, electrodes configured to apply dielectrophoretic forces to the organism and any combination thereof.
• controller refers, without limitation, to any hardware device or a software program, or a combination of the two that manages or directs the flow of data between two or more entities. In a general sense, a controller can be thought of as something or someone that interfaces between two systems and manages communications between them.
• the system of the present invention may comprise a control center, wherein said control center is configured to monitor the system performance.
• the system e.g., using the detection/classifi cation region, processor and controller parts is configured to classify an individual organism into classes selected from a non-exhaustive list comprising male, female, and/or other by a set of parameters indicative of the sex of the organism (e.g., based on optical and/or other data of the organism).
• the above-mentioned list may comprise other classes , defined by other attributes (e.g., dead or alive; healthy or unhealthy, for example based on color and color transmission ) or by probability of future attributes (e.g., a class of estimated mature size, for example based on previous statistics data and the current size).
• other attributes e.g., dead or alive; healthy or unhealthy, for example based on color and color transmission
• probability of future attributes e.g., a class of estimated mature size, for example based on previous statistics data and the current size.
• the features/parameters/attributes for the analyzing and/or sorting ( classification) include one or more features from the following three groups defined in a non-limiting manner: a first non-exhaustive group of morphologic features/parameters/attributes, comprising: overall size of the organism, morphology of the organism, shape, segment size ratio, a second non-exhaustive group of color-related features comprising: absorption, transmission, IR (Infrared) absorption, IR transmission, color, fluorescence, a third non- exhaustive group of mainly sex-related features/parameters/attributes, comprising: gonad disc morphology, secondary sex organ morphology, gonad size, gonad morphology, gonad autofluorescence, size of testes, size of male
• Stroboscopic illumination system refers to a system capable of emitting short pulses of light (1 microsecond-10 milliseconds).
• the pulses of lights can be generated by: light emitting diodes (LEDs), lasers, solid state light sources and Xenon lamps. This system minimizes illumination time, thus reducing generated heat, phototoxicity and photobleaching artifacts as well as blurring due to motion.
• High Frame Rate Camera refers to a camera capable of capturing images with exposures of less than 1/1,000 second or frame rates in excess of 30 frames per second. It might be useful for recording fast-moving objects as photographic images onto a storage medium.
• system performance parameters refers to key system capabilities that must be met in order for a system to meet its operational goals.
• a nonlimiting list of system performance parameters of the invention includes: flow rate, sorting rate, sorting accuracy, organism count, specificity, sensitivity, survival, fitness and throughput.
• TP the number of cases correctly identified as positive for the tested parameter.
• Sensitivity The sensitivity of a test is its ability to determine the cases positive for the tested parameter correctly. To estimate it, the proportion of true positive in cases evaluated positive for the tested parameter should be calculated. Mathematically, this can be stated as:
• TN/(TN+FP) (Number of true negative assessment)/(Number of all negative assessment)
• Morphology is a branch of biology dealing with the study of the form and structure of organisms and their specific structural features. This includes aspects of the outward appearance (shape, structure, color, pattern, size), i.e. external morphology (or eidonomy), as well as the form and structure of the internal parts like bones and organs, i.e. internal morphology (or anatomy). Morphology is a branch of life science dealing with the study of gross structure of an organism or taxon and its component parts.
• Electro-optical module refers to a module or unit or region comprising devices (e.g., lasers, LEDs, waveguides etc.) and/or systems and/or sensors which operate by the propagation and interaction of light (electromagnetic or optical) and the electrical (electronic) state.
• Electro-optical sensor is an electronic detectors that convert light, or a change in light, into an electronic signal. These sensors are able to detect electromagnetic radiation from the infrared up to the ultraviolet wavelengths.
• An optical sensor converts light rays into electronic signals. Without wishing to be bound by theory, the electro-optic effect is a change in the optical properties due to interaction with light.
• the electro-optical module of the detection region comprises at least one of: one or more light sources configured to illuminate the classification region, one or more acoustic sources configured to generate sound waves that pass through the classification region, one or more image sensors, at least one optical element and an internal control unit in-communication with the processor.
• the electro-optical module comprises more than one light source.
• each light source is configured to emit light with predetermined spectrum and/or intensities. More particularly, illumination may be broadband (for example, white light) or monochromatic.
• sensor generally refers to a device, module, machine, or system capable of detecting or measuring a property or changes in its environment and sending the information or data (e.g., optical or image or acoustic data) to other electronics, frequently a computer processor.
• information or data e.g., optical or image or acoustic data
• Non limiting examples of sensor types within the scope of the present invention include, but are not limited to acoustic, sound and/or vibration sensors, electric current, electric potential, magnetic and/or radio sensors, flow and/or fluid velocity sensors, optical, light, imaging and/or photon sensors, pressure sensors, thermal, heat and/or temperature sensors and position/location sensors.
• optical sensor or “optical element” as used herein is meant to include photoconductive devices which convert a change of incident light into a change of resistance, photodiodes which convert an amount of incident light into an output current, phototransistors are a type of bipolar transistor where the base-collector junction is exposed to light.
• Optical sensors within the scope of the present invention may include: optical detector, a camera, a photodiode, a photomultiplier, an image acquisition sensor, an optical acquisition sensor, an electro-optical sensor, light detector, a photon sensor, a reflectometer, a photodetector, a spectral image sensor, and any combination thereof.
• This term further encompasses an optical element such as a lens, a mirror, a polarizer, an excitation filter, an emission filter, a dichroic mirror, an optical coating such as antireflective coating, an optical grating, at least one stereoscopic microscope, at least one fluorescence microscope and any combination thereof.
• an optical element such as a lens, a mirror, a polarizer, an excitation filter, an emission filter, a dichroic mirror, an optical coating such as antireflective coating, an optical grating, at least one stereoscopic microscope, at least one fluorescence microscope and any combination thereof.
• imaging sensor or “image sensor” or “image acquisition sensor” as used herein refers to a sensor that detects and conveys information used to make an image. Without wishing to be bound by theory, an imaging sensor conveys the variable attenuation of light waves, passed through or reflected off objects, into signals, that convey the information.
• the waves can be light or other electromagnetic radiation.
• Image sensors are used in electronic imaging devices of both analog and digital types, which include digital cameras, camera modules, camera phones, and others.
• imaging sensors within the scope of the present invention include: RGB (red, green, blue) frequency spectrum, broad spectrum, hyperspectral, visible light frequency range, near infrared (NIR) frequency range, infrared (IR) frequency range, monochrome, specific light wavelengths (e.g., LED or laser and/or laser and/or halogen and/or xenon and/or fluorescent), UV frequency range, a reflectometer and combinations of the aforementioned.
• optical data generally refers to data retrieved from optical media which can be stored on an optically readable medium. Examples of optical storage devices, include CD, DVD and Blu-ray discs etc. In other aspects it refers to an optically formed reproduction of an object, such as one formed by a lens or mirror.
• optical data within the context of the present invention encompasses image data.
• image data herein means a photographic or trace objects that represent the underlying pixel data of an area of an image element, which is created, collected and stored using image constructor devices.
• Image data attributes include for example, image resolution, data-point size and spectral bands.
• computer stands for but no limited to a machine or device that performs processes, calculations and operations based on instructions provided by a software or hardware program.
• the term "computer” also means in the context of the present invention a control unit or controller. It is designed to process and execute applications and provides a variety of solutions by combining integrated hardware and software components.
• the computer of the invention is configured to extract a predetermined set of feature vectors from the image data of the organism; to compute sex-characteristics of the organism based on the set of feature vectors, attributes or parameters; to generate sex-classification output and to transmit the output to the controller unit.
• the present invention may be a system, a method, and/or a computer program product.
• the computer program product may include a computer readable storage medium (or media) having computer readable program code/ instructions thereon for causing a processor to carry out aspects of the present invention.
• the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
• the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
• a non- exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, and any suitable combination of the foregoing.
• RAM random access memory
• ROM read-only memory
• EPROM or Flash memory erasable programmable read-only memory
• SRAM static random access memory
• CD-ROM compact disc read-only memory
• DVD digital versatile disk
• memory stick a floppy disk, and any suitable combination of the foregoing.
• a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
• Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
• a network for example, the Internet, a local area network, a wide area network and/or a wireless network.
• These computer readable program instructions may be provided to a processor of a general-purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
• These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
• the method of the present invention comprises steps of applying a machine learning process with the computer implemented trained algorithm to detect organs and/or to determine the sex of an organism.
• the algorithm or computer readable program
• the algorithm is implemented with a machine learning process using a neural network with the processed data.
• training in the context of machine learning implemented within the system of the present invention refers to the process of creating a machine learning algorithm.
• a source of training data can be used to train machine learning models for a variety of use cases, from failure detection to consumer intelligence.
• the neural network may compute a classification category, and/or the embedding, and/or perform clustering, for identifying sex of an individual organism i.e. male or female.
• the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks herein disclosed.
• classifying may sometimes be interchanged with the term clustering or tagging, for example, when multiple organism's images are analyzed, each image may be classified according to its predefined features or feature vectors and used to creating clusters, and/or the images may be embedded and the embeddings may be clustered.
• classification category may sometimes be interchanged with the term embedding, for example, the output of the trained neural network in response to an image of an organism may be one or more classification categories, or a vector storing a computed embedding. It is noted that the classification category and the embedding may be outputted by the same trained neural network, for example, the classification category is outputted by the last layer of the neural network, and the embedding is outputted by a hidden embedding layer of the neural network.
• the architecture of the neural network(s) may be implemented, for example, as convolutional, pooling, nonlinearity, locally- connected, fully-connected layers, and/or combinations of the aforementioned.
• the tagging and classifying of the organism in the images or the sex-sorting characteristic targets may be manually or semi manually entered by a user (e.g., via a graphical user interface, for example, selected from a list of available phenotypic characteristic targets), obtained as predefined values stored in a data storage device, and/or automatically computed.
• feature vector refers hereinafter in the context of machine learning to an individual measurable property or characteristic or parameter or attribute of a phenomenon being observed e.g., detected by a sensor. It is herein apparent that choosing an informative, discriminating and independent feature is a crucial step for effective algorithms in pattern recognition, machine learning, classification and regression. Algorithms using classification from a feature vector include nearest neighbor classification, neural networks, and statistical techniques.
• a feature is an information which is relevant for solving the computational task related to a certain application.
• Features may be specific structures in the image such as points, edges or objects.
• Features may also be the result of a general neighborhood operation or feature detection applied to the image.
• features are defined in terms of local neighborhood operations applied to an image, a procedure commonly referred to as feature extraction is executed.
• the present invention provides a device, system and method for sex-sorting of pre-adult insects as well as of shrimp and fish in any stage (whether in the larva or fry or fingerling or adult developmental stages).
• the system comprises a fluidic channel comprising an inlet and a number of outlets; a classification region and a separation or destruction region.
• the channel is configured to receive organisms suspended in a liquid.
• the organisms pass through a detection (classification) region where they are imaged and classified according to their sex by a processing unit.
• the organisms are separated into different outlets, where they can then be collected.
• the organisms are classified according to their sex with one class passing undamaged to the single outlet for collection, and the rest being destroyed.
• a liquid typically water, containing an organism enters the system, through the inlet and flows, under pressure or gravity, towards the outlet(s).
• the inner dimensions of the flow channel may be designed such that only a single organism passes at a time. Furthermore, these dimensions may be set so as to limit motion of the organism (e.g., a narrow enough channel constricts it to extend parallel to the channel walls).
• the temperature of the liquid may be changed or controlled, namely heated (elevated) or cooled (reduced) according to the need.
• One such use may be to control or affect movements of the organism during analyzing, sorting or separating process (i.e. cooled to reduce wriggling movements and heated to increase wriggling movements).
• the temperature of the liquid may be cooled so as to reduce wriggling movements.
• As the organisms pass through the classification region they are imaged. The pictures/images are analyzed by a processing unit and the organism may be then classified.
• a separation region is used where each individual is guided to the outlet associated with its class. This may be done by leaving only the appropriate outlet open and closing the rest. Alternatively, all outlets may be left open and one or more of the following techniques may be used to guide each individual to the appropriate one: (1) acoustic radiation forces generated by an external transducer; (2) electroosmotic forces generated by electrodes (forces applied on the liquid); (3) dielectrophoretic forces generated by electrodes (forces applied on the organisms) or (4) a controlled liquid flow from one more additional directions toward a selected outlet of the fluidic channel. Such flows may be created at a so-called separation region where the channel is divided into two or more outlets.
• the outlets themselves may also be designed to have different hydrodynamic resistance. Thus, when no external force is applied, the organisms flow to the outlet with the lowest resistance.
• the number of individuals dispensed into each outlet channel(s) is monitored. This is particularly important when one channel is connected at its end to a container to be used for release.
• a destruction region through which all individuals pass, is used.
• the organisms identified as belonging to the class that is needed for the application e.g., males or females, according to the purpose/application
• pass unharmed e.g., males or females, according to the purpose/application
• the rest also herein referred to as 'other' or 'unspecified' or 'unclassified'
• the number of individuals dispensed unharmed is monitored.
• an electro- optical setup (module) next to the detection region may be used to recognize the sex of the organism, for example fish. It includes at least one of: one or more light sources to illuminate the detection region (possibly from different directions and in different colors); one or more image sensors, to photograph the detection region (possibly from different directions); an optical setup, comprising one or more optical elements, e.g., lenses, mirrors, polarizers, excitation filters, emission filters, dichroic mirrors, optical coatings (e.g., antireflective coatings) and optical gratings, located between the light source(s) and the detection region and/or the classification region and the image sensor(s); and a connection to a processing unit.
• the optical setup is comprised of one or more stereoscopic microscopes and/or fluorescence microscopes and the image sensors are cameras attached to the microscopes.
• the organisms may be classified at least into two classes, wherein said two classes being selected from at least the following pairs: male and non-male, female and nonfemale, male and female, alive and dead, healthy and sick, strong and weak, quickly developing or retarded.
• two classes being selected from at least the following pairs: male and non-male, female and nonfemale, male and female, alive and dead, healthy and sick, strong and weak, quickly developing or retarded.
• three classes may exist (male, female and undetermined).
• pictures taken by the image sensors are analyzed in the processing unit to determine the class of an individual organism.
• One or more visual attributes/parameters/features from at least one of the following first second and third non-exhaustive groups may be used for analyzing the organisms and also for sorting the organisms according to a desired classification.
• first and second groups of morphologic and color-related features may be applied, comprising: overall size of the organism, morphology of the organism, shape, segment size ratio, absorption, transmission, IR (Infrared) absorption, IR transmission, color, fluorescence.
• the third group of sex-related features/attributes/parameters may be applied, comprising: gonad disc morphology, secondary sex organ morphology, gonad size, gonad morphology, gonad autofluorescence, size of testes, size of male accessory glands, morphology of testes, morphology of male accessory glands, autofluorescence of testes, autofluorescence of male accessory glands, size of the developing male and female primary or secondary reproductive structures, primitive sex organs, morphology of the developing male and female primary or secondary reproductive structures, autofluorescence of the developing male and female primary or secondary reproductive structures and any combination thereof.
• the third group of attributes/features/parameters may be applied to the sex-sorting.
• differences between male and female organisms may be sorted by one or more attributes (parameters indicative of the sex) including, but are not limited to, the overall size of the organism, morphology of the organism, shape, segment size ratio, absorption, transmission, IR (Infrared) absorption, IR transmission, color, fluorescence, gonad disc morphology, secondary sex organ morphology, gonad size, gonad morphology, gonad autofluorescence, size of testes, size of male accessory glands, morphology of testes, morphology of male accessory glands, autofluorescence of testes, autofluorescence of male accessory glands, size of the developing male and female primary or secondary reproductive structures, primitive sex organs, morphology of the developing male and female primary or secondary reproductive structures, autofluorescence of the developing male and female primary or secondary reproductive structures and any combination thereof.
• attributes including, but are not limited to, the overall size of the organism, morphology of the organism, shape, segment
• classification may be carried out automatically using one or more of the following techniques: image processing, machine learning and trained neural network.
• the neural network may be a classical neural network or a deep network (deep learning network), including a convolutional neural network, or any other kind of neural network.
• the flow velocity of each organism may be monitored in order to determine when they reach the separation region and need to be guided to the appropriate outlet or reach the destruction region and need to pass freely or be destroyed.
• this is done by taking successive images in the detection (classification) region.
• a flow-meter or an additional camera with a wide field of view are used for tracking the organisms along the system.
• Fig- 1 illustrates one example of the proposed flow-system for determining structures or organs of interest and/or for classifying organisms such as pre-adult insects, shrimp or fish.
• FIGs 2A-2C schematically illustrate non-limiting examples of a flow system with integrated detection/classification region and sorting region (Figs 2A-B) and optionally destruction region (Fig. 2C), according to some embodiments of the present invention
• Fig- 3 is an illustrative flow chart of one version of the proposed technology, describing an exemplary method and an exemplary flow system for classifying a specific type of the organisms of interest and sorting them accordingly;
• Fig. 4 illustrates a non-limiting example of an imaging unit of an electro-optical module according to the present invention
• FIGS. 5A-5B schematically illustrate two exemplary embodiments of the imaging units arranged along the flow system
• Fig. 6 schematically illustrates an exemplary embodiment of the optical arrangement in the flow system, where a single camera can image more than one direction with the use of mirrors;
• Fig. 7 schematically illustrates an exemplary embodiment of the optical arrangement in the flow system, where a camera can image more than one direction by rotating it about the detection region.
• Fig. 8 schematically illustrates the proposed way of imaging, where the depth of field is at least 1/4 of the thickness of the organism of interest.
• Embodiments of the proposed technique will be illustrated and commented by referring to a non-limiting example of detecting sex organs in pre-adult organisms, and on non-limiting example of further sex separation of the organisms.
• the invention provides, in some of its aspects, a novel fluidic channel comprising an inlet and outlet(s); a detection/classifi cation region or unit or zone or module. Optionally, it also provides a separation region or unit or zone or module and/or a destruction region or unit or zone or module.
• the channel is configured to receive preadult insects or shrimp or fish in any stage (whether in the larva or fry or fingerling or adult developmental stages), suspended in a liquid.
• the pre-adult insects may be in the shape of a larva or a pupa or a nymph; the shrimp may be in the shape of a larva or a post-larva or a juvenile; and the fish may be in the shape of a larva or a fry or a fingerling or an adult.
• the organisms pass through the detection/classification region where they are imaged, analyzed and optionally classified according to their sex. According to this classification, in some embodiments, the organisms are separated into different outlets.
• Fig- 1 shows a schematic block diagram of one embodiment of the present invention, intended for analyzing organisms like pre-adult insects, shrimp and fish in a flow system 10.
• the figure shows a fluidic channel 24 through which individual organisms 26 flow from left to right. Their flow may be passive or active.
• the channel 24, or part of it, may be manufactured of glass.
• the organisms may be introduced in the fluidic channel from a container 22 which is shown in dashed lines).
• the monitoring may be provided by an electro-optical module which is schematically marked with the same number 28. For monitoring, the organisms in the channel 24 are usually illuminated.
• the electro- optical module then performs imaging of an individual organism using one or more optical sensors, and transmits the obtained optical data to a processor 32.
• the processor analyzes the received optical data (the images) and detects, for example, presence or absence of structures and/or organs of interest on the imaged organism. Additional steps may be taken according to results of the detection process (for example, steps of sex classification and separation) while the imaged organism continues flowing though the channel. Such steps will be illustrated and described in the following figures.
• Figs 2A-C show schematic illustrations of exemplary flow-systems for sorting organisms like pre-adult insects, shrimp and fish according to the present invention.
• the processing unit/processor 32 Upon the imaging of individual organisms within the detection region 28 (which may also be called a classification region), the processing unit/processor 32 performs analysis and classification thereof while the organisms continue flowing until reaching a separation region 30. In the separation region 30, the individual organisms are guided to different channels (for example, by a number of controlled flows) according to the classification performed by the processor 32.
• the processor 32 may be in control communication with a controller and/or suitable devices (not shown) of the flow system. Classification attributes exist for any organism of interest.
• such attributes may include, but are not limited to the overall size of the organism, morphology of the organism, shape, segment size ratio, absorption, transmission, IR (Infrared) absorption, IR transmission, color, fluorescence, gonad disc morphology, secondary sex organ morphology, gonad size, gonad morphology, gonad autofluorescence, size of testes, size of male accessory glands, morphology of testes, morphology of male accessory glands, autofluorescence of testes, autofluorescence of male accessory glands, size of the developing male and female primary or secondary reproductive structures, primitive sex organs, morphology of the developing male and female primary or secondary reproductive structures, autofluorescence of the developing male and female primary or secondary reproductive structures and any combination thereof.
• Fig. 2A schematically illustrates a flow system 20, where the individual organisms are guided to either a 'male-only' or ‘female-only’ channel 34 or an 'anything else' channel 36.
• Fig. 2B shows a schematic illustration, where the individual organisms may be guided to either a 'male-only' channel 34, a 'female-only' channel 38 or to an 'anything else' channel 36.
• Fig. 2C shows a schematic illustration, where after the classification region only a single class of organisms continue flowing, undamaged, through the main channel. The rest are destroyed in a destruction region 40.
• Fig- 3 illustrates a flow chart schematically showing one version of the process according to the present invention.
• a method for male-female sorting in a flow system may begin with the provision of an unsorted organism (for example, fish) as shown in step 50.
• the individual organism flows preferably in a single-file towards the detection region (which may also serve as a classification region), as shown in step 52.
• the detection region which may also serve as a classification region
• each individual fish is imaged for detection and/or classification purposes as shown in step 54 of the flow chart.
• each individual is classified. Classes used may include males and the rest (other, unspecified, unclassified or anything else), or males, females and the rest (other, unspecified, unclassified or anything else) or females and the rest (other, unspecified, unclassified or anything else).
• the individual organisms reach a separation region, where they are guided, according to their classification, to a predefined flow outlet (step 58). Alternatively, if only a single class is needed, the unnecessary individuals may be destroyed in the channel (step 60). The outcome of the above described process is sex- sorted organisms (step 62).
• the organisms of interest are transparent or partially transparent (like larvae or the like), though they may be non-transparent (like some adult fish).
• FIG.4 A first exemplary embodiment of automated detection of structures in the mentioned organisms (say, pre-adult) in a flow channel by imaging is shown in Fig.4.
• Fig. 4 is a schematic cross-section of the flow system at its detection region.
• the detection region includes a portion of the fluidic channel 102 (24) surrounded by components of an electro-optical module schematically marked 28.1.
• the organisms for example in the form of larvae, flow one at a time through a transparent square capillary (102, 24), where they are illuminated, from two different directions, by light sources (104, 106) [all shown from top-view].
• Images of the organisms are captured by image capturing units (108, 110), such as CCD or CMOS sensors, after being focused by a single lens or an array of lenses (112, 114).
• the angle between their observation directions is preferably 90° as this is the angle with the greatest information gain.
• observation takes place from three or more directions.
• the acquired images are transmitted to a processing unit (116, 32) where they may be preprocessed before being analyzed by an image recognition algorithm.
• the processing unit 116 may also be in communication with the flow-system control module (30, see Fig.2A). As mentioned, it can alter, based on the algorithm’s results, the flow path or the condition of the organisms downstream of the detection region (e.g., separate the larvae into different containers according to their sex, destroy some classified organisms, sterilize some classified organisms by irradiation, etc.)-The color of the illumination is chosen according to the level of penetration in the organism analyzed and the level of highlighting of the structures and/ or organs of interest.
• a multichannel image capturing unit e.g., RGB sensor
• RGB sensor multichannel image capturing unit
• the recognition of the structures and/ or organs of interest may be improved according to their broad spectrum absorbance profile.
• using Infra-red (IR) or near-IR illumination may be beneficial when visible light is strongly attenuated as it passes through the organism (for example when the organisms are non-transparent or only partially transparent).
• the IR or near-IR illumination could be used as the sole illumination source(s) or alongside visible illumination as well.
• a blocking structure for example an aperture associated with the image capturing units ensures that only light (depicted by dashed line) from the flow-channel can reach them. Furthermore, the image capturing units and light sources are placed one opposite the other, with the capillary in between. Since the image capturing units 108, 110 (cameras, sensors) are imaging the moving (flowing) larvae, steps should be taken to make sure that the resulting images are sharp and not smeared. One approach would be to use a camera with a fast, global shutter ( ⁇ 50uSec) and a light source constantly turned on.
• the illumination pulses may be synchronized to take place in the periods when the image sensor performs the imaging (and not when the image sensor saves the pictures to memory).
• Smearing of the obtained images may be minimized down to 1 -2 pixels by selecting a suitable duration of the illumination pulses, for example using an equation (1). In percent, the smearing of the obtained images may be less than 0.25% of the full frame.
• the illumination pulse duration may be less than 50 microseconds.
• Fig. 5 shows two different embodiments for the automated detection of structures in the organisms of interest in a flow-channel [both in isometric-view]. The detection is performed using measurements of the electro-optical module indicated 28.2.
• the different image capturing units (two are shown) image the organism at the same location along the capillary (120).
• Fig. 5B the different image capturing units (two are shown), belonging to the electro-optical unit 28/3, image the organism at different locations along the capillary (122, 124).
• the images of one and the same object taken from different locations along the flow channel may be not all simultaneous.
• the images are taken while one and the same light source is used, and the light source provides pulsed illumination, the images will be naturally synchronized by this light source.
• the two image capturing units image one and the same organism at locations 122 and 124.
• the imaging system typically works in video mode, such that all passing objects are recorded. If two objects pass one after the other, the images associated with each one will be analyzed separately. It is also possible the two objects are so close to one another that it is not feasible to separate them downstream and so they may both be discarded.
• the organisms are conveyed one by one with a space therebetween.
• the safe space can be measured in units of time. In the proposed embodiment, it is ⁇ 0.5sec or more between successive larvae. Unsorted larvae may be fed into the flow system through an entrance container, like in Fig. 1.
• the larvae in the container may be gently mixed, to overcome their self-movement and prevent clustering, through bubbling of air.
• the air may also be used to pressurize the container and drive the larvae through its outlet and into the flow system.
• a sensor e.g., photodiode. This triggers the system to switch a valve, such that (1) no more larvae (or water) can flow out of the entrance container, and (2) water flows into the system from a second, different container which contains only water. (No containers and valves are shown in this drawing).
• the water flow from the second container drives the larva that entered the system through the flow system and passes the detection region.
• imaging of the pre-adult insects, shrimp and fish in the flowchannel can be done from N>1 different directions, while using M image capturing units, with M ⁇ N.
• Fig.6 depicts one such embodiment 28.4 of the electro-optical module, where a single image capturing unit (108) images the pre-adult insects, or shrimp or fish in any stage from two different directions [shown from top-view].
• the first direction faces the sensor of the image capturing unit
• the second direction is at an angle to it and the light is guided to the sensor by a setup of mirrors (126, 128) and a beam splitter or dichroic mirror (130).
• the optical path from the organism to the sensor will not necessarily be the same along the two directions. This may be corrected with an optical element (e.g., retarder) placed along the shorter path (132) to ensure that the images associated with the different directions are all in focus.
• an optical element e.g., retarder
• each image will be formed on a different region of the sensor (e.g., by adjusting the angle of the beam splitter or dichroic mirror (130).
• the different illumination sources associated with different imaging directions captured on the same image capturing sensor may be toggled on and off such that for any given image, only one illumination source will be switched on.
• Fig.7 depicts yet another embodiment where the number of imaging directions is bigger than the number of image capturing units [shown from side-view]. Theoretically, the number of imaging directions may be more than four.
• the flow-channel is stationary, whereas the electro-optical unit 28.5 with the illumination and imaging elements, is rotatable about the flow channel.
• a single image capturing unit together with its focusing optics, blocking element and illumination source are all attached to a rotating stage (134).
• the rotation itself may either be continuous or back-and-forth.
• the former option is less strenuous on the motor, but requires the use of a slip-ring or the like if any wires are attached to the rotating system (e.g., electronic, data) to prevent their twisting. This problem is also circumvented if the latter option of back-and-forth rotation is used.
• Fig- 8 illustrates how the imaging units may operate according to one option of the proposed imaging technology.
• the figure shows a portion of the electro-optical module at the detection region.
• a transparent organism say, larva 136) flowing via a fluidic channel 102 (24) is illuminated by while light from light source 104.
• the imaging unit or sensor 108 (say, an RGB camera) images the larva using the depth-of-field D.
• Dash lines 140 indicate the borders of D, within which sex organs of the larva are expected to be found.
• the organism 136 is imaged from different directions. Here, for simplicity, only one direction is shown using camera 108, with the extended depth of field D (138). In this example, D spans one-fourth of the organism thickness and is marked as the region between the dashed lines (140).
• the present invention provides a system for sex separation of organisms such as insects fish and shrimp, comprising: (a) a fluidic channel having an inner space and an outer wall, the fluidic channel comprising an inlet and at least one outlet, wherein the channel is configured to allow flow of the organisms suspended in liquid media; (b) a processor; and, (c) a controller in- communication with the processor; wherein the fluidic channel comprises a classification region, and the classification region comprises an electro-optical module in-communication with the processor, the electro-optical module comprises at least one sensor configured to acquire optical data of an individual organism and to transmit the optical data to the processor, further wherein the processor is configured to process the acquired optical data, to classify the organism based on the acquired optical data, and to instruct the controller to sort the organism based on the classification of the organism.
• theinner space of the fluidic channel is adjustable to the dimensions of a single organism.
• thefluidic channel is designed to allow passage of a single organism at a time.
• system is further configured to control movement of the organism when passing through the fluidic channel.
• theflow of the organism suspended in the liquid media through the fluidic channel is a passive flow.
• the flow of the fish suspended in the liquid media through the fluidic channel is an active flow.
• the flow is driven by a pump, vacuum, pressurizing means such as pressurized gas, gravity forcesor any combination thereof.
• system comprises more than one outlet, wherein each outlet is independently operated by the controller.
• eachof the one or more outlets is associated with a classification category selected from the group consisting of male, female and other.
• the at least one outlet having an “open” and a “closed” configuration.
• theoutlet is operably coupled to a container.
• the fluidic channel further comprises a separation region, wherein the separation region is located between the detection region and the at least one outlet; and wherein the separation regioncomprises at least one guiding tool configured to guide the organism to an outlet associated with the classification.
• the guiding tool is selected from the group consisting of one or more valves, a transducer configured to generate acoustic radiation forces, electrodes configured to apply forces to the liquid media, electrodes configured to apply dielectrophoretic forces to the organism, controlled, liquid flow from one more additional directions and any combination thereof.
• the fluidic channel further comprises a destruction region; wherein the destruction region comprises at least one destroying tool in-communication with the controller; and wherein the destroying region has an “on” and “off’ configurations operated by the controller.
• the at least one destroying tool is selected from the group consisting of a laser beam, a high-powerelectric field, ultrasonic blasts, a grinding tool, a squashing tool and any combination thereof.
• the controller is configured to execute functions selected from the group consisting of: receiving instructions from the processor, controlling flow rate, switching on the guiding tool, switching off the guiding tool, switching on the destroying tool, switching off the destroying tool, switching configuration of the outlet from “open” to “close”, and, switching configuration of the outlet from “closed” to “open”.
• the senor is selected from the group consisting of an optical detector, a camera, a photodiode, a photomultiplier, an image acquisition sensor, an optical acquisition sensor, an electro-optical sensor, light detector, a photon sensor, a reflectometer, a photodetector, a spectral image sensor, and any combination thereof.
• the senor is an imaging sensor selected from the group of consisting of RGB frequency spectrum, broad spectrum, hyperspectral, visible light frequency range, near infrared (NIR) frequency range, infrared (IR) frequency range, monochrome, specific light wavelengths (e.g., LED and/or laser and/or halogen and/or xenon and/or fluorescent), UV frequency range and any combination thereof.
• RGB frequency spectrum broad spectrum
• hyperspectral visible light frequency range
• NIR near infrared
• IR infrared
• monochrome monochrome
• specific light wavelengths e.g., LED and/or laser and/or halogen and/or xenon and/or fluorescent
• UV frequency range any combination thereof.
• the electro-optical module of the classification region comprises at least one of: one or more light sources configured to illuminate the classification region, one or more image sensors, one or more acoustic sources configured to generate sound waves that pass through the classification region, at least one optical element and an internal control unit in-communication with the processor.
• the electro-optical module comprises more than one light source, wherein each light source is configured to emit light with predetermined spectrum and/or intensity.
• the electro-optical module is configured to obtain multiple images of the organism from different angles.
• the image acquisition sensor is a high-frame rate.
• the light source illumination is stroboscopic or pulsed.
• the optical element is selected from the group consisting of a lens, a mirror, a polarizer, an excitation filter, an emission filter, a dichroic mirror, an optical coating such as antireflective coating, an optical grating and any combination thereof.
• the optical element comprises at least one stereoscopic microscope, at least one fluorescence microscope, or a combination thereof.
• the senor is attached to the at least one microscope.
• the system of the present invention further comprises a sensor in-communication with the controller, wherein the controller is further configured to control the number of individual organisms at the entrance to the fluidic channel based on the data acquired by the sensor, to thereby allow passage of single organism at a time to the fluidic channel.
• the processor is configured to perform functions selected from receiving the optical data of an individual organism acquired by the sensor of the electro-optical module, processing the optical data, extracting a set of parameters indicative of the sex of the organism from the optical data, classifying the organism based on the set of parameters extracted from the optical data, providing instructions to the controller based on the classification of the organism.
• the processor is configured to classify the individual organism into at least two classes selected from the group consisting of male and non-male.
• the processor is configured to classify the individual fish into at least two classes selected from thegroup consisting of female and non-female.
• the processor is configured to classify the individual fish into three classes selected from the groupconsisting of male, female, and other.
• the features of the organism including the features (parameters) indicative of the sex of the organism are selected from at least one of a first, a second and a third group respectively comprising the following: the first group: overall size of the organism, morphology of the organism, shape, segment size ratio, the second group of color-related features comprising: absorption, transmission, IR (Infrared) absorption, IR transmission, color, fluorescence, the third group: gonad disc morphology, secondary sex organ morphology, gonad size, gonad morphology, gonad autofluorescence, size of testes, size of male accessory glands, morphology of testes, morphology of male accessory glands, autofluorescence of testes, autofluorescence of male accessory glands, size of the developing male and female primary or secondary reproductive structures, primitive sex organs, morphology of the developing male and female primary or secondary reproductive structures, autofluorescence of
• thedetection/classification region further comprises a glass capillary through which the organism flows.
• the system of the present invention further comprises a thermostat optionally operated by the controller, wherein the thermostat isconfigured to control the temperature in the fluidic channel.
• theliquid media is water or isotonic aqueous solution.
• the sorting rate is about 0.1 to 100 organisms per second.
• the inner space of the fluidic channel has the size in the range of 0.1 mm to 10 mm.
• the sorting accuracy is in the range of 50% to 100%, particularly 80% to 100%, more parti cularlyat least 90%.
• the sorting specificity is in the range of 50% to 100%, particularly 80% to 100%, more particularly at least 90%.
• the sorting selectivity is in the range of 50% to 100%, particularly 80% to 100%, more particularly at least 90%.
• the organism is a pre-adult insect or shrimp or fish larva or fish fry or fish fingerling or adult fish.
• the system of the present invention further comprises a container in fluid connection with the fluidic channel, the container contains unsorted organisms suspended in a liquid media, the container has an outlet configured to allow passage of a single organism at a time to the fluidic channel, the container further comprises pressurizing means, such as pressurized gas, vacuum, gravity forces and/or one or more pumps to drive a flow of the unsorted organisms towards the outlet of the container to the fluidic channel.
• pressurizing means such as pressurized gas, vacuum, gravity forces and/or one or more pumps to drive a flow of the unsorted organisms towards the outlet of the container to the fluidic channel.
• the container in fluid connection with fluidic channel for sex separation of organisms, contains unsorted organism suspended in a liquid media, the container has an outlet configured to allow passage of a single organism at a time to the fluidic channel, the container further comprises pressurizing means, such as pressurized gas, vacuum, gravity forces and/or one or more pumps to drive a flow of unsorted organisms towards the outlet of the container to the fluidic channel.
• pressurizing means such as pressurized gas, vacuum, gravity forces and/or one or more pumps to drive a flow of unsorted organisms towards the outlet of the container to the fluidic channel.
• a computer implemented method of sex separation of organisms such as pre-adult insects, fish or shrimp, the method comprising: (a) providing the system for sex separation of organism as defined in any of the above; (b) streaming the organism suspended in the liquid media through the inlet into the fluidic channel towards the classification region; (c) acquiring optical data of the individual organism at the classification region by one or more sensors of the electro-optical module; (d) transmitting the optical data of the individual organism to the processor; (e) processing the optical data and classifying the individual organism based on a set of parameters indicative of the sex of the organism extracted from the optical data; (f) providing instructions to the controller based on the classification of the individual organism; and, (g) sorting the organism according to the instructions received by the controller.
• step of sorting of the organisms according to the instructions received by the controller comprises at least one of: controlling flow rate, controlling sorting rate, switching on the guiding tool, switching off the guiding tool, switching on the destroying tool, switching off the destroying tool, switching the configuration of the outlet from “open” to “closed”, and switching the configuration of the outlet from “closed” to “open”.
• any of the above comprising steps of acquiring optical data by a sensor selected from the group consisting of an optical detector, a camera, a photodiode, a photomultiplier, an image acquisition sensor, an electro-optical sensor, an optical acquisition sensor, a light detector, a photon sensor, a reflectometer, a photodetector, a spectral image sensor, and any combinationthereof.
• a sensor selected from the group consisting of an optical detector, a camera, a photodiode, a photomultiplier, an image acquisition sensor, an electro-optical sensor, an optical acquisition sensor, a light detector, a photon sensor, a reflectometer, a photodetector, a spectral image sensor, and any combinationthereof.
• the electro- optical module of the classification region comprises at least one of: one or more light sources configured to illuminate the classification region, one or more acoustic sources configured to generate sound waves that pass through the classification region, one or more image sensors, at least one optical element and an internal control unit in-communication with the processor.
• the classifying comprises using a trained neural network.
• step of processing comprises steps of analyzing the optical data using computer implemented algorithm trained to generate output based on the optical data.
• the method comprises steps of implementing with the algorithm a training process according to a training dataset comprising a plurality of training images of a plurality of organisms captured by the at least one imaging sensor, wherein each respective training image of the plurality of training images is associated with the sexdetermination of the organism depicted in the respective training image.
• the training process comprises steps of (a) capturing images of organism using an imaging sensor; (b) classifying images into classification categories by applying a tag associated with parameters or attributes indicative of the sex of the organism extractedfrom the optical data; and (c) applying a computer vision algorithm to determine a set of feature vectors associated with each classification category.
• the machine learning process comprises computing by theat least one neural network, a tag of at least one classification category for the at least one organism, wherein the tag of at least one classification category is computed at least according to weights of the at least one neural network, wherein the at least one neural network is trained accordingto a training dataset comprising a plurality of training images of a plurality of organisms captured by the at least one imaging sensor, wherein each respective training image of the plurality of training images is associated with the tag of at least one classification category of at least one organism depicted in the respective training image; and generating according to the tag of at least one classification category, instructions for execution by the controller.
• the senor is configured for image capturing and processing, with or without using Artificial Intelligence (Al) and/or machine learning and/or neural networks.
• Al Artificial Intelligence
• the computer implemented algorithm may be a machine learning algorithm.
• the machine learning algorithm may use verified training data.
• a software product comprising computer-implementable instructions and data stored on a non-transitory computer readable storage medium and designed to cause performing steps of the method as described in any of the described embodiments.
• EXAMPLE 1 A system for sex-sorting of fish
• An exemplified sex-sorting system essentially is based on the following: fish larvae or fries or fingerlings or adults are kept in their natural environment, namely, water.
• the inner dimensions of the flow channel were chosen such that only a single fish can pass at a time. Furthermore, these dimensions constrict the fish to extend parallel to the channel walls.
• the temperature of the liquid may also be cooled so as to reduce wriggling movements. As the fish pass through a classification region, they are imaged.
• Aqueous medium used to flow the fish typically water
• the unsorted fish may be kept dispersed in the liquid before entering the sortingsystem through agitation. This may help fish enter the system one by one and at a constant rate.
• a sensor at the entrance to the system may be used to detect if two or more fish have entered it at close proximity. If this occurs, the larvae are diverted outside to prevent possible mistakes.
• Pressurized gas may be used to mix the fish and drive them through the system.
• Imaging with a color camera enables to improve differentiation by using thedifferent absorption characteristics of the different organs.
• Imaging from two or more distinct angles in order to improve detection of objects of interest (e.g., gonads should be visible from one of the angles). This may be done using two or more cameras or by using a single camera and directing the light from different regions on different parts of the imaging sensor.
• High frame rate >30 frames per second
• stroboscopic illumination may beused in order to image the flowing larvae without having to stop them before capturing the image.
• the fish may flow through a glass capillary to improve image quality.
• a square or rectangular capillary may be used, for a more uniform background and to remove spherical aberrations.
• Both male and female primary and secondary primitive (not fully developed) sex organs can be used.
• the system receives a container with water and unsorted larvae
• the larvae pass one-by-one into a flow system
• the flowing larvae are imaged, After imaging, the system switches the larvae to appropriate outlets (male and female containers) according to a predesigned computer algorithm,
• the larvae are not harmed by the process.
• the viability of an organism is determined by its light absorption pattern, i.e., its general opacity and the level of uniformity in light absorption. In most cases, but not exclusively, live organisms are less opaque (less absorptive) and less uniform in their absorption profile than dead ones.
• the growth potential of an organism is determined by a combination of its overall size and shape (e.g., width to length ratio) and/or number of specific structures like head and mouth parts.
• the health of an organism is determined by its color under a broad spectrum illumination.
• the observed color and color ratios of the organism are partly the result of light refraction characteristics of its exterior and/ or morphological signals such as the presence or absence of structures caused by disease or the organism’s response to a pathogen, or a combination thereof.
• the determining factor may be the existence or shape of parts of the reproductive system. For example, the existence of secondary male glands in the ninth (anal) segment of mosquito larvae correlates with male outcome.

Landscapes

• Life Sciences & Earth Sciences (AREA)
• Environmental Sciences (AREA)
• Zoology (AREA)
• Animal Husbandry (AREA)
• Biodiversity & Conservation Biology (AREA)
• Marine Sciences & Fisheries (AREA)
• Animal Behavior & Ethology (AREA)
• Farming Of Fish And Shellfish (AREA)
• Investigating Or Analysing Biological Materials (AREA)
• Investigating Or Analysing Materials By Optical Means (AREA)
• Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)

Priority Applications (3)

Applications Claiming Priority (4)

Related Child Applications (1)

Publications (1)

Family

ID=82159132

Family Applications (1)

Country Status (3)

Cited By (3)

Citations (4)

Family Cites Families (3)

• 2021
  • 2021-12-27 IL IL303323A patent/IL303323A/en unknown
  • 2021-12-27 WO PCT/IL2021/051536 patent/WO2022137243A1/en unknown
  • 2021-12-27 EP EP21909721.9A patent/EP4247153A4/de active Pending

Patent Citations (4)

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events