JP2015516077A - Machine vision system for frozen aliquoters for biological samples - Google Patents

Abstract

A machine vision system for use with a system for taking a frozen sample core from a sample in a container includes a camera. The processor is adapted to receive image data from the camera and determine a location where the frozen sample core has already been taken. A method for determining one or more locations where a frozen sample core has already been taken from a frozen sample is for receiving a container on a platform while the frozen sample core is withdrawn from a frozen sample contained in the container. Manipulating the robotic system to place one of the containers in the mechanism. The camera is used to take an image of a frozen sample. The contrast of the captured image is evaluated to identify one or more hole candidates. The processor uses the image to determine if the hole candidate is a real hole or artifact. [Selection] Figure 6

Description

  The present invention relates generally to machine vision systems and methods, and more particularly to controlling a robotic system for collecting multiple frozen sample cores from a frozen sample in a container without thawing the frozen sample. It relates to a machine vision system that facilitates.

  Biological samples are generally stored to support a wide variety of biomedical and biological studies, including but not limited to bridging studies, molecular medicine, and biomarker exploration. Biological samples include any sample derived from animals (including humans), plants, protozoa, fungi, bacteria, viruses, or other living organisms. For example, biological samples include, but are not limited to, organisms and plasma, serum, urine, whole blood, umbilical cord blood, other blood-based derivatives, cerebrospinal fluid, mucus (from the respiratory tract, cervical canal), Ascites, saliva, amniotic fluid, semen, tears, sweat, and / or biological fluids separated or secreted from organisms such as plant-derived fluids (including sap) and / or cells (eg, buffy coat cells) Animals, plants, protists, fungi, or bacterial cells), cell lysates, homogenates or suspensions, microsomes, cell organs (eg mitochondria), chromosomal DNA, mitochondrial DNA And nucleic acids (eg, RNA, DNA) and small molecule compounds (eg, in DMSO) in suspension or solution, including plasmids (eg, seed plasmids) A molecular compound), it includes the other fluid-based biological samples. Biological samples can also include plants, plant parts (eg, seeds) and tissues (eg, muscle, fat, skin, etc.).

  Biobanks generally store these valuable samples in containers (eg, well plates or arrays, test tubes, vials, etc.) and cryopreserve them. Test tubes, vials, and similar containers can be arranged in arrays and stored in well plates, racks, partitioned containers, and the like. Some samples are stored at relatively high temperatures (eg, -20 degrees Celsius), while other samples are stored at much lower temperatures. For example, some samples may be stored in a freezer at -80 degrees Celsius or less using liquid nitrogen or a gas phase above liquid nitrogen, and the biochemistry of the frozen sample as close as possible to in vivo conditions. Preserve composition and integrity to facilitate accurate and reproducible sample analysis.

  Often, it may be desirable to perform one or more tests on a frozen sample. For example, a researcher may want to test a set of samples with certain characteristics. A particular sample may contain enough material to support many different tests. In order to conserve resources, a smaller amount of sample, commonly known as an aliquot, from a larger cryopreserved sample (sometimes referred to as the parent sample) for use in one or more tests Is removed, making the remainder of the parent sample available for one or more different subsequent tests.

  Biobank has adopted several different methods to address the need to provide this sample aliquot. One option is to freeze the sample in large quantities, thaw it when an aliquot is sought, and then refreeze all the rest of the parent sample for storage in the cryopreservation until future aliquots are needed. That is. This option allows efficient use of the frozen storage space, but this efficiency still comes at the expense of sample quality. Exposure of the sample to repeated freeze / thaw cycles can degrade the sample's important biological molecules (eg, RNA) and damage the biomarker, any of which was obtained from the damaged sample The results of any study using the data can be compromised.

  Another option is to freeze the sample in large amounts, thaw it when an aliquot is sought, and divide the remainder of the parent sample into small portions to create additional aliquots for later testing, then These smaller aliquots are refrozen and each aliquot is cryopreserved separately until needed for further testing. This approach limits the number of freeze / thaw cycles to which the sample is exposed, but the larger frozen storage space volume, effort, and more sample containers (e.g., required to maintain cryopreservation aliquots) , Test tubes, vials, etc.) increases the costs associated with inventory. Furthermore, aliquots can be degraded or damaged even if the number of freeze / thaw cycles is limited.

  Yet another approach is to divide large samples into smaller aliquots before first freezing them. While this approach can limit the number of freeze-thaw cycles that a sample can undergo to only one, there are still disadvantages associated with the effort associated with this approach, the cryopreservation space, and the cost of sample container inventory required. .

  US Patent Publication No. 20099001987 discloses a system for extracting a frozen sample core from a frozen biological sample without thawing the original (parent) sample, the contents of which are hereby incorporated by reference. doing. This system uses a drill containing a hollow coring bit to take a frozen core sample from the original parent sample without thawing the parent sample. The frozen sample core obtained by this drill is used as an aliquot for the test. After the frozen core is removed, the rest of the sample is returned to its frozen container in its original container until further aliquots are needed from the parent sample in later tests.

US Patent Application Publication No. 2009/0019877

  The inventors have developed the systems and methods described below. This facilitates automatic recognition of whether the frozen sample contains one or more frozen sample core holes from previous extractions, as well as automatic recognition of the location of all such holes, Perform automatic withdrawal of additional frozen sample cores from the sample.

  One aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a platform for supporting one or more of the containers. The platform has a mechanism for receiving at least one of the containers and a set of calibration marks on the platform in a fixed position relative to the mechanism. The system has a camera for taking an image of the container while the container is received by the mechanism. The processor is configured to receive image data representing an image of the container from the camera. The processor (a) evaluates the contrast in the image to identify one or more hole candidates, and (b) uses information regarding the position of the calibration mark with respect to the hole candidates, One or two in which a frozen sample core has already been taken from a frozen sample contained in a container by determining whether two or more candidates are likely to be artifacts rather than true holes in the sample It is configured to determine one or more positions.

  Another aspect of the present invention is a method for collecting a frozen sample core from a frozen sample contained in a container. The method includes placing the container in a mechanism for receiving the container on the platform. The platform has a set of calibration marks in a fixed position on the platform relative to this mechanism. While the container is received by the mechanism, an image of the container is taken. One or more locations where a frozen sample core has already been taken from a frozen sample contained in a container (a) evaluate the contrast in the image to identify one or more hole candidates And (b) using information about the position of the calibration mark relative to the hole candidate, to determine whether one or more candidates are likely to be artifacts rather than real holes in the frozen sample Is configured to determine. The frozen sample core is taken from the sample at a position where the frozen sample core has not yet been taken, as determined in the determining step.

  Yet another aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a platform and a camera to take an image of one of the containers while it is on the platform. The processor is configured to receive image data representing an image captured by the camera from the camera. The processor (a) evaluates the contrast in the image to identify one or more hole candidates, and (b) one or more hole candidates are not present in the real hole of the sample. Rather, it is configured to determine one or more locations where the frozen sample core has already been taken from the frozen sample contained in the container by determining whether it is likely to be an artifact. The processor is formed as follows: (i) the size of the hole candidate, (ii) the distance between the hole candidate and the center of gravity axis of the container, and (iii) the first line and the second line. An angle, wherein a first line extends between the hole and the centroid axis of the container, and a second line extends between the centroid axis of the container and another hole candidate, The angle formed between the second line and the second line, (iv) the relationship between the position of one or more candidate holes and the expected pattern of holes in the sample, (v) The location of one or more hole candidates relative to the periphery of the container, (vi) the number of hole candidates identified, (vii) the amount of contrast between the hole candidates and the area surrounding the hole candidates, and ( viii) Using at least one of these combinations, one or more hole candidates are likely to be artifacts Configured to determine whether.

  Another aspect of the present invention is a method for collecting a frozen sample core from a frozen sample contained in a container. The method includes the step of taking an image of the container. The captured image (a) evaluates the contrast in the image to identify one or more hole candidates, and (b) one or more hole candidates are Used to determine one or more locations where a frozen sample core has already been taken from a frozen sample contained in a container by determining whether it is likely an artifact rather than a hole in It is done. The following information: (i) the size of the hole candidate, (ii) the distance between the hole candidate and the center of gravity axis of the container, (iii) the angle formed between the first line and the second line The first line extends between the hole and the container's center of gravity axis, and the second line extends between the container's center of gravity axis and another hole candidate, The angle formed between the line and the second line, (iv) the relationship between the position of one or more candidate holes and the expected pattern of holes in the frozen sample, (v) The location of one or more hole candidates relative to the periphery of the container, (vi) the number of hole candidates identified, (vii) the amount of contrast between the hole candidates and the surrounding area, and (viii) these Using at least one of the combinations, one or more hole candidates can be artifacts rather than real holes It is determined whether or not the high. The frozen sample core is taken from the sample at a position where the frozen sample core has not yet been taken, as determined in the determining step.

  Another aspect of the present invention is a calibration system configured to calibrate a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The calibration system includes a platform for supporting the container. The platform is disposed with one or more fixed targets thereon. The camera captures an image of one or more containers while the container is supported by the platform, and to capture an image of one or more fixed targets placed on the platform. Is mounted on the robot system. The processor is configured to receive image data representing an image formed by the camera from the camera. The processor is configured to calibrate the robotic system using images of one or more fixed targets on the platform.

  Another aspect of the present invention is a method for calibrating a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The method includes one or more immobilizations on the platform while the container is supported by the platform of the robotic system that determines whether one or more frozen sample cores have already been taken from the frozen sample. Using a camera to capture an image of one or more containers to capture an image of the target. The image of one or more targets is used to calibrate the robot system.

  Another aspect of the present invention is a machine vision system for use with a robotic system adapted to collect a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a camera for taking an image of the container while the container is supported by the platform. The camera has an optical axis. The system has a ring light for illuminating the container on the platform. The ring light includes a plurality of light sources arranged in an annular pattern. The optical axis of the camera extends through the central portion of the annular pattern. The processor receives image data representing an image taken by the camera from the camera, and evaluates contrast in the image, thereby allowing one or two frozen sample cores to be taken from the sample contained in the container. Adapted to determine more than one position.

  Yet another aspect of the present invention is a method for determining one or more locations where a frozen sample core has already been taken from a frozen sample, each of which is contained in a respective container. The method includes moving the camera relative to the first one of the containers and manipulating the robotic system such that the camera is directed at the frozen sample in the first container. The frozen sample is illuminated using a ring light. The ring light has a plurality of light sources arranged in an annular pattern. The camera has an optical axis that extends through the central portion of the annular pattern. The camera is used to take an image of the illuminated frozen sample. The contrast of the captured image is evaluated, the image is processed to identify one or more candidate holes in the captured image, and the candidate hole is likely to be an artifact or a true hole in the frozen sample Determine whether or not. The robotic system is operated to move the camera relative to the second of the containers so that the camera is directed at the frozen sample in the second container. Imaging is repeated for the frozen sample in the second container.

  Yet another aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a container. The system includes a camera configured to take a monochromatic image of the container while the container is supported by the platform. The illumination is arranged to illuminate the container and the sample contained therein while the container is supported by the platform. The processor is adapted to receive a grayscale image data representing an image formed by the camera from the camera and determine a position where the frozen sample core has already been taken from the sample by evaluating the contrast in the image. The illumination emits light having a color other than white.

  Another aspect of the invention is a method of determining one or more locations from a frozen sample where a frozen sample core has already been taken. Each frozen sample is contained in a respective container. The method includes moving the camera relative to the first one of the containers and manipulating the robotic system such that the camera is directed at the frozen sample in the first container. The frozen sample is illuminated with colored light. The camera is used to take a grayscale image of the illuminated frozen sample. The contrast of the captured image is evaluated, the image is processed to identify one or more candidate holes in the captured image, and the candidate hole is likely to be an artifact or a true hole in the frozen sample Determine whether or not. The robotic system is operated to move the camera relative to the second of the containers so that the camera is directed at the frozen sample in the second container. Imaging is repeated for the frozen sample in the second container.

  Another aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a container. The system includes a camera for taking an image of the container while the container is supported by the platform. The illumination is arranged to illuminate the container and the sample contained therein while the container is supported by the platform. The illumination has a red light emitting element, a blue light emitting element, and a green light emitting element. The intensity of the light emitted by the red, blue, and green light emitting elements can be selectively adjusted so that any of a plurality of different light colors can be selected as the color of the light emitted by the illumination. The processor is adapted to receive image data representing an image formed by the camera from the camera and determine a position where the frozen sample core has already been taken from the sample by evaluating the contrast in the image. The processor receives input relating to the color of the sample in the container and is adapted to adjust the color of the light emitted by the illumination to reduce the difference between the color of the sample and the light emitted by the illumination.

  Another aspect of the invention is a method of determining one or more locations from a frozen sample where a frozen sample core has already been taken. Each frozen sample is contained in a respective container. The method includes moving the camera relative to the first one of the containers and manipulating the robotic system such that the camera is directed at the frozen sample in the first container. The frozen sample is illuminated with colored light. The color of the light is selected to match the color of the frozen sample. The camera is used to take an image of the illuminated frozen sample. The contrast of the captured image is evaluated, the image is processed to identify one or more candidate holes in the captured image, and the candidate hole is likely to be an artifact or a true hole in the frozen sample Determine whether or not. The robotic system is operated to move the camera relative to the second of the containers so that the camera is directed at the frozen sample in the second container. The imaging process is repeated for the frozen sample in the second container.

  Another aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a container. The machine vision system includes a platform for supporting the container. The platform has a mechanism for receiving one of the containers while the frozen sample core is withdrawn from the frozen sample contained in the container. The system has a camera on the platform for taking an image of the container while the container is received by the mechanism. The illumination is arranged to illuminate the container from a location that provides at least one of backlighting and sidelighting.

  Yet another embodiment of the invention is a method of determining one or more locations from a frozen sample where a frozen sample core has already been taken. Each frozen sample is contained in a respective container. The method includes operating a robotic system to place one of the containers in a mechanism for receiving the container on the platform while the frozen sample core is withdrawn from the frozen sample contained in the container. Including. The illumination is used to provide at least one of backlighting and side lighting for the container. The camera is used to take an image of the frozen sample while the light is illuminated. The contrast of the captured image is evaluated and the image is processed to identify one or more hole candidates in the captured image.

  Another aspect of the present invention is a machine vision system for use with a robotic system adapted to collect a plurality of frozen sample cores, each from a frozen sample contained in a container. A machine vision system is a camera for taking an image of a container while a container is supported in a mechanism for receiving the container by a platform while the frozen sample core is withdrawn from the frozen sample contained in the container. including. The system has red light to illuminate the container from above with a substantially monochromatic red light while it is in a mechanism on the platform. The processor receives image data representing an image taken by the camera from the camera, and evaluates contrast in the image, thereby allowing one or two frozen sample cores to be taken from the sample contained in the container. Adapted to determine more than one position.

  Yet another aspect of the invention is a method for determining one or more locations from a frozen sample where a frozen sample core has already been taken. Each frozen sample is contained in a respective container. The method includes operating a robotic system to place one of the containers in a mechanism for receiving the container on the platform while the frozen sample core is withdrawn from the frozen sample contained in the container. Including. The container is illuminated from above with a substantially monochromatic red light while it is in a mechanism on the platform. The camera is used to take an image of the frozen sample while illuminated with red light. The contrast of the captured image is evaluated and the image is processed to identify one or more hole candidates in the captured image.

  Another aspect of the invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a platform for supporting one or more of the containers. The platform has a mechanism for receiving at least one of the containers. The system has a camera for taking an image of the container while the container is received by the mechanism. The processor is configured to receive image data representing an image of the container from the camera. The processor evaluates the contrast in the image to identify one or more candidate holes and to identify the edge of the container, and using information about the position relative to the candidate holes, one or two One or more of which the frozen sample core has already been taken from the frozen sample contained in the container by determining whether these candidates are likely to be artifacts rather than true holes in the sample Is configured to determine the position of.

  Another aspect of the invention is a method of determining one or more locations from a frozen sample where a frozen sample core has already been taken. Each frozen sample is contained in a respective container. The method includes operating a robotic system to place one of the containers in a mechanism for receiving the container on the platform while the frozen sample core is withdrawn from the frozen sample contained in the container. Including. The camera is used to take an image of a frozen sample. The contrast of the captured image is evaluated to identify one or more hole candidates and to identify the edge of the container. Information about the position of the edge relative to the hole candidates is used to determine whether one or more hole candidates are likely to be artifacts rather than real holes in the sample.

  One aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a platform for supporting one or more of the containers. The platform has a mechanism for receiving at least one of the containers. The system includes a camera for taking an image of the container while the container is received by the mechanism. The system includes a fill level detection system adapted to detect the position of the surface of the frozen sample. The processor is configured to receive a signal from the fill level detection system and use the signal to determine where to position the camera to obtain an image of the frozen sample.

  Another aspect of the invention is a method for determining one or more locations where a frozen sample core has already been taken from a frozen sample, each of which is contained in a respective container. The method includes using a fill level detection system to measure the position of the surface of the frozen sample spaced from the bottom of the container. Information from the fill level detection system is used to determine where to position the camera so that the camera has a predetermined position relative to the surface of the sample and the camera moves to that position. An image of the frozen sample in the container is taken from that position. The image is used to identify the location of one or more holes in the sample.

  Yet another aspect of the present invention is a machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container. The machine vision system includes a platform for supporting one or more of the containers. The platform has a mechanism for receiving at least one of the containers. The system includes a coring probe for collecting a frozen sample core from a frozen sample. The system includes a camera for taking an image of the container while the container is received by the mechanism. The processor is configured to receive image data representing an image of the container from the camera and to determine from the frozen sample contained in the container one or more positions where the frozen sample core has already been taken. The The processor is configured to move the coring probe into the open end of the at least one hole to remove debris from the open end of the hole.

  Another aspect of the present invention is a method for collecting a frozen sample core from a frozen sample contained in a container. The method includes placing the container in a mechanism for receiving the container on the platform. While the container is received by the mechanism, an image of the container is taken. One or more locations where the frozen sample core has already been collected from the frozen sample contained in the container are determined. The frozen sample core is taken from the sample at a position where the frozen sample core has not yet been taken, as determined in the determining step. After taking the frozen sample core from the frozen sample, the coring probe is inserted into one or more locations where the frozen sample core was taken, and one or more where the frozen sample core was taken. Remove the strip from the position of.

  Yet another aspect of the present invention is a machine vision system for use with a robotic system adapted to collect a plurality of frozen sample cores, each from a frozen sample contained in a container. The machine vision system includes a camera for taking an image of the container while the container is supported by the platform. The system includes illumination for illuminating the container on the platform. Most of the energy of the light emitted by the illumination is selected from the group consisting of red light with a wavelength in the range from 620 nm to 750 nm and green light with a wavelength in the range from 495 nm to 570 nm. The processor receives one or more image data representing an image taken by the camera, and evaluates the image to evaluate one or more frozen sample cores from a sample contained in the container. Adapted to determine position.

  Another aspect of the invention is a method for determining one or more locations where a frozen sample core has already been taken from a frozen sample, each of which is contained in a respective container. The method includes moving the camera relative to the first one of the containers and manipulating the robotic system such that the camera is directed at the frozen sample in the first container. The frozen sample is illuminated with light, and most of the light energy emitted by the illumination is selected from the group consisting of red light with a wavelength in the range of 620 nm to 750 nm and green light with a wavelength in the range of 495 nm to 570 nm. Is done. The camera is used to take an image of the illuminated frozen sample. The image is used to identify one or more hole candidates in the captured image and determine whether the hole candidate is likely to be an artifact or a true hole in the frozen sample. The robotic system is operated to move the camera relative to the second of the containers so that the camera is directed at the frozen sample in the second container. Imaging is repeated for the frozen sample in the second container.

  Other objectives and features will be clarified and pointed out below.

  Corresponding reference characters indicate corresponding parts throughout the drawings.

1 is a perspective view of an example of a frozen aliquoter that includes one embodiment of a machine vision system of the present invention. It is a top view of a frozen aliquoter. FIG. 3 is a top view of a frozen aliquoter with parts removed so as not to obscure the view of one embodiment of the platform. FIG. 5 is an enlarged perspective view of the platform taken along the plane including line 4-4 of FIG. FIG. 5 is a perspective view of a portion of a frozen aliquoter shown in cross section taken along the plane including line 5-5 in FIG. 1 is a perspective view of one embodiment of a robot end effector for use with a frozen aliquoter. FIG. It is a bottom view of the robot end effector of FIG. It is the schematic which shows the one part component of a freezing aliquoter. It is the schematic which shows the hole candidate from which a dimension differs. FIG. 6 is a schematic diagram illustrating one embodiment of a geometric pattern in which a frozen sample core is extracted from a frozen sample. It is the schematic which shows the hole candidate arrange | positioned at different distances from the center of a container. It is the schematic which shows the hole candidate arrange | positioned from the center of a container at various different angles mutually. FIG. 6 is a schematic diagram showing expected and not expected hole candidates planned for withdrawal of a frozen sample core from a frozen sample. FIG. 6 is a photograph of a container illustrating the use of an edge detection algorithm to identify the position of the container edge from image data. FIG. 6 is a schematic diagram illustrating one embodiment of using fixed calibration marks to identify the location of features in image data. 4 is a photograph showing a set of calibration marks in a fixed position relative to a container. 3 is a photograph of one embodiment of a density step target. FIG. 3 is a schematic view of a coring probe placed over a hole in a frozen sample. FIG. 19 is a schematic view of the coring probe of FIG. 18 inserted into the hole.

  Referring now to the drawings, and in particular to FIGS. 1-3, one embodiment of a robotic system for collecting a frozen sample core from a frozen sample contained in a container is generally designated 101. The system 101 includes a platform 103 for supporting a plurality of containers 105 and a robot end effector 111 movable relative to the platform by a motor drive system 113 controlled by a processor 114 (FIG. 8). In the illustrated embodiment, the robot system 101 is a Cartesian gantry type robot, but this is not essential, and other types of robot systems may be used within the scope of the present invention. Further details regarding robotic systems for taking frozen sample cores from frozen samples can be found in US Published Patent Application No. 20099001987, PCT International Application No. PCT / US2011 / 61214 filed November 17, 2011, and 2012. No. 13 / 359,301, filed Jan. 26, the contents of which are hereby incorporated by reference.

  In the illustrated embodiment, the platform 103 includes a recess 115 sized and shaped to receive one or more movable trays 117 for holding the containers 105. For example, one or more of the trays 117a are preferably source trays holding a plurality of source containers 105, each of the source containers containing a frozen sample core to be collected. The one or more trays 117b are preferably destination trays that hold a plurality of destination containers, each of the destination containers being taken from one or more containers on the source tray. For holding a frozen sample core.

  As shown in FIGS. 3 and 4, platform 103 is also a feed adapted to receive one of the source containers while the frozen sample core is withdrawn from the frozen sample contained therein. An original container mechanism 107 and a destination container mechanism 109 adapted to receive an empty container in which one or more frozen sample cores are placed. As shown in FIG. 4, the source container mechanism 107 includes a receptacle 106 for receiving the container 105 and a set of clamping jaws 108, 110 on either side of the top of the receptacle. Optionally, the container 105 is held in place in the mechanism 107 and held in place during extraction of the frozen sample, or the container is released and removed from the mechanism and then returned to the tray 117. In order to achieve this, at least one of the jaws 108 can be selectively brought into and out of contact with the other jaw 110, such as by a pneumatic actuator (not shown). If desired, a similar jaw can be used to hold the container 105 with the mechanism 109 for receiving the sample.

  The illustrated system 101 is adapted for use with frozen samples stored in individual containers 105. However, it is understood that the system can be adapted for use with a well plate or well array in which multiple different frozen samples are stored in a single container. For example, suitable components can be provided (e.g., on an end effector) to move a well plate or well array rather than individual containers, and mechanisms 107, 109 for receiving containers include: It can be adapted to receive a well plate or well array without departing from the scope of the present invention. Similarly, the clamping system may be adapted to hold a well plate or well array within the scope of the present invention.

  A cleaning mechanism 119 for cleaning the sample coring probe 121 used to extract the frozen sample core from the frozen sample is also on the platform. Details regarding the structure and operation of a suitable cleaning mechanism are provided in PCT International Application No. PCT / US2011 / 61214 filed November 17, 2011, and need not be repeated here.

  A cooling system 131 for keeping the frozen sample and the frozen sample core extracted therefrom in a frozen state is located below the platform 103 in the illustrated embodiment. However, the cooling system can be located elsewhere or other cooling systems can be used, or both, without departing from the scope of the present invention. 5-7, the end effector 111 of the robotic system 101 moves the sample coring probe 121 and the sample coring probe into one of the frozen samples in one of the containers 105. And then a sample core extraction system 123 operable to withdraw the coring probe from the frozen sample and obtain the frozen sample core from the frozen sample. In the illustrated embodiment, for example, the sample core extraction system 123 rotates the sample coring probe 123 when the robot drive system 113 lowers the sample coring probe into the container and then pulls it out of the container. Including a motor 125 adapted to do so. For further details regarding the operation of the coring probe to extract a frozen sample core from a frozen sample, see US Published Patent Application No. 20099001987, PCT International Application No. PCT / US2011 / 61214 filed November 17, 2011, and It is described in US patent application Ser. No. 13 / 359,301 filed Jan. 26, 2012 and need not be repeated herein. The frozen sample core can be manipulated to be extracted from the frozen sample, and as a result, within the scope of the present invention, as long as there is limited or no thawing of the frozen sample and the frozen sample core extracted therefrom It will be appreciated that other sample coring probes and sample extraction systems may be used.

  The end effector 111 also moves the container back and forth between the tray 117 and the mechanism 107, 109 on the platform for the container from which the frozen sample core is collected and the container in which the frozen sample core is placed. A gripping system 127 operable to selectively hold and release the container 105 for use by the robotic system 101. Those skilled in the art are familiar with the various commercially available gripping systems that can be used. In the illustrated embodiment, for example, the gripping system includes a plurality of movable fingers 129 that can be moved by one or more pneumatic actuators (not shown) under the control of the processor 114. It will be appreciated that other gripping systems may be used within the scope of the present invention. For example, the gripping system can be modified to move a well plate or well array containing multiple frozen samples, if desired, to facilitate use of the gripping system.

  As schematically shown in FIG. 8, the robotic system 101 is configured to automatically recognize a position where a frozen sample core has already been taken from a frozen sample in a container 105 (if any). In conjunction with facilitates the collection of additional frozen sample cores from already cored frozen samples. At these locations, there are holes or holes in the frozen sample. In some cases, the holes may be empty, but in other cases, the holes are frost crystals, or debris that have grown on the sample (eg, while the sample is in a frozen storage location), or other reasons Or a combination of these may be included or blocked by this. The machine vision system 141 is also configured to recognize that there are no holes in the frozen sample from which any frozen sample core has not been drawn. After the machine vision system 141 has been sampled earlier to obtain an aliquot, a further frozen sample core from that sample is required and frozen for a period of time before a further aliquot is provided for later testing. Facilitates the use of the robotic system 101 to collect frozen sample cores from frozen samples returned to storage.

  The machine vision system 141 includes a camera 143 attached to take images of the container 105 and the frozen sample contained therein while the container is supported by the platform 103. In the illustrated embodiment, the machine vision system 141 includes a display 146 that is coupled to the processor 114 to display captured and / or processed image data. The camera 143 is preferably mounted on the robot system 101 for movement relative to the container 105 by the robot system. For example, in the illustrated embodiment, the camera 143 is mounted on the end effector 111 for movement with the end effector. It will be appreciated that the camera may be mounted in a fixed position relative to the platform within the scope of the present invention.

  The camera 143 and the processor 114 are configured to communicate with each other such that the processor instructs the camera to capture an image at an appropriate time and receives image data representing the image captured by the camera from the camera. The processor 114 for the vision system 141 may preferably be the same processor that controls the operation of the robotic system 101, but a separate processor may be used within the scope of the present invention. Various cameras can be used within the broad scope of the present invention. For example, the camera 143 can be a digital camera including a CCD array (not shown) that converts a captured image into an electrical signal. Although the camera 143 in the illustrated embodiment is configured to capture a single color (eg, grayscale) image, rather than a color image, for reasons detailed below, the camera is within the broad scope of the present invention. And can be configured to capture color images.

  The machine vision system 141 also includes an illumination 145 for illuminating the container 105 imaged by the camera 143. One or more lights having a variety of different configurations, arrangements, and colors can be used within the scope of the present invention. The illumination may be movable (eg, mounted on the end effector 111) or fixed (eg, fixed or built in the platform 103) within the scope of the present invention. One or more illuminations include bright field illumination, dark field illumination, indirect illumination (eg, side lighting), backlighting, direct illumination (ie, illumination directed perpendicular to the illuminated surface), and these Can be arranged to provide a combination of FIG. 4 shows three optional lights 181, 183, 185 that can be placed in a fixed position relative to a mechanism for receiving a container 105 holding a frozen sample. For example, fiber optic cables may be laid through or provided on the platform to provide illumination at the location indicated at 181, or 183, or 185, or combinations thereof. Other types of lighting may also be fixed at these locations on or in the platform.

  The container 105 is generally transparent, or at least translucent, allowing light to pass through the side or bottom of the container and interact with the frozen sample inside. An illumination 181 at the bottom of the receptacle 106 for receiving the container 105 provides an option for backlighting. An illumination 183 at the top of the container is preferably secured in one of the jaws 108, 110 to provide side lighting, dark field illumination, or both options for the sample surface. I will provide a. The illumination 185 on the side of the receptacle 106 preferably provides a sidelighting option below or on the surface of the frozen sample. When side lighting or back lighting or both options are used, the holes in the frozen sample generally have a brighter appearance than the frozen sample of the corresponding image. Side lighting or back lighting, or both, can be useful in detecting true holes that are filled or completely obstructed by frost or other debris. The machine vision system 141 may include a plurality of different lights, and the processor 114 is configured to manipulate the lights in sequence to utilize the frozen sample image data under different lighting conditions, if desired. obtain.

  As shown in FIGS. 6 and 7, the illumination 145 in the illustrated embodiment is a ring light. The ring light 145 includes a plurality of light sources 147 (eg, LEDs) arranged in an annular (eg, circular) pattern. For example, the ring light 145 may preferably have a hollow cylindrical housing 151 having a groove 153 at one end. The light source 147 is disposed at a deep position in the groove 153 so that the housing blocks the direct path of light from the wide-angle light source. A cover 149 such as a transparent window, a transparent or translucent diffuser, or a lens can be attached to the groove to surround the light source 147 if desired.

  In the embodiment shown in FIGS. 6-7, the camera 143 is arranged such that the optical axis 155 of the camera 143 extends through the central portion of the annular pattern of the ring light 151. For example, the annular ring light 145 preferably has a central axis that is collinear with the optical axis 155 of the camera. The ring light 145 and the camera 143 are preferably arranged such that there is no direct path from the light source 147 in the ring light to the camera. In FIG. 5, for example, the camera 143 has a front end 157 for receiving light from the object being imaged, and the ring light 145 is preferably arranged to extend further forward than the camera. The light emitted by the ring light is emitted from a position in front of the camera. As further shown in FIG. 5, the edge of the housing 151 of the ring light 145 preferably extends between the light source 147 and the camera 143 so as to block the path of light entering the camera directly from the light source. To.

  The processor 114 receives the image data representing the image of the container 105 and the frozen sample therein from the camera 143 and uses the image data from the camera to generate a frozen sample core from the frozen sample contained in the container. It is configured to determine one or more collected positions. The processor 114 may be configured to make this determination using various methods. For example, the processor 114 preferably evaluates the contrast in the image to identify one or more candidate holes, and then uses the information from the image to determine one or more holes. Candidates may be configured to determine whether a candidate is likely an artifact rather than a true hole in the sample.

  The processor 114 preferably processes images taken while the container 105 is illuminated in various ways by the illumination 145 to facilitate this determination. For example, in one embodiment, the processor 114 applies a thresholding filter to the raw image data, and one or more morphological filters (eg, shrink or dilate, or open, or close, Or a combination of these) and then particle analysis is applied to identify one or more hole candidates.

  It will be appreciated that the hole candidates identified by the processor may also include features that are artifacts rather than real holes. For example, the sample surface can be mottled or mottled over time (eg, undesirable formation of frost crystals on a frozen sample, surface profile of a frozen sample due to the rate at which the sample is frozen) Due to the unevenness of the ice, pieces of ice and other debris that can accumulate on the top surface of the frozen sample, such as falling from the lid or side of the container). Furthermore, the true pores resulting from the withdrawal of the frozen sample core generally have a very uniform (eg, circular) appearance initially, but grow on the frozen sample after the frozen sample is returned to the frozen storage location The possible frost crystals can extend in the hole or above the opening at the top of the hole to change the appearance of the hole. Thus, a machine vision system that looks for beautifully and perfectly formed holes and excludes everything else from the list of hole candidates, in particular, a frozen sample requires an additional frozen sample core from that frozen sample. It has been found that when returned to a cryopreservation site for an extended period of time, there is a significant risk of failure to recognize the actual pores present in the sample.

Accordingly, the processor 114 is preferably configured to use multiple types of information to determine whether a hole candidate is likely a true hole candidate or artifact. For example, the processor 114 is preferably
The dimensions of the hole candidates,
The distance between the hole candidate and the center of gravity axis of the container,
An angle formed between the first line and the second line, wherein the first line extends between the hole and the center of gravity axis of the container, and the second line is the center of gravity of the container An angle formed between the first line and the second line, extending between the axis and another hole candidate,
The relationship between the location of one or more candidate holes and the expected pattern of holes in the sample;
The location of one or more candidate holes relative to the periphery of the container;
The total number of hole candidates associated with a particular container,
Configured to use an amount of contrast between the hole candidate and the area surrounding the hole candidate, and information selected from the group consisting of these combinations, the hole candidate may be an artifact or a real hole. Helps determine if it is high.

  In many cases, there is a range of expected dimensions (eg, diameter) for holes formed by extracting a frozen sample core from a frozen sample. However, some hole candidates are substantially larger or substantially smaller than expected, as shown in FIG. Thus, based on the dimensions being too large (eg, D1 in FIG. 9) or too small (eg, D2 in FIG. 9), the processor 114 determines that a particular hole candidate is likely to be an artifact. It can also be judged.

In many cases, the frozen sample core is extracted from the frozen sample according to a predetermined geometric pattern or a geometric pattern that can be recognized by the processor 114 from the captured image data. The geometric pattern varies depending on the objective to be achieved, such as maximizing the number of frozen sample cores that can be extracted from the frozen sample, or collecting as many frozen sample cores as possible at a particular radial location from the center. obtain. FIG. 10 shows an example of one geometric pattern in which five sample cores are extracted from a frozen sample. All the holes resulting from this pattern are separated by approximately the same distance D3 from the center of the container, and all the angles θ ₁ between the lines extending between corresponding points (eg, centers) in adjacent holes are approximately equal. The number of holes in the geometric pattern can vary within the scope of the invention. The pattern of FIG. 10 is a regular pattern, i.e., the holes are all the same size, all spaced from the center by the same distance, and all spaced by the same angle, but the pattern is within the scope of the present invention. It is understood that it can be irregular within.

As shown in FIG. 11, some hole candidates are very close to the center (eg, see distance D4 in FIG. 11) or very far from the center of the container (eg, the distance in FIG. 11). (See D5) placed or, conversely, if the edge of the container can be detected, it can be placed very close to or far from the edge of the container to enter the geometric pattern. Similarly, as shown in FIG. 12, the angular spacing between one or more of the hole candidates may differ from the expected angular spacing (very large θ ₂ or very large Small θ ₃ ). In this way, the processor 114 may indicate that the distance between the hole candidate and the center of gravity axis of the container (or from the edge of the container) is not within the expected range, or corresponding points in the two hole candidates ( The center or edge as shown) and the angle formed between the lines extending between the container's center of gravity axis is not within the expected range, or both It can be determined that there is a high possibility that the hole candidates are artifacts.

  In many cases, the frozen sample core is extracted from the frozen sample according to a specific regular order. As shown in FIG. 10, for example, the frozen sample core starts at the top position and is then extracted by moving clockwise around the geometric pattern until the last sample core is taken. . As shown in FIG. 13, in some cases, one or more of the hole candidates may not be appropriate even though they may be placed at the exact location in the geometric pattern. For example, there may be an empty gap 307 in the pattern between one of the hole candidates 305 and the other hole candidates 301, 303, and if all hole candidates are real holes, the result is This indicates that it was not done in the expected order. In this case, the processor 114 is in accordance with the order in which the plurality of hole candidates 301, 303 are expected, in particular based on the improperness regarding the order in which the frozen sample core is expected to be extracted from the frozen sample accordingly, If only one hole candidate 305 is out of order, it can be determined that the hole candidate is an artifact.

  In some cases, the number of hole candidates may be higher than expected. The processor 114 is preferably configured to recognize this as an indication that one or more of the hole candidates are likely to be artifacts. The processor 114 may apply stricter criteria to help eliminate those likely artifacts if there are too many hole candidates.

  The amount of contrast between a hole candidate and its surrounding area may help distinguish between true holes and artifacts. For example, a large contrast may indicate that it is a good candidate for a true hole, while a small contrast may mean that for other reasons (for example, there are too many hole candidates, there are only two hole candidates and these are expected. May indicate that one of the set of hole candidates that remains questionable (such as not following a geometric pattern) is more likely to be an artifact than the others.

  There is one way in which the processor can evaluate the position of the hole candidates with reference to the position of the center of gravity axis of the container 105 holding the frozen sample or relative to the peripheral edge of the container. The processor 114 may be configured to identify the edge and / or center of the container 105 in various ways within the scope of the present invention. For example, one option is to use an edge detection algorithm to identify the inner or outer periphery of the container 105 and then calculate the geometric center of that edge to identify the container center. It is to be. For example, FIG. 14 shows an image of the container 105 with an overlay including a set of concentric circles 163, 165 and a plurality of radially extending scan lines 167 extending between the circles. Circles 163, 165 define the area to be scanned in an attempt to identify the edge of container 105. The processor 114 is preferably configured to evaluate the image data and determine points 169 with a clear contrast along each line. Each point 169 potentially represents an intersection between the edge of the container 105 and a respective scan line 167. If the attempt to identify the edge of the container is successful, a significant number of points 169 are aligned on the same circle (or other shape if the container is not circular), in which case the processor 114 is over it. It is concluded that a point 169 defines the edge of the container 105. As used herein, in the context of using edge detection techniques as well as information about container edges to identify and / or evaluate hole candidates, container edges are It will be understood that it may refer to other separation regions where a frozen sample is stored on a well or well plate, or the edge of other containers adapted to hold multiple different samples.

  Edge detection techniques can work very well in certain situations, but edge detection may not work properly if the contrast between the edge of the container and the background is low. In the context of machine vision systems for frozen aliquoters, this can present problems when relying solely on edge detection. It is that one of the most common container colors is white or translucent, and white frost can form on the surface adjacent to the container, and in the image the contrast between the edge of the container and the surroundings This leads to a potential problem that can be low. Ultraviolet or infrared illumination can help enhance the contrast between the edge of the container and the surroundings in the image. This enhanced contrast improves the detection and identification of the container edges. In one embodiment, a separate UV or IR light source may be placed to illuminate the container. The UV or IR light source may be movable (eg, mounted on the end effector 111) or fixed (eg, fixed or built in the platform 103) within the scope of the present invention. . In one embodiment, a separate UV or IR light source may be placed on the platform to provide indirect illumination (eg, backlighting or side lighting) to the container to aid in edge detection. In another embodiment, any one or combination of the illuminations 145, 181, 183, 185 may include a UV or IR light source.

  As shown in FIGS. 15 and 16, the set of calibration marks 161 is preferably provided in a fixed position relative to the source container mechanism 107 on the platform 103. The calibration mark 161 is any structure or marking that has sufficient contrast to the background and may allow the processor 114 to reliably identify the calibration mark from the image data. For example, the calibration mark 161 may suitably be a dark opening, a dark marking (eg, a dot), or other structure. The calibration mark 161 is preferably a heater (eg, a small, low power resistance heater) or includes a heater to prevent frost that may block the calibration mark from accumulating on the calibration mark. Suppress.

  The processor 114 preferably uses the calibration mark 161 (eg, in combination with edge detection or without edge detection) to determine whether one or more hole candidates are real samples of the frozen sample. It is configured to determine whether it is likely an artifact rather than a hole. The calibration mark 161 in the image, even if frost is formed or in the image data, even if it is other conditions that minimize the contrast between the edge of the container 105 and its surroundings. It is designed to ensure that there is a strong color contrast between the calibration mark and the surrounding objects. Since the calibration marks 161 are in a fixed position relative to the mechanism 107 that receives the source container 105, the processor 114 determines their positions by comparing the positions of the hole candidates with the positions of the calibration marks. Can do.

  For example, the calibration mark 161 is preferably arranged to form a triangle with the center of the container. The angles α and β formed between the lines connecting the calibration marks and the respective lines drawn from the calibration marks to the center can be grasped before the vision system 141 inspects the frozen sample. Thus, the processor 114 may be configured to identify the center of gravity axis of the container by triangulating the center from the calibration mark. A machine vision system 141 that includes a processor 114 configured to identify the center of the container using calibration marks 161 does not react to errors in camera rotation or camera movement. If it is impractical to use edge detection to determine the center of the container (eg, due to low contrast), the processor 114 may detect the position of the calibration mark without detecting the edge of the container. Accordingly, it may be configured to identify the edge of the container 105 and / or the center of gravity axis of the container. Alternatively, the processor 114 may be configured to identify the center of the container using both calibration marks and perimeter detection. The processor 114 compares the position of the hole candidate directly with the position of the calibration mark 161 without calculating the relative distance between the hole candidate and the center of the container or the edge of the container. Or configured to determine which hole candidates are more likely to be artifacts without calculating the center of the container, without calculating the edge of the container, or a combination thereof. obtain.

  If the processor determines from the image data whether there is a hole in a particular frozen sample and the location of such a hole, then the processor 114 is freezing the robot system 101 from which additional frozen sample cores can be taken. Configured to automatically select the preferred location in the sample (or, if the frozen sample core has not yet been collected, configured to automatically select the location from which the first frozen sample core is removed ) This does not require the processor 114 to obtain information about the number of previous frozen sample cores that may have been extracted from the sample, or the location within the frozen sample from which such sample core was taken, It facilitates the collection of frozen sample cores from samples that may have already been extracted from previous frozen sample cores. This eliminates the need for manual labor to line up the containers 105 in a particular manner, and to better manage and process the sample as the frozen sample core is withdrawn from the frozen sample to satisfy the order for the sample aliquot. Significantly reduce the amount of data about the sample to be tracked. This also applies to one or more different approaches to sample core extraction so far (eg, those using sample core geometric patterns that maximize the number of samples that can be acquired). Even if it was to reduce the maximum number of cores, it used a sample core geometric pattern in which some or all samples were taken from a portion of a frozen sample at a specific radial distance from the center) Help the biobank to install the robot system 101. Thus, if a biobank has already adopted a strategy or a specific set of operational procedures for extracting frozen sample cores, the previously extracted frozen sample core may be If it is being extracted according to the procedure of the system 101 and not otherwise, the system 101 may still recognize the hole in the frozen sample, even if the hole is not in the expected location.

  For example, if the processor 114 detects one or more already existing holes in the frozen sample, the processor will select a position for the next frozen sample core that will continue the geometric pattern already started. Can be configured. If it is desired that the next sample core be taken from a particular radial location in the frozen sample, another option is that the processor 114 is at the desired radial distance from the center of the container and in the frozen sample. It can be configured to select a position that is sufficiently spaced from the existing holes. The processor 114 may be configured to allow the user to select which of these options to use for any particular container or set of containers.

  The processor 114 is also configured to select an appropriate initial geometric pattern for the location from which a plurality of frozen sample cores are withdrawn if the processor determines that there are no existing holes in the frozen sample. The processor 114 may be configured to select a geometric pattern that maximizes the number of frozen sample cores that may be taken from the frozen sample, or the processor 114 may include one or more frozen sample cores (eg, all Of the frozen sample core) can be configured to select a geometric pattern such that it is taken from a particularly desired radial distance from the center of the container, or both. The processor 114 may be configured to allow a user to select which of several different strategies to use for planning the geometric pattern of locations at which frozen sample cores are drawn in different containers or sets of containers. If desired, the processor 114 displays the geometric pattern selected by the processor and / or the location (s) selected by the processor as the site (s) for extracting the frozen sample. And / or configured to facilitate confirmation and / or intervention by a human operator.

  The color of the illumination emitted by the illumination 145 has been found to be important. In general, better results are obtained when the illumination used to illuminate the frozen sample matches the color of the frozen sample. For example, the color of the illumination used to illuminate the frozen sample is preferably the same as the color of the sample, or red, yellow, green, blue-green, blue, magenta and red in the following order: Is approximately the same as the color of the sample compared to one of the two adjacent colors in the RGB hue ring with six colors extending around the ring. For example, red light is suitable for red samples, orange samples, and yellow samples. Because there are a large number of blood (red) and urine (yellow or orange) samples that are frozen for research, it may be desirable for the illumination 145 to emit red light to illuminate the frozen sample. In some cases, it may be desirable for the illumination to emit green or blue light. However, the illumination can emit any color within the broad scope of the invention.

  In one embodiment, the illumination 145 includes a red light emitting element, a blue light emitting element, and a green light emitting element, and the intensity of light emitted by the red, blue, and green light emitting elements can be selectively adjusted and optionally A plurality of different light colors can be selected as light colors used to illuminate the sample. In one embodiment, the illumination 145 includes a red light emitting element that emits red light having a wavelength in a range from about 620 nm to about 750 nm (from about 400 THz to about 484 THz). For example, the majority of the light energy emitted by illumination (eg, substantially all of the light energy) is preferably in the range of about 620 nm to about 750 nm. The light source may include an LED that emits light concentrated in a wavelength range from about 620 nm to about 750 nm. In another embodiment, the illumination 145 includes a green light emitting element that emits green light with a wavelength ranging from about 495 nm to about 570 nm (from about 526 THz to about 606 THz). For example, the majority of the light energy emitted by illumination (eg, substantially all of the light energy) is preferably in the range of about 495 nm to about 570 nm. The light source may include an LED that emits light concentrated in a wavelength range from about 495 nm to about 570 nm. The light sources 147 may include some light sources that emit only red light, other light sources that emit only green light, and other light sources that emit only blue light. Another option is that the light source includes multicolored LEDs that are each operable to emit red, green, blue, and combinations thereof.

  If the illumination color is adjustable, the processor 114 preferably receives input relating to the color of the sample in the container and is adapted to adjust the color of the light emitted by the illumination so that the color of the sample and the illumination Reduce the difference between the color of the emitted light. For example, the vision system 141 includes a user interface configured to allow a user to enter information regarding the color of the sample, and the processor 114 is configured to adjust the color of the lighting to match the color input by the user. Can be done. Another option may be applied such that the camera 143 captures a color image of a sample (or a representative sample of a group of samples) so that the processor 114 matches the color of the sample in the captured color image. It can be configured to adjust the color of the illumination. The processor preferably adjusts the color of the illumination used to illuminate the sample to white when taking an image that is used to determine the color of the sample and facilitate accurate color determination, The illumination color is then adjusted to match the sample color. In some cases, the entire sample set may be known to be similar in color, in which case the processor takes a color image of one of the samples and determines the color of all samples in the set. It is configured to evaluate and adjust the color only once to match the color of all samples in the set.

  The vision system is configured within the scope of the present invention such that the camera takes a color image of the frozen sample and the processor uses information from the color image to identify where the frozen sample core has already been taken. Although the vision system may be non-white, the camera takes a single color (eg, grayscale) image of a frozen sample (the illumination that illuminates the sample is selected to match the color of the sample, for example). The processor is configured to use a grayscale image to determine whether a frozen sample core has already been collected from the frozen sample and, if so, to identify the location where the frozen sample core has already been collected. Surprisingly good results are obtained. Since digital cameras are available that can take both grayscale and color images, the camera takes one or more color images (eg, to identify the color of the sample and illuminate the sample). Can be adjusted to match the color of the sample), and a grayscale image can be taken for the processor to identify the location of the hole in the frozen sample. .

  The machine vision system 141 is preferably a component of a calibration system configured to calibrate the robot system 101. As shown in FIGS. 1-3, the platform 103 has one or more fixed targets 171 disposed thereon. A camera 143 is mounted on the robot system 101 and allows it to move to capture each image of the fixed target 171. The processor is preferably configured to receive image data from the camera representing the image of the target (s) formed by the camera, and the image of one or more fixed targets 171 on the platform 103. And configured to calibrate the robot system 101. As shown in FIG. 1, at least one of the targets 173 has an image (eg, crosshairs) that has intersections of points or lines for calibration in the x and y directions, and for calibration in the z direction. A shape having a known dimension (for example, a circle). The calibration system is preferably configured from the position where the user does not align the camera with one of the targets (eg, the reticle overlying the captured image is not aligned with the crosshairs). It has a user interface (not shown) that is configured to be guided toward a position that is aligned with the target (eg, for this reason, the reticle coincides with the crosshairs).

  Also, as shown in FIGS. 1-3, one of the targets 171 (e.g., a target 173 having a crosshair and a circle) is the top surface of the work deck outside the recess 115 for receiving the tray. Fixed to. Further, one of the targets 175 is preferably secured to the bottom of the recess for receiving the container 105 (eg, between the trays 117 and as shown in FIG. Next to). In the illustrated embodiment, the target 175 in the recess 115 also includes crosshairs.

  Another target in the illustrated embodiment preferably has an image comprising a series of one or more blocks arranged from light to dark for calibration of the light and dark settings of the camera 143. This is a density step target 177 (see FIG. 17).

The processor 114 is preferably configured to help calibrate the robotic system 101 using a plurality of additional features on the platform 103 as targets. For example, the processor 114 is preferably
A mechanism 107 for receiving a container 105 from which a frozen sample core is collected;
A mechanism 109 for receiving a container 105 in which a frozen sample core is placed;
A mechanism 119 for cleaning the coring probe 121 of the robot system 101;
Calibrate the robotic system 101 using images of multiple features on the platform 103 selected from the group consisting of one or more sample trays 117 on the platform 103 for holding the container 105, and combinations thereof. To be configured.

  For example, FIG. 3 shows 13 calibration points that may be used in accordance with one particular embodiment of the calibration system, with each of the calibration points being sequentially numbered from 201 to 213. Point 201 corresponds to target 175 on platform 103 in recess 115. Points 201, 202, and 203 correspond to mechanisms 107 and 109 for receiving container 105 and mechanism 119 for cleaning coring probe 121, respectively. Points 204-208 correspond to various points (eg, corner points) on one side 117a of the tray, and points 209-212 correspond to various points (eg, corner points) on the other side 117b of the tray. doing. Point 213 corresponds to target 173 on the deck of platform 103. The points included in the calibration are preferably selected to extend in a cluster over at least a substantial portion of the operational area, although certain points used in the calibration process are within the scope of the present invention. Can change.

  The calibration system is preferably a robot without any physical contact between (i) the end effector 111 or any component movable with the end effector and (ii) the platform 103 or components on the platform. The system 101 is configured to complete the calibration.

  The calibration system also preferably determines the position of the camera 143, the coring probe 143, and the gripper 127 relative to each other to compensate for variations related to the position offsets associated with the camera, probe, and gripper. Consists of. For example, the calibration system preferably allows the user to guide the movement of the end effector 111 to align each of the camera 143, coring probe 121, and gripping system 127 with a target 171 or other reference point, Each time one of these is aligned there, it is configured to provide instructions to the processor 114. This allows the processor 114 to calculate the offset between these components, thereby compensating for position variations that may occur during assembly of the robotic system 101 or for any other reason. This facilitates more accurate positioning of the components of the robot system 101.

  The machine vision system 141 is preferably used to calibrate the robotic system during the initial installation of the robotic system 101 and sometimes thereafter as needed. The robot drive system 113 moves the camera 143 to a position that is inferred by the processor to be aligned with one of the targets 171. The image data from the camera 143 is then used (either by the processor or the user) to instruct the robot drive system 113 to adjust the position of the camera until it is aligned with the target. If the target 171 includes a circle or other shape with a known dimension for calibration in the z direction, the image data from the camera 143 is used (either by the processor or the user) and the camera is removed from the target. The robot drive system 113 is instructed to raise and lower the camera until the dimensions of the shape in the image taken by the camera indicate that it is at an appropriate z-direction distance. Alternatively, z-direction calibration can be accomplished using the lens settings of camera 143 with a known focal length and then adjusting the camera height until the image is in focus. If the camera 143 is aligned with the target 171 and is at the correct distance from the target, the processor may be from the robot system 101 (eg, from an encoder or other device that provides position related feedback regarding the position of various components of the robot system). Data) is recorded, and the information is designated as corresponding to the setpoint corresponding to each target. The process is repeated for each of the targets 171. For example, in the embodiment shown in FIG. 3, the process is repeated for each of the calibration points 201-213.

Although the target 171 and / or calibration point may be located at various locations on the platform 103, in the illustrated embodiment, the target preferably places one target 173 on the top surface of the platform deck. In addition, another target 175 is included in the recess of the platform. The target 171 and / or the calibration point are preferably
A mechanism 107 for receiving a container 105 from which a frozen sample core is collected;
A mechanism 109 for receiving a container 105 in which a frozen sample core is placed;
A mechanism 119 for cleaning the coring probe 121;
And a plurality of targets including at least one of one or more trays 117a, 117b on the platform 103 for holding the container 105, and combinations thereof.

  For example, in one embodiment, calibration target 171 and calibration points include each of points 201-213 in FIG.

  A density step target 177 is also used during the calibration process to adjust camera settings and light intensity to provide standard imaging conditions. While the illumination 145 is turned on and the camera captures an image of the density step target 177, the camera 143's until a specific dark block on the density step target camera 143 is read by the camera 143 as a specific gray level. Either the diaphragm and / or the intensity of the current supplied to the illumination is adjusted. For example, good results have been obtained when the illumination 145 and camera 143 are adjusted so that the brightest third color block of the standard density step target is read by the camera as a 200 gray level.

  The robotic system 101 is also calibrated to adjust for any variations in the offset between the positions of the camera 143, coring probe 121, and gripping system 127. For example, the camera 143 is first placed in alignment with one of the targets 171 or other reference points on the platform 103, at which point the user has the camera aligned with the processor 114. To be instructed. The user then adjusts the position of the end effector 111 until the coring probe 121 is aligned with the target 171 or reference point and indicates to the processor 114 that the coring probe is aligned there. Finally, the user adjusts the position of the end effector until the gripping system 127 is aligned with the target 171 or other reference point and indicates to the processor that the gripping system is aligned there. The order of the method steps is not important. The processor 114 uses the information provided in this process to determine a position related offset between the camera 143, the coring probe 121, and the gripping assembly 127. The entire calibration process is preferably completed without any physical contact between the end effector 111 or any component movable with the end effector and the platform 103 or any component on the platform.

  A set of containers 105 containing frozen samples is placed on the platform 103 to extract the frozen sample core from the frozen sample. For example, one or more trays 117a may be placed on the platform 103 (eg, in the recess 115) after loading the sample container 105. A set of empty containers 105 for receiving frozen sample cores after withdrawal is loaded into one or more additional trays 117b and placed on the platform 103. The robot system 101 uses the gripping system 127 to move one of the containers 105 containing the frozen sample to a mechanism 107 for receiving the container from which the frozen sample core is withdrawn, and out of the empty containers One of these is moved to a mechanism 109 for receiving a destination container into which the frozen sample core is placed.

  The robotic system then moves the camera 143 to a predetermined position on the mechanism 107 for holding the container 105 containing the frozen sample while the frozen sample core is extracted therefrom. The robotic system preferably includes a fill level detection system for detecting the height of the top surface of the frozen sample. Details regarding the construction and operation of the preferred fill level detection system are: Robotic End Effector for Frozen Aliquoter and Methods of Taking a Frozen Aliform 301, the contents of which are incorporated herein by reference. The fill level detection system provides information regarding the position of the top surface of the frozen sample. The fill level detection system can be used to position the camera 143 at a desired height above the frozen sample to improve camera focusing. For example, the processor 114 uses information about the position of the top surface of the frozen sample from the fill level detection system to obtain a camera for acquiring an image of the frozen sample while the camera is within an optimal range with respect to the distance from the top surface of the sample. Determine the altitude to place. Illumination 145 is used to illuminate the container 105 of mechanism 107 and the frozen sample contained therein. If the machine vision system 141 includes an option to adjust the color of the illumination 145, the color of the frozen sample is measured (eg, using image data from the camera and / or user input), as described above. Thus, the color of the illumination is adjusted to match the color of the frozen sample. For example, if the frozen sample is red, orange, or yellow, the illumination 145 can be adjusted to emit red light. Similarly, if additional lighting options are used, additional images of the container 105 are captured with one or more of the lights 181, 183, 185 that provide the lighting.

  The camera 143 takes one or more raw images of the illuminated container 105 and frozen sample. The raw image is preferably processed to facilitate recognition of hole candidates. For example, the threshold processing filter is preferably applied to raw images acquired with illumination from illumination 145. A morphological filter may also be applied to the image. After filtering the image, a particle analysis image algorithm is performed to recognize hole candidates. The processor then determines whether the frozen sample core has been collected from the frozen sample and, if so, the processor uses the image data to identify any holes Evaluate whether the candidate is an actual hole or just an artifact.

  If the option of using the calibration mark 161 to evaluate the position of the hole candidate is used, the method is preferably a low output resistance heater to ensure that the calibration mark is not obstructed by frost. Heating the calibration mark using.

  Once the processor 114 has identified hole candidates and determined which hole candidates are likely to be artifacts and which are more likely to be true holes, the processor is already present in the frozen sample. If there is a hole, the position where the frozen sample core is extracted is automatically selected in consideration of the position. The processor 114 then instructs the robot system 101 to move the coring probe 121 to a predetermined position above the selected position and also instructs the sample extraction system to extract the frozen sample core from that position. Make it. As a result, the coring probe 121 is extended into the frozen sample (eg, rotating if the sample extraction system 123 uses a drilling operation), and then includes the frozen sample core therein and is withdrawn from the frozen sample. . The robot system 101 moves the coring probe 121 to a predetermined position on the top of the container 105 by a mechanism 109 for holding the destination container, and takes out the frozen sample core from the coring probe into the destination container. If more than one frozen sample core is required to provide sufficient material for the ordered aliquot, the frozen sample core withdrawal process will continue until a sufficient amount of sample material is placed in the destination container 105. It repeats at another suitable location within the frozen sample, automatically determined by the processor.

  After a sufficient amount of sample material has been placed in the destination container 105, the frozen parent sample is further processed to clean up any frost crystals and other debris in each of the frozen sample holes and stored frozen. If the sample is later collected again from the location to obtain additional frozen sample cores, better accuracy and reliability is ensured in the machine vision system 141. As mentioned above, the holes are frost crystals that grow on the sample (eg, while the sample is in a cryopreservation location), or strips (eg, due to previous drilling of the frozen sample core), or other Reasons, or combinations thereof, may be included or blocked by this. In order to improve the ability to recognize holes in the frozen parent sample the next time it is retrieved from the frozen storage location, the processor 114 is preferably obtained from the sample before the last frozen sample core is withdrawn. Using image data (ie, image data used to evaluate hole candidates, determination of whether a frozen sample core has already been taken from a frozen sample, and if a frozen sample core has already been taken, it is already Collected location (s). The processor 114 preferably also uses data acquired during the extraction of the last frozen sample core (eg, the geometric sampling pattern used to acquire the last frozen sample core (s), the last frozen sample core). Information about where the sample core was taken). Using this image data for every hole location (s) prior to the last extraction and data for any hole location (s) created during the last extraction, the processor 114 may Determine the location (s) of the hole and the suspicious hole. Once the processor 114 has identified all the hole (s) in the frozen sample, the processor instructs the robotic system to move the coring probe 121 in turn over each hole. Reprocess or clean the holes with the strips. The processor 114 may or may not have information regarding whether the hole has any strips that obstruct it or are contained therein, and therefore instructs the robotic system to correlate the probe 121. In order to move into a predetermined position above each hole in order to descend into the hole, regardless of whether any strips are identified in the hole. In one embodiment, the processor 114 instructs the robotic system to reprocess or clean any hole (s) identified using the image data acquired before the last frozen sample core was extracted, The robot system is then instructed to reprocess or clean any hole (s) created during removal of the last frozen sample core. In another embodiment, the processor 114 instructs the robotic system to reprocess or clean any hole (s) created during the extraction of the last frozen sample core, and then continue to the robotic system. Command and reprocess or clean any hole (s) identified using the image data acquired before the last frozen sample core was drawn. In another embodiment, the processor 114 instructs the robot system to reprocess or clean only any hole (s) identified using the image data acquired before the last frozen sample core was extracted. To do. In another embodiment, the processor 114 instructs the robotic system to reprocess or clean only any hole (s) created during removal of the last frozen sample core. In yet another embodiment, the processor 114 instructs the robotic system to acquire image data acquired before the last frozen sample core is extracted before instructing the robotic system to extract the last frozen sample core. Reprocess or clean any hole (s) identified using.

  Any order or combination of cleaning processes is within the broad scope of the present invention. As shown in FIGS. 18 and 19, for example, the processor 114 may guide the extraction pin 190 of the end effector 111 to move it to an extended position where the extraction pin extends from the distal end of the coring probe 121. . The coring probe 121 is placed over the identified hole 192 (FIG. 18) and then lowered into the hole to clean the hole with the strip (FIG. 19). The coring probe 121 is lowered into the identified hole 192 with or without a strip in the hole. If the pick pin 190 meets resistance (eg, the identified hole is actually an artifact, not a real hole), this resistance is detected (eg, using a component of the fill level detection system) and the core The downward movement of the ring probe 121 and take-out pin is stopped so that the frozen sample and the robot system are not damaged. The machine vision system 141 includes a sensor as described in US patent application Ser. No. 13 / 359,301, filed Jan. 26, 2012, and is not lowered and contacted with a frozen sample. It can be determined whether the pin 190 has been lowered into the hole. When the extraction pin 190 enters the open end of the hole, any frost, debris or other similar object that can obscure the visibility of the hole is either wiped off the bottom of the hole or pushed aside It is wiped off from the open end of the hole. Removing debris and frost from the hole (s) requires additional frozen sample cores, and the machine vision system 141 will accurately pin the frozen sample holes when the frozen sample is next retrieved from the frozen storage location. To make it easy to identify. Since the coring probe and take-out pin 190 are already forced to contact the sample to complete the rest of the process, the coring probe and take-out pin to remove debris from the open end of the hole By using pins, there is virtually no additional risk of contaminating the sample.

  The following description is intended to provide a brief overview of a suitable processing environment in which aspects of the present invention may be implemented. Although not required, aspects of the invention will be described in the general context of computer-executable instructions, such as program modules, being executed by computers or processors in network environments. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. Computer-executable instructions, associated data structures, and program modules represent examples of program code means for executing steps of the methods disclosed herein. Certain sequences, such as executable instructions or associated data structures, represent examples of corresponding actions for performing the functions described in such steps.

  Aspects of the present invention can be used with many types of computer system configurations, including personal computers, portable devices, multiprocessor systems, microprocessor-based or programmable electronic devices, network PCs, minicomputers, mainframe computers, etc. Those skilled in the art will appreciate that they can be implemented in a network computing environment. Aspects of the invention may also be used in distributed computing environments where tasks are performed by local and remote processing devices that are linked through a communications network (via a wired link, or a wireless link, or a combination of wired or wireless links). Can be implemented. In a distributed computing or processing environment, program modules may be located in both local and remote memory storage devices.

  An exemplary system for implementing aspects of the present invention is a general purpose computer in the form of a conventional computer including a processing device, a system memory, and a system bus that couples various system components including the system memory to the processing device. including. The system bus may be any of several types of bus structures including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. The system memory includes read only memory (ROM) and random access memory (RAM). A basic input / output system (BIOS) that includes basic routines that help transfer information between elements in the computer, such as at startup, may be stored in ROM.

  The computer also provides a magnetic hard disk drive for reading and writing to a magnetic hard disk, a magnetic disk drive for reading and writing to a removable magnetic disk, and a read / write to a removable optical disk such as a CD-ROM or other optical media. An optical disk drive may be included. The magnetic hard disk drive, magnetic disk drive, and optical disk drive are connected to the system bus by a hard disk drive interface, a magnetic disk drive interface, and an optical drive interface, respectively. The drive and associated computer readable media provide non-volatile storage of computer-executable instructions, data structures, program modules, and other computer data. Exemplary environments described herein utilize magnetic hard disks, removable magnetic disks, and removable optical disks, but include magnetic cassettes, flash memory cards, digital video disks, Bernoulli cartridges, RAM, ROM, etc. Other types of computer readable media for storing data may also be used.

  The program code means comprising one or more program modules comprises an operating system, one or more application programs, other program modules, and program data, a hard disk, a magnetic disk, or an optical disk, or It can be stored in ROM, RAM, or a combination thereof. A user may enter commands and information into the computer through a keyboard, pointing device, or other input device such as a microphone, joystick, game pad, satellite antenna, or scanner. These and other input devices are often connected to the processing unit via a serial port interface coupled to the system bus. Alternatively, the input device may be connected by other interfaces such as a parallel port, a game port, or a universal serial bus (USB). A monitor or another display device is also connected to the system bus via an interface, such as a video adapter. In addition to the monitor, personal computers typically include other peripheral output devices (not shown) such as speakers and printers.

  A computer may operate in a networked environment using logical coupling to one or more remote computers, such as a remote computer. Each remote computer is another personal computer, server, router, network PC, peer device or other common network node and may generally include most or all of the elements described above with respect to the computer. Logical connections include a local area network (LAN) and a wide area network (WAN), which are shown here by way of example and not limitation. Such network environments are common in office-scale or enterprise-scale computer networks, intranets and the Internet.

  When used in a LAN network environment, the computer is connected to the local network via a network interface or adapter. When used in a WAN network environment, the computer may include a modem, a wireless link, or other means for establishing communications over a wide area network such as the Internet. A modem, which can be internal or external, is connected to the system bus via a serial port interface. In a network environment, the program modules described with respect to the computer, or portions thereof, may be stored in a remote memory storage device. Of course, the network connections presented are exemplary and other means for establishing communications over a wide area network may be used.

  Embodiments within the scope of the present invention also include computer-readable media for carrying or having computer-executable instructions or data structures stored therein. Such computer-readable media can be any available media that can be accessed by a general purpose or special purpose computer. By way of example and not limitation, such computer readable media may be RAM, ROM, EEPROM, CD-ROM or other optical disk storage device, magnetic disk storage device, or other magnetic storage device, or as desired. Any other medium that can be used to carry or store the program code means in the form of computer-executable instructions or data structures and that can be accessed by a general purpose or special purpose computer. When information is transferred or provided to a computer over a network or another communication connection (wired or wireless, or a combination of wired or wireless), the computer suitably considers this connection as a computer-readable medium. As such, any such connection is properly termed a computer-readable medium. Combinations of the above should also be included within the scope of computer-readable media. Computer-executable instructions comprise, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing device to perform a certain function or group of functions.

  In one mode of operation, the frozen sample contained in the container 105 is placed (eg, by a robotic system) on a mechanism 107 on the platform 103 having a calibration mark 161 in a fixed position relative to the mechanism. The camera 143 takes images of the container 105 and the sample therein while the container 105 is received by the mechanism 107. The processor 114 (a) evaluates the contrast in the image to identify one or more hole candidates in the frozen sample, and (b) uses information regarding the position of the calibration mark relative to the hole candidates. A frozen sample core has already been taken from the frozen sample contained in the container 105 by determining whether one or more candidates are likely to be an artifact rather than a true hole in the frozen sample. One or more positions are determined. The frozen sample core is taken from the sample at a location where the frozen sample core determined by the processor has not yet been taken.

In another mode of operation, the camera takes an image of the container 105 containing the frozen sample. The processor 114 (a) evaluates the contrast in the image to identify one or more hole candidates, and (b) one or more hole candidates are the real samples of the frozen sample. One or more locations where a frozen sample core has already been taken from a frozen sample contained in a container using the captured image by determining whether it is likely an artifact rather than a hole Determine. In order to make this determination, the processor 114 has the following information:
The dimensions of the hole candidates,
The distance between the hole candidate and the center of gravity axis of the container 105,
An angle formed between the first line and the second line, wherein the first line extends between the hole and the center of gravity axis of the container, and the second line is the center of gravity of the container An angle formed between the first line and the second line, extending between the axis and another hole candidate,
The relationship between the location of one or more candidate holes and the expected pattern of holes in the frozen sample;
The location of one or more candidate holes relative to the periphery of the container;
The number of hole candidates identified,
Information is used that includes at least one of the amount of contrast between the hole candidate and the surrounding region, and combinations thereof.

  The system collects the frozen sample core from the frozen sample at a location where the frozen sample core determined by the processor has not yet been collected.

  In yet another mode of operation, the robotic system 101 is calibrated by taking images of one or more fixed targets 171 on the platform 103 using the camera 143. The processor 114 uses the image of one or more targets 171 to calibrate the robot system. The same camera 143 is then used to take an image of one or more containers 105 while the containers are being supported by the platform, so that one or more frozen sample cores are obtained from the frozen sample. Determine whether it has already been collected.

  In yet another mode of operation, the robotic system 101 is operated to move the camera 143 relative to the frozen sample in the first container 105 so that the camera is directed at the frozen sample in the first container. To. The frozen sample in the container 105 is illuminated using a ring light 145. The camera 143 is used to take an image of the illuminated frozen sample. The processor 114 evaluates the contrast of the captured image, processes the image, and identifies one or more hole candidates in the captured image. The processor 114 determines whether the hole candidate is likely to be an artifact or a true hole in the frozen sample. The robotic system 101 moves the camera relative to the second of the containers 105 so that the camera is pointed at the frozen sample in the second container and the process is repeated.

  In yet another mode of operation, the robotic system 101 moves the camera 143 relative to the first container 105 so that the camera is pointed at the frozen sample in the first container. The frozen sample in the container 105 is illuminated with colored light. The camera 143 takes a grayscale image of the illuminated frozen sample. The processor 114 evaluates the contrast of the captured image, processes the image, and identifies one or more hole candidates in the captured image. The processor 113 determines whether the candidate hole is likely to be an artifact or a true hole in the frozen sample. The robotic system 101 moves the camera 143 relative to the second of the containers 105 so that the camera is pointed at the frozen sample in the second container. The process is repeated.

  In another mode of operation, the robotic system 101 moves the camera 143 relative to the first container 105 so that the camera is pointed at the frozen sample in the first container. The frozen sample in the container 105 is illuminated with illumination 145 having a color selected to match the color of the frozen sample. The camera 143 takes an image of the illuminated frozen sample. The processor 114 evaluates the contrast of the captured image, processes the image, and identifies one or more hole candidates in the captured image. The processor 114 determines whether the hole candidate is likely to be an artifact or a true hole in the frozen sample. The robotic system moves the camera 143 relative to the second of the containers 105 so that the camera is pointed at the frozen sample in the second container and the process is repeated.

  In another mode of operation, the robotic system 101 places one of the containers 105 on the platform 103 into a mechanism 107 for receiving a container while the frozen sample core is withdrawn from the frozen sample contained in the container. Deploy. One or more of the illuminations 181, 183, 185 provide at least one of backlighting and sidelighting for the container 105. In another embodiment, one or more lights may provide light directly to the container 105. The camera 143 captures an image of the frozen sample while it is illuminated directly or indirectly (eg, side lights and / or backlight). The processor 114 evaluates the contrast of the captured image, processes the image, and identifies one or more hole candidates in the captured image.

  The modes of operation described above can be used in combination within the scope of the present invention, or they can be used separately.

  In introducing an element of the present invention or a preferred embodiment thereof, the articles “a”, “an”, “the”, and “said” are such that one or more of the elements are present. Is intended to mean The terms “comprising”, “including”, and “having” are intended to be inclusive and there may be additional elements other than the listed elements. Means that.

  In view of the above, it will be seen that the several objects of the invention are achieved and other advantageous results are obtained.

  Since various modifications can be made to the above construction without departing from the scope of the present invention, all matters included in the above description and shown in the accompanying drawings are construed in an illustrative manner. It is intended and should not be construed in a limiting sense.

Claims (120)

A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the machine vision system comprising:
A platform for supporting one or more of the containers, the platform having a mechanism for receiving at least one of the containers and on the platform with respect to the mechanism; A platform having a set of calibration marks in a fixed position
A camera for taking an image of the container while the container is received by the mechanism;
Receiving image data representing the image of the container from the camera, and (a) evaluating the contrast in the image to identify one or more hole candidates, and (b) ) Using information about the position of the calibration mark relative to the hole candidate, determine whether the one or more candidates are likely to be artifacts rather than real holes in the sample A machine vision system comprising: a processor configured to determine one or more positions from which frozen sample cores have already been collected from frozen samples contained in the container.   Using information about the position of the calibration mark relative to the hole candidate, identifying the center of gravity axis of the container using the calibration mark; and (i) the one for the center of gravity axis of the container Or using information about the position of two or more hole candidates, and (ii) the hole relative to the centroid of the container compared to another angular position of the hole candidates relative to the centroid of the container The machine vision system of claim 1, comprising using at least one of information about an angular position of one of the candidates.   Using the information regarding the size of the one or more hole candidates to determine, the one or more candidates may be an artifact rather than a real hole in the sample. The machine vision system of claim 1 or 2, further comprising determining whether it is high.   4. The processor according to claim 2, wherein the processor is configured to identify the center of gravity axis of the container according to the position of the calibration mark without detecting an edge of the container. The machine vision system according to item 1.   The processor is configured to identify an edge of the container, and the determining uses information regarding the position of the one or more candidate holes relative to the edge of the container, 4. Machine vision according to any one of the preceding claims, comprising determining whether one or more candidates are likely to be artifacts rather than real holes in the sample. system.   The machine vision system according to any one of claims 1 to 5, wherein the calibration mark includes a low output resistance heater and suppresses accumulation of frost on the calibration mark.   The machine vision system of claim 1, wherein the processor is further configured to control a position of the camera relative to a position of the mechanism.   The processor uses information about the position of the one or more hole candidates relative to the center of gravity axis of the container so that the one or more hole candidates are not real holes in the sample; The machine vision system of claim 1, configured to determine whether an artifact is likely.   The processor uses the information regarding the angular position of one of the hole candidates for the centroid axis of the container compared to another angular position of the hole candidates for the centroid axis of the container. 9. The method of any one of claims 1-8, configured to determine whether one or more hole candidates are likely to be artifacts rather than real holes in the sample. Machine vision system.   The machine vision system according to claim 1, wherein the processor is configured to identify the centroid axis of the container by triangulating a center from the calibration mark. A method for collecting a frozen sample core from a frozen sample contained in a container, the method comprising:
Placing the container in a mechanism for receiving a container on the platform, the platform having a set of calibration marks in a fixed position relative to the mechanism on the platform; Placing step;
Taking an image of the container while the container is received by the mechanism;
(A) evaluating the contrast in the image to identify one or more hole candidates; and (b) using the information regarding the position of the calibration mark relative to the hole candidates, A frozen sample core has already been collected from the frozen sample contained in the container by determining whether one or more candidates are likely to be artifacts rather than the true pores of the frozen sample Determining one or more locations that have been performed;
Collecting the frozen sample core from the frozen sample contained in the container, the method comprising: collecting the frozen sample core from the sample at a position where the frozen sample core is not yet collected as determined in the determining step .   The determining step helps to determine whether the one or more candidates are likely to be an artifact rather than a real hole in the frozen sample. The method of claim 11, further comprising using information regarding the dimensions of one or more hole candidates.   Further comprising detecting a peripheral edge of the container in the captured image, wherein the determining step uses information regarding the position of the one or more hole candidates relative to the edge of the container; 13. The method of any one of claims 11 to 12, further comprising the step of helping determine whether the one or more candidates are likely to be artifacts rather than true holes in the frozen sample. The method according to item.   The method according to any one of claims 11 to 13, further comprising heating the calibration mark to prevent frost from accumulating on the calibration mark.   Using the information about the position of the calibration mark with respect to the one or more hole candidates identifies the center of gravity axis of the container; and the position of the hole candidate with respect to the center of gravity axis of the container. And determining whether the one or more candidates are likely to be artifacts rather than real holes in the frozen sample. Method.   The step of evaluating the position of the hole candidate with respect to the center of gravity axis of the container is compared with another angle position of the hole candidates with respect to the center of gravity axis of the container, and Determining whether the one or more candidates are likely to be an artifact rather than a real hole in the frozen sample, comprising using information about the angular position of The method of claim 15.   17. A method according to any one of claims 15 to 16, wherein identifying the center of gravity axis of the container comprises using triangulation.   18. The method of any one of claims 11 to 17, wherein evaluating the contrast of the image to identify one or more candidate holes comprises applying a thresholding filter to the image. Method.   19. The method of claim 11, wherein evaluating the contrast of the image to identify one or more hole candidates comprises applying a morphological filter to the image. Method.   Evaluating the contrast of the image to identify one or more hole candidates further includes applying a morphological filter to the image after applying the thresholding filter to the image. The method of claim 18 comprising.   21. The method of any one of claims 18-20, wherein evaluating the contrast of the image to identify one or more hole candidates comprises applying a particle analysis image algorithm after the filtering. The method described in 1. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the machine vision system comprising:
Platform,
A camera for taking an image of one of the containers while it is on the platform;
Receiving image data representing the image taken by the camera from the camera, and (a) evaluating the contrast of the image to identify one or more hole candidates; and (B) whether the one or more hole candidates are likely to be artifacts rather than real holes in the sample;
(I) the size of the hole candidate;
(Ii) a distance between the hole candidate and the center of gravity axis of the container;
(Iii) an angle formed between a first line and a second line, wherein the first line extends between the hole and the barycentric axis of the container; An angle formed between a first line and a second line, wherein two lines extend between the center of gravity axis of the container and another hole candidate;
(Iv) a relationship between the position of the one or more hole candidates and the expected pattern of holes in the sample;
(V) the position of the one or more hole candidates relative to the periphery of the container;
(Vi) number of hole candidates identified,
(Vii) the amount of contrast between the hole candidate and the area surrounding the hole candidate, and (viii) the determination of at least one of these combinations to determine whether the container contained in the container A machine vision system comprising: a processor configured to determine one or more locations where a frozen sample core has already been taken from a frozen sample.   The processor uses information regarding the location of the one or more hole candidates relative to the periphery of the container so that the one or more hole candidates are not real holes in the frozen sample, 23. The machine vision system of claim 22, configured to determine whether an artifact is likely.   The processor may use the information regarding the dimensions of the one or more hole candidates to cause the one or more hole candidates to be an artifact rather than a real hole in the frozen sample. 24. A machine vision system as claimed in any one of claims 22 to 23 configured to determine whether or not is high.   The processor may use the information about the distance between the hole candidate and the center of gravity axis of the container to allow the one or more hole candidates to be an artifact rather than a real hole in the frozen sample. 25. A machine vision system according to any one of claims 22 to 24, configured to determine whether or not the property is high.   The processor has the angle formed between a first line and a second line, the first line extending between the hole and the center of gravity axis of the container; By using information about the angle formed between the first line and the second line, wherein the second line extends between the center of gravity axis of the container and another hole candidate, 26. The method of any one of claims 22-25, wherein the one or more hole candidates are configured to determine whether they are likely to be artifacts rather than real holes in the sample. The machine vision system described.   The processor uses the information about the relationship between the position of the one or more candidate holes and the expected pattern of holes in the frozen sample to thereby determine the one or more 27. A machine vision system according to any one of claims 22 to 26, configured to determine whether a hole candidate is likely to be an artifact rather than a real hole in the frozen sample.   Whether the processor uses the information about the number of identified hole candidates to determine whether the one or more hole candidates are likely to be artifacts rather than real holes in the frozen sample. 28. A machine vision system according to any one of claims 22 to 27, wherein the machine vision system is configured to determine   The processor uses the information about the amount of contrast between the hole candidates and the region around the hole candidates so that the one or more hole candidates are real holes of the frozen sample. 29. The machine vision system of claim 28, configured to determine whether it is likely an artifact rather than. A method for collecting a frozen sample core from a frozen sample contained in a container, the method comprising:
Taking an image of the container;
(A) evaluating the contrast of the image to identify one or more hole candidates; and (b) the one or more hole candidates being real holes in the frozen sample. And whether it ’s likely to be an artifact,
(I) the size of the hole candidate;
(Ii) a distance between the hole candidate and the center of gravity axis of the container;
(Iii) an angle formed between a first line and a second line, wherein the first line extends between the hole and the barycentric axis of the container; An angle formed between a first line and a second line, wherein two lines extend between the center of gravity axis of the container and another hole candidate;
(Iv) a relationship between the position of the one or more candidate holes and the expected pattern of holes in the frozen sample;
(V) the position of the one or more hole candidates relative to the peripheral edge of the container;
(Vi) number of hole candidates identified,
(Vii) the amount of contrast between the hole candidate and the surrounding region, and (viii) the frozen sample contained in the container by determining using information including at least one of these combinations Using the captured image to determine one or more locations from which frozen sample cores have already been collected;
Collecting the frozen sample core from the frozen sample contained in the container, the method comprising: collecting the frozen sample core from the sample at a position where the frozen sample core is not yet collected as determined in the determining step .   31. The method of claim 30, wherein the determining comprises using information regarding the location of the one or more hole candidates relative to a peripheral edge of the container.   32. A method according to any one of claims 30 to 31, wherein the determining step comprises using information regarding the dimensions of the one or more hole candidates.   33. A method according to any one of claims 30 to 32, wherein the determining step comprises using information regarding the distance between the hole candidate and the center of gravity axis of the container.   The determining step is the angle formed between a first line and a second line, the first line extending between the hole and the center of gravity axis of the container. And using the information about the angle formed between the first line and the second line, wherein the second line extends between the center of gravity axis of the container and another hole candidate. 34. The method according to any one of claims 30 to 33, comprising:   31. The determining step comprises using information regarding the relationship between the location of the one or more hole candidates and an expected pattern of holes in the frozen sample. 35. The method according to any one of 34.   36. A method according to any one of claims 30 to 35, wherein the determining step comprises using information regarding the number of hole candidates to be identified.   37. A method according to any one of claims 30 to 36, wherein the determining step comprises using information regarding the amount of contrast between the hole candidate and the surrounding region. A calibration system configured to calibrate a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the calibration system comprising:
A platform for supporting the container, the platform having one or more fixed targets thereon;
While the container is supported by the platform, to image one or more containers and also to image one or more fixed targets disposed on the platform A camera mounted on the robot system;
A processor,
Receiving image data representing an image formed by the camera from the camera;
A calibration system comprising: a processor configured to calibrate the robotic system using images of the one or more fixed targets on the platform.   The processor is further configured to determine one or more locations where a frozen sample core has already been taken from a frozen sample of one of the containers by evaluating a contrast in the image of the container. 40. The calibration system of claim 38.   The one or more fixed targets comprise a target having an image for calibration in the x and y directions and a shape having a known dimension for calibration in the z direction. 40. The calibration system according to any one of 39.   The platform includes a work deck having a recess for receiving the container, the target having the image for calibration in the x and y directions, and a known dimension for calibration in the z direction. 41. The calibration system of claim 40, wherein the shape having is secured to an upper surface of the work deck outside the recess.   42. The calibration system of claim 41, wherein the one or more fixed targets include a target that is fixed to a bottom of the recess.   43. The calibration system of claim 42, wherein the target secured to the bottom of the recess has an image for calibration in the x and y directions.   44. The calibration system according to any one of claims 38 to 43, further comprising a density step display on the platform for light / dark calibration of the camera. The processor is
(I) a mechanism for receiving the container from which the frozen sample core is collected;
(Ii) a mechanism for receiving a container in which the frozen sample core is placed;
(Iii) a mechanism for cleaning the coring probe of the robot system;
(Iv) one or more trays on the platform for holding the container; and (v) images of features on the platform selected from the group consisting of combinations thereof 45. A calibration system according to any one of claims 38 to 44, further configured to calibrate the robot system.   The user interface further configured to allow a user to guide the camera from a position that is not aligned with one of the targets toward a position that is aligned with the target. 46. The calibration system according to any one of 45.   The robot system comprises an end effector, and the camera is mounted on the end effector for movement with the end effector relative to the platform and is movable with the end effector or the end effector 47. Any one of claims 38 to 46, wherein the calibration system is configured to complete calibration of the robotic system without any physical contact between the platform and components on the platform. The calibration system according to claim 1.   The robot system includes an end effector, and the camera is mounted on the end effector for movement with the end effector relative to the platform, the end effector taking a frozen sample core from the frozen sample. And a coring probe for moving the container relative to the platform, the calibration system being adapted to selectively hold and release the container, wherein the calibration system comprises the camera 47. Any one of claims 38-46, further configured to determine a position of the probe, and the gripper relative to each other to compensate for variations in position offset associated with the camera, probe, and gripper. Calibration system according to item. A method of calibrating a robotic system for collecting a plurality of frozen sample cores from frozen samples, each contained in a respective container, the method comprising:
One or more immobilizations on the platform while the container is supported by the platform of the robotic system to determine if one or more frozen sample cores have already been taken from the frozen sample Using a camera to capture an image of one or more containers to capture an image of the target;
Calibrating the robotic system using images of the one or more targets.   The platform includes a work deck having a recess for receiving the container, and at least one of the one or more targets is secured to an upper surface of the work deck outside the recess. 50. The method of claim 49.   51. The method of claim 50, wherein the one or more fixed targets also include at least one target that is fixed to the bottom of the recess.   52. A method according to any one of claims 49 to 51, further comprising using a density step display on the platform to calibrate the camera's light and dark settings. Using a plurality of additional feature images on the platform to aid calibration of the robotic system, the plurality of additional features comprising:
(I) a mechanism for receiving the container from which the frozen sample core is collected;
(Ii) a mechanism for receiving a container in which the frozen sample core is placed;
(Iii) a mechanism for cleaning the coring probe of the robot system;
53. In any one of claims 49 to 52, comprising (iv) one or more trays on the platform for holding the container, and (v) at least one of a combination thereof. The method described.   54. The method of claims 49-53, further comprising: taking an image of the container using the camera; and determining the position of one or more holes in the sample using the image. The method according to any one of the above. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A camera for taking an image of a container while the container is supported on a platform, the camera having an optical axis;
A ring light for illuminating the container on the platform, the ring light comprising a plurality of light sources arranged in an annular pattern, the optical axis of the camera extending through a central portion of the annular pattern With a ring light,
A frozen sample core has already been collected from the sample contained in the container by receiving image data representing the image taken by the camera from the camera and by evaluating the contrast in the image. A machine vision system comprising a processor adapted to determine one or more positions.   56. The machine vision system of claim 55, wherein the ring light emits red light.   56. The machine vision system of claim 55, wherein the ring light emits green light.   The ring light includes a red light emitting element, a blue light emitting element, and a green light emitting element, and the intensity of light emitted from the red, blue, and green light emitting elements can be selectively adjusted, and any number of different 56. The machine vision system of claim 55, wherein a light color can be selected as a color of light emitted by the ring light.   56. The machine vision system of claim 55, wherein the ring light comprises multi-color LEDs, each of the multi-color LEDs being operable to emit red, green, blue, and combinations thereof.   In combination with a container and a frozen sample contained in the container, the camera is arranged to take an image of the frozen sample and the ring light is adapted to emit light that matches the color of the frozen sample 60. A machine vision system according to any one of claims 55 to 59.   The light emitted by the ring light has a first color, the color of the frozen sample has a second color, and the first color is (i) the same as the second color. 61. The machine vision of claim 60, wherein the machine vision is selected from the group consisting of a color and (ii) substantially the same color as the second color compared to one of two adjacent colors in a six-color RGB hue circle. system.   62. The frozen sample of any one of claims 60 to 61, wherein the frozen sample has a color selected from the group consisting of yellow, orange, and red, and the light emitted by the ring light is red. Machine vision system.   63. A machine vision system according to any one of claims 55 to 62, wherein the ring light and camera are arranged such that there is no direct path from the light source in the ring light to the camera.   The camera has a front end for receiving light from an object, the ring light extends further forward than the camera, and the light emitted by the ring light is from a position in front of the camera. 64. The machine vision system of any one of claims 55 to 63, wherein the machine vision system is to be released.   65. The machine vision system according to any one of claims 55 to 64, wherein the ring light includes a housing having an annular groove, and the light source is disposed at a recessed position in the groove. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) moving a camera relative to the first one of the containers and manipulating a robotic system so that the camera is directed at the frozen sample in the first container;
(B) illuminating the frozen sample with a ring light, wherein the ring light comprises a plurality of light sources arranged in an annular pattern, and the camera extends through a central portion of the annular pattern An illuminating step having an axis;
(C) using the camera to take an image of the illuminated frozen sample;
(D) identifying one or more hole candidates in the captured image and determining whether the hole candidate is likely to be an artifact or a true hole in the frozen sample; Evaluating the contrast of the captured image and processing the image;
(E) operating a robotic system to move the camera relative to a second of the containers so that the camera is directed toward the frozen sample in the second container;
(F) repeating steps (b)-(d) on the frozen sample in the second container.   68. The method of claim 66, wherein step (b) comprises illuminating the frozen sample with red light.   68. The method of claim 66, wherein step (b) comprises illuminating the frozen sample with green light.   68. The method of claim 66, wherein step (b) comprises illuminating the frozen sample with light having a color that matches the color of the frozen sample.   68. The method of claim 66, wherein the frozen sample has a color selected from the group consisting of yellow, orange, and red, and step (b) comprises illuminating the frozen sample with red light.   71. A method according to any one of claims 66 to 70, wherein step (c) comprises taking a grayscale image of the illuminated frozen sample. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A camera configured to take a monochromatic image of the container while the container is supported by a platform;
Illumination arranged to illuminate the container and the sample contained therein while the container is supported on the platform;
Adapted to determine a position where a frozen sample core has already been taken from the sample by receiving grayscale image data representing the image formed by the camera from the camera and evaluating the contrast in the image With a processor,
The illumination emits light having a color other than white;
Machine vision system.   The machine vision system of claim 72, wherein the camera is configured to capture a grayscale image.   74. A machine vision system according to claim 72 or 73, wherein the illumination emits red light.   74. A machine vision system according to claim 72 or 73, wherein the illumination emits green light.   The illumination includes a red light emitting element, a blue light emitting element, and a green light emitting element, and the intensity of light emitted by the red, blue, and green light emitting elements can be selectively adjusted, and any number of different 74. A machine vision system according to claim 72 or 73, wherein the color of light can be selected as the color of light emitted by the illumination. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) moving a camera relative to the first one of the containers and manipulating a robotic system so that the camera is directed at the frozen sample in the first container;
(B) illuminating the frozen sample with colored light;
(C) using the camera to take a grayscale image of the illuminated frozen sample;
(D) identifying one or more hole candidates in the captured image and determining whether the hole candidate is likely to be an artifact or a true hole in the frozen sample; Evaluating the contrast of the captured image and processing the image;
(E) operating the robot system to move the camera relative to a second of the containers so that the camera is directed to the frozen sample in the second container;
(F) repeating steps (b)-(d) on the frozen sample in the second container.   78. The method of claim 77, wherein step (b) comprises illuminating the frozen sample with red light.   78. The method of claim 77, wherein step (b) comprises illuminating the frozen sample with green light.   78. The method of claim 77, wherein step (b) comprises illuminating the frozen sample with light having a color that matches the color of the frozen sample.   78. The method of claim 77, wherein the frozen sample has a color selected from the group consisting of yellow, orange, and red, and step (b) comprises illuminating the frozen sample with red light. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A camera for taking an image of the container while the container is supported by a platform;
Illumination arranged to illuminate the container and the sample contained therein while the container is on the platform, the illumination comprising a red light emitting element, a blue light emitting element, and a green light emitting element The intensity of light emitted by the red, blue, and green light emitting elements is selectively adjustable, and any plurality of different light colors can be selected as the color of light emitted by the illumination To enable, with lighting,
Adapted to determine a position where a frozen sample core has already been taken from the sample by receiving grayscale image data representing the image formed by the camera from the camera and evaluating the contrast in the image With a processor,
The processor is adapted to receive input relating to the color of the sample in the container and to adjust the color of the light emitted by the illumination, so that the color of the sample and the light emitted by the illumination are Reduce the difference between colors,
Machine vision system.   82. The machine vision system of claim 81, wherein the system further comprises a user interface adapted to allow a user to input information regarding the color of the sample.   The camera is a color camera and the processor is adapted to determine the color of the sample using information in the image data received from the camera and automatically adjust the color of the illumination 82. A machine vision system according to claim 81. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) moving a camera relative to the first one of the containers and manipulating a robotic system so that the camera is directed at the frozen sample in the first container;
(B) illuminating the frozen sample with colored light, the illuminating step wherein the color of the illumination is selected to match the color of the frozen sample;
(C) using the camera to take an image of the illuminated frozen sample;
(D) identifying one or more hole candidates in the captured image and determining whether the hole candidate is likely to be an artifact or a true hole in the frozen sample; Evaluating the contrast of the captured image and processing the image;
(E) operating a robotic system to move the camera relative to a second of the containers so that the camera is directed toward the frozen sample in the second container;
(F) repeating steps (b)-(d) on the frozen sample in the second container.   86. The method of claim 85, wherein step (b) comprises illuminating the frozen sample with red light.   86. The method of claim 85, wherein step (b) comprises illuminating the frozen sample with green light.   86. The method of claim 85, wherein the frozen sample has a color selected from the group consisting of yellow, orange, and red, and step (b) comprises illuminating the frozen sample with red light. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A platform for supporting the container, the platform having a mechanism for receiving one of the containers while a frozen sample core is withdrawn from a frozen sample contained in the container; Platform,
A camera for taking images of the containers on the platform while they are received by the mechanism;
Illumination arranged to illuminate the container from a location that provides at least one of backlighting and sidelighting.
Machine vision system.   A frozen sample core is obtained from the sample by receiving image data representing the image formed by the camera from the camera and evaluating how much light passes through the container at various positions shown in the image. 94. The machine vision system of claim 90, further comprising a processor adapted to determine an already sampled position.   91. A machine vision system according to any one of claims 89 to 90, wherein the illumination is arranged to illuminate the container from a location that provides backlighting.   91. The machine vision system of any one of claims 89-90, wherein the illumination is arranged to illuminate the container from a location that provides side lighting.   93. The machine vision system according to any one of claims 89 to 92, wherein the illumination comprises a fiber optic cable.   94. A machine vision system according to any one of claims 89 to 93, comprising a second illumination, wherein the second illumination is arranged to provide bright field illumination to the container.   95. The machine vision system of claim 94, wherein the second illumination comprises a ring light that is coaxial with the camera. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) while placing a frozen sample core from the frozen sample contained in the container to place one of the containers in a mechanism for receiving the container on a platform; Steps to operate;
(B) using illumination to provide at least one of backlighting and sidelighting to the container;
(C) using a camera to take an image of the frozen sample while the illumination is illuminated;
(D) evaluating the contrast of the captured image and processing the image to identify one or more hole candidates in the captured image. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A camera for taking images of the container while the container is supported in a mechanism for receiving the container by a platform while a frozen sample core is withdrawn from the frozen sample contained therein;
Red light for illuminating the container from above with a substantially monochromatic red light while it is in the mechanism on the platform;
A frozen sample core has already been collected from the sample contained in the container by receiving image data representing the image taken by the camera from the camera and by evaluating the contrast in the image. A machine vision system comprising a processor adapted to determine one or more positions. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) while placing a frozen sample core from the frozen sample contained in the container to place one of the containers in a mechanism for receiving the container on a platform; Steps to operate;
(B) illuminating the container from above with a substantially monochromatic red light while it is in the mechanism on the platform;
(C) using a camera to take an image of the frozen sample while illuminated by the red light;
(D) evaluating the contrast of the captured image and processing the image to identify one or more hole candidates in the captured image. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the machine vision system comprising:
A platform for supporting one or more of the containers, the platform having a mechanism for receiving at least one of the containers;
A camera for taking an image of the container while the container is received by the mechanism;
(A) evaluating the contrast in the image to identify one or more hole candidates and identifying an edge of the container; and (b) the edge of the edge relative to the hole candidate. An image representing the image of the container by determining whether the one or more candidates are likely to be an artifact rather than a real hole in the sample using information about the location A machine vision comprising: a processor configured to receive data from the camera and to determine one or more positions where a frozen sample core has already been taken from a frozen sample contained in the container system. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) while placing a frozen sample core from the frozen sample contained in the container to place one of the containers in a mechanism for receiving the container on a platform; Steps to operate;
(B) using a camera to take an image of the frozen sample;
(C) identifying one or more hole candidates and evaluating the contrast of the captured image to identify the edge of the container;
(D) information regarding the position of the edge relative to the hole candidate to determine whether the one or more candidates are likely to be artifacts rather than real holes in the sample; Using the method. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the machine vision system comprising:
A platform for supporting one or more of the containers, the platform having a mechanism for receiving at least one of the containers;
A camera for taking an image of the container while the container is received by the mechanism;
A filling level detection system adapted to detect the position of the surface of the frozen sample;
A machine vision system comprising: a processor configured to receive the signal from the fill level detection system and to use the signal to determine where to position the camera to acquire an image of the frozen sample. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
Using a fill level detection system to measure the position of the surface of the frozen sample spaced from the bottom of the container;
Using information from the fill level detection system to determine where to position the camera such that the camera has a predetermined position relative to the surface of the sample, and moving the camera to that position; ,
Taking an image of the frozen sample in the container from that position;
Using the image to identify the location of one or more holes in the sample. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores, each from a frozen sample contained in a respective container, the machine vision system comprising:
A platform for supporting one or more of the containers, the platform having a mechanism for receiving at least one of the containers;
A coring probe for collecting a frozen sample core from the frozen sample;
A camera for taking an image of the container while the container is received by the mechanism;
Configured to receive image data representative of the image of the container from the camera and to determine one or more positions where a frozen sample core has already been collected from a frozen sample contained in the container A processor configured to move the coring probe into an open end of at least one hole to remove debris from the open end of the hole. Machine vision system. A method for collecting a frozen sample core from a frozen sample contained in a container, the method comprising:
Placing the container in a mechanism for receiving the container on the platform;
Taking an image of the container while the container is received by the mechanism;
Determining one or more positions where a frozen sample core has already been collected from the frozen sample contained in the container;
Collecting the frozen sample core from the frozen sample at a position where the frozen sample core has not yet been collected, as determined in the determining step;
After taking the frozen sample core from the frozen sample, the one or two where the frozen sample core was taken to remove debris from one or more locations from which the frozen sample core was taken. A method of collecting a frozen sample core from a frozen sample stored in a container, comprising the step of inserting a coring probe into the position described above. A machine vision system for use with a robotic system for collecting a plurality of frozen sample cores from a frozen sample each contained in a container, the machine vision system comprising:
A camera for taking images of the container while the container is supported by the platform;
Illumination illuminating the container on the platform, wherein most of the energy of the light emitted by the illumination is red light with a wavelength in the range of 620 nm to 750 nm and green light with a wavelength in the range of 495 nm to 570 nm Lighting selected from the group consisting of
One or two frozen sample cores already taken from the sample contained in the container so as to receive image data representing the image taken by the camera from the camera and by evaluating the image A machine vision system comprising a processor adapted to determine one or more locations.   106. The machine vision system of claim 105, wherein the illumination emits red light.   106. The machine vision system of claim 105, wherein the illumination emits green light.   108. Any one of claims 105-107, wherein the illumination is a ring light comprising a plurality of light sources arranged in an annular pattern, and the camera has an optical axis extending through a central portion of the annular pattern. Machine vision system as described in   109. The method of any one of claims 105-108, wherein by evaluating a contrast in the image, the processor is adapted to determine a location from which the frozen sample core has already been taken. Machine vision system.   By evaluating how much light passes through the container at various positions shown in the image, the processor is adapted to determine from the frozen sample a position where a frozen sample core has already been taken. The machine vision system according to any one of claims 105 to 109.   111. The machine vision system of any one of claims 105-110, wherein the illumination is arranged to illuminate the container from a location that provides at least one of direct illumination and indirect illumination.   The camera is configured to take a monochromatic image of the container while the container is supported by the platform, and the processor receives grayscale image data representing the image formed by the camera from the camera. 112. Any one of claims 105-111, adapted to determine one or more locations where a frozen sample core has already been taken from the sample by evaluating contrast in the image. Machine vision system as described in the section.   113. The machine vision system of claim 112, wherein the camera is configured to take a grayscale image.   In combination with a container and a frozen sample contained in the container, the camera is arranged to take an image of the frozen sample and the illumination is adapted to emit light that matches the color of the frozen sample 114. A machine vision system according to any one of claims 105 to 113. A method of determining one or more locations from which frozen sample cores have already been collected from a frozen sample, wherein each of said frozen samples is contained in a respective container, said method comprising:
(A) moving a camera relative to the first one of the containers and manipulating a robotic system so that the camera is directed at the frozen sample in the first container;
(B) illuminating the frozen sample with illumination, wherein most of the energy of the light emitted by the illumination is red light with a wavelength in the range of 620 nm to 750 nm and a wavelength in the range of 495 nm to 570 nm; Illuminating the frozen sample selected from the group consisting of green light;
(C) using the camera to take an image of the illuminated frozen sample;
(D) identifying one or more hole candidates in the captured image and determining whether the hole candidate is likely to be an artifact or a true hole in the frozen sample; Using an image;
(E) operating a robotic system to move the camera relative to a second of the containers so that the camera is directed toward the frozen sample in the second container;
(F) repeating steps (b)-(d) on the frozen sample in the second container.   116. The method of claim 115, wherein step (b) comprises illuminating the frozen sample with red light.   The method of claim 15, wherein step (b) comprises illuminating the frozen sample with green light.   118. The method of any one of claims 115-117, wherein step (b) comprises illuminating the frozen sample with light having a color that matches the color of the frozen sample.   119. The method of any one of claims 115-118, wherein step (c) comprises taking a grayscale image of the illuminated frozen sample.   120. The method of any one of claims 115-119, further comprising illuminating the container with at least one of ultraviolet light and infrared light.