EP2188588A1 - Computer controllable led light source for device for inspecting microscopic objects - Google Patents
Computer controllable led light source for device for inspecting microscopic objectsInfo
- Publication number
- EP2188588A1 EP2188588A1 EP08796485A EP08796485A EP2188588A1 EP 2188588 A1 EP2188588 A1 EP 2188588A1 EP 08796485 A EP08796485 A EP 08796485A EP 08796485 A EP08796485 A EP 08796485A EP 2188588 A1 EP2188588 A1 EP 2188588A1
- Authority
- EP
- European Patent Office
- Prior art keywords
- leds
- lens
- computer
- micro
- camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/17—Systems in which incident light is modified in accordance with the properties of the material investigated
- G01N21/25—Colour; Spectral properties, i.e. comparison of effect of material on the light at two or more different wavelengths or wavelength bands
- G01N21/255—Details, e.g. use of specially adapted sources, lighting or optical systems
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/645—Specially adapted constructive features of fluorimeters
- G01N21/6456—Spatial resolved fluorescence measurements; Imaging
- G01N21/6458—Fluorescence microscopy
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
- G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
- G01N21/64—Fluorescence; Phosphorescence
- G01N21/6486—Measuring fluorescence of biological material, e.g. DNA, RNA, cells
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B01—PHYSICAL OR CHEMICAL PROCESSES OR APPARATUS IN GENERAL
- B01L—CHEMICAL OR PHYSICAL LABORATORY APPARATUS FOR GENERAL USE
- B01L3/00—Containers or dishes for laboratory use, e.g. laboratory glassware; Droppers
- B01L3/06—Crystallising dishes
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B01—PHYSICAL OR CHEMICAL PROCESSES OR APPARATUS IN GENERAL
- B01L—CHEMICAL OR PHYSICAL LABORATORY APPARATUS FOR GENERAL USE
- B01L3/00—Containers or dishes for laboratory use, e.g. laboratory glassware; Droppers
- B01L3/50—Containers for the purpose of retaining a material to be analysed, e.g. test tubes
- B01L3/508—Containers for the purpose of retaining a material to be analysed, e.g. test tubes rigid containers not provided for above
- B01L3/5085—Containers for the purpose of retaining a material to be analysed, e.g. test tubes rigid containers not provided for above for multiple samples, e.g. microtitration plates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N35/00—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor
- G01N35/02—Automatic analysis not limited to methods or materials provided for in any single one of groups G01N1/00 - G01N33/00; Handling materials therefor using a plurality of sample containers moved by a conveyor system past one or more treatment or analysis stations
- G01N35/04—Details of the conveyor system
- G01N2035/0474—Details of actuating means for conveyors or pipettes
- G01N2035/0491—Position sensing, encoding; closed-loop control
- G01N2035/0493—Locating samples; identifying different tube sizes
Definitions
- the present invention relates to automated inspection devices, and in particular to automated inspection devices having computer controllable light sources.
- the specifications of U.S. provisional application serial no. 60/961,722, filed 07/23/2007; U.S. provisional application serial no. 60/997,839 filed 10/01/2007; US patent application Serial No. 10/627,386 filed July 25, 2003 (soon to issue as U.S. Patent No. 7,406,189 on 07/29/2008); and US Patent No. 6,985,616 which issued on 01/10/2006, are incorporated by reference herein.
- the determination of the three dimensional atomic structure of matter is one of the most important areas of pure and applied research.
- One way in which the three dimensional atomic structure of matter can be determined is through X-ray crystallography.
- X-ray crystallography utilizes the diffraction of X-rays from crystals in order to determine the precise arrangement of atoms within the crystal. The result may reveal the atomic structure of substances such as metal alloys, deoxyribonucleic acid (DNA), or the structure of proteins.
- Protein crystals are commonly grown in the wells of micro-well plates.
- a micro-well plate is also known as a micro-titer plate or a microplate.
- Micro-well plates typically come with either 24, 48, 96, 384 or 1536 wells.
- a 96-well micro-well plate is shown in detail in FIG. 2.
- One of the main techniques available for growing crystals is a method wherein a drop of a solution containing protein is applied to a glass cover slip and placed upside down in an apparatus such as a vapor diffusion chamber where conditions lead to supersaturation in the protein drop and the initiation of precipitation of the protein crystal.
- Another method is the sitting drop method where the drop sits in a small well adjacent the growing solution instead of hanging over it. This method provides a more stable drop and location.
- Another method is the aqueous drop in oil method.
- the drop is placed in a micro- well and is covered with an oil based solution.
- the drop stays at the bottom of the well as the crystal grows.
- the protein solution is contained within a semi-permeable size exclusion membrane and then placed in a solution of fixed pH and precipitant concentration. As the precipitant diffuses through the membrane into the protein compartment, the solubility of the protein is reduced and crystals may form.
- This method involves the placement of a gel into the end of small diameter glass capillaries. After the solutions have gelled, a protein solution is placed into one end (top) of the capillary and the other end is submerged in a solution of precipitating agent. If the conditions are appropriately selected, crystal growth occurs at a point in the gel where the protein and precipitating agent reach the proper concentrations as the solutions slowly mix by diffusion. Since this is a diffusion limited process, it thus only occurs after an extended period of time. Crystals however, grown by this method are often larger and of higher quality. Regardless of the method chosen, protein crystal growth is a very delicate and time- consuming process. It can take several days to several months before crystals of sufficient size and quality are grown and ready for x-ray crystallography.
- the current minimum size that is typically stated is a crystal of at least 50 microns thick by 100 microns in extent.
- the protein crystal growing environmental conditions need to be rigorously maintained, from the chemistry, to the surrounding air humidity and temperature, cleanliness to prevent contamination, and even lighting conditions.
- a protein crystallographer working with unknown protein families may only be about 5% successful in growing proper sized quality crystals. With this success rate, for example, a 96-well micro-well plate may only have 5 wells in which good crystals are growing.
- a laboratory technician or operator, aided by a microscope and a laboratory notebook manually inspects crystals grown in micro-well plates.
- a laboratory technician dons a clean-room gown suit and enters a cold room in which the crystals are growing.
- the technician then puts a micro-well plate underneath the microscope and examines each well in the micro-well plate until all of the wells in the micro-well plate have been inspected.
- the technician then makes a mental judgement as to how he shall classify (also known as "score") the crystal. For example, the technician may feel that he is observing an image that shows "grainy precipitation" or "ugly precipitation". Or, he may feel that the image shows "no crystal growth".
- the technician then records the classification into a laboratory notebook.
- the above system is riddled with opportunities for human error.
- An operator manually inspecting a 96-well micro-well plate will take approximately 5 to 20 minutes depending on the skill of the operator and the number of wells that contain interesting features, microcrystals, or crystals.
- the operator may be subject to physical fatigue, suffer eyestrain, and may be uncomfortably cold in the temperature controlled and generally high humidity room.
- the operator can be tired and confused and can easily make errors in manually recording data in the notebook. For example, the operator may observe crystal growth at well H5 (FIG. 2), but incorrectly record in the notebook that the crystal growth was at well H6. Additional transcription errors may occur when the data is transferred to a computer database.
- Typical prior art techniques for protein crystal growth include a camera as a part of the robotic system used for monitoring the growth process. In most cases this camera views the growth through a microscope and in most cases the camera is a visible light camera and the proteins are identified by their crystalline shape as they develop.
- One of the problems with these prior art techniques is that salt crystals may also form in the hanging or sitting drops and these salt crystals typically cannot be distinguished from protein crystals with the visible light cameras.
- amino acids There are 20 amino acids that have been identified as the building blocks of proteins. Of these 20 amino, three are known to fluoresce when illuminated with light at a particular wavelength. These are: (1) Tryptophan which fluoresces at wavelengths of about 348 nm when illuminated at wavelengths of about 280 nm, (2) Tyrosine which fluoresces at wavelengths of about 303 nm when illuminated at wavelengths of about 274 nm and (3) Phenylalanine which fluoresces at wavelengths of about 257 nm when illuminated at wavelengths of about 282 nm.
- Salt crystals do not fluoresce when illuminated with ultraviolet light in the range of 257 nm to 282 nm. For this reason attempts have been made to monitor protein crystal growth using microscopes with ultraviolet illumination.
- Prior art techniques have used mercury vapor lamps and deuterium lamps as light sources. A known technique is to direct light from these lamps into optical fibers that then carry the light to the microscope where it is directed to the drop being examined. These lights sources are typically rather broad banned. Therefore either very precise filtering is required and even with careful filtering there is a risk of over heating the target drop by the ultraviolet light.
- ACTORTM is a trademark owned by the Rigaku Americas Corporation with offices in The Woodlands, TX.
- ACTOR refers to a robotic system for Automated Crystal Transfer, Orientation and Retrieval of crystal samples. This system is the winner of the 2002 R&D 100 Award for technical innovation, and is the world's first commercial robotic system for automated crystal sample handling.
- ACTOR eliminates much of the physical handling of protein samples by crystallographers required during routine crystal screening and data collection.
- the high-throughput ACTOR system provides automatic sample transport, orientation, and retrieval at synchrotron beam lines and in home laboratories. Changing samples as soon as data have been collected also maximizes the use of the X-ray source.
- ACTOR can collect data unattended 24 hours a day, seven days a week.
- the ACTOR system stores up to 60 cryogenically frozen samples held in loops on commercially available pins. Magnetic magazines hold the pins in mapped positions for robotic retrieval. The magazines sit in the insulated ACTOR staging dewar, which is automatically filled with liquid nitrogen.
- ACTOR is installed at numerous laboratories and beam lines around the world. ACTOR, with its associated software and tools, is a complete system designed to increase productivity and allow for unattended sample analysis for the high throughput crystallography labs of today. These systems are commercially available from Rigaku Americas Corporation with several sales offices located in several states in the United States.
- the pins referred to in the above paragraph typically are Hampton Pins.
- a drawing of one of these pins is shown in FIGS. 59 and 60.
- the pin is comprised of an 18 mm pin mounted in a frame that is easily positioned in one of the magnetic magazines referred to above.
- At the end of the pin at position 22 beam position as shown in FIG. 60 is a loop made of horse hair.
- a technician while viewing through a microscope, dips the loop into a solution containing protein crystals. Some of the solution adheres to the horse hair loop.
- the technician puts a cap on the pin and transports the capped pin to the ACTOR and inserts it in one of the magnetic magazines. There the crystal containing solution collected on the loop is frozen for storage to await x-ray analysis.
- the ACTOR is designed to pick up the pin and orient it for analysis.
- the protein crystals occupy only a relatively small region within or attached to the relatively much larger loop.
- the prior art ACTOR contains a visible light microscope for orienting the loop and for identifying the protein crystals.
- the position of the protein crystals to be examined must be precisely located so that the x-ray beam can be pointed directly at it for analysis. Often the crystals are difficult to precisely locate with the visible light.
- the present invention provides a device for inspecting microscopic objects.
- a plurality of LEDS is arranged in an array underneath a lens. Some of the LEDS are lighted and some of the LEDS are unlighted.
- a computer is in control of the LED array. The computer turns on selected LEDS from the array to form the lighted LEDS. Also, the computer turns off selected LEDS from the array to form the unlighted LEDS.
- the lighted LEDS form a pattern of lighted LEDS underneath the lens.
- the lens is connected to a computer controlled camera and the microscopic objects are microscopic crystals.
- UV LEDS are utilized and illuminate crystals from above.
- UV LEDS are utilized to illuminate a loop of a Hampton pin to locate a crystal in the loop of a Hampton pin for the purpose of x-ray crystallography.
- FIG. 1 shows a preferred embodiment of the present invention.
- FIG. 2 shows a micro-well plate.
- FIG. 3 shows a top view of the fixture plate.
- FIGS. 4 and 5 show top views of micro- well plates on the fixture plate.
- FIG. 6 shows a block diagram of a preferred embodiment of the present invention.
- FIG. 7 shows a preferred monitor.
- FIGS. 8 - 10 and 18 - 25 show steps in the sequence of operations of a preferred embodiment of the present invention.
- FIG. 11 shows hanging drops of liquid in a micro-well plate.
- FIG. 12 shows an example of aqueous drop in oil protein crystallization.
- FIG. 13 shows a top view of a micro- well plate on the fixture plate.
- FIG. 14 shows a side view of the light source shining upwards onto a micro- well plate.
- FIG. 15 shows a magnified view of two wells of a micro-well plate, wherein each well has a drop of liquid.
- FIGS. 16 and 17 show a detail view of the drops of liquid shown in FIG. 15.
- FIG. 26 shows a preferred monitor screen after a run has been completed.
- FIGS. 27 and 28 show details of other preferred monitor screens.
- FIG. 29 shows a hanging drop of liquid with crystal growth.
- FIG. 30 shows a preferred embodiment of the present invention.
- FIG. 31 shows a flowchart of an auto-focus subroutine of the present invention.
- FIG 32 shows a flowchart of a focus value subroutine.
- FIG. 33 shows a flowchart of the auto score and classify subroutine.
- FIGS. 34a - 34d show flowcharts of the classify subroutine.
- FIG. 35a -35b show the sub-classification of the crystal class.
- FIG. 36 shows the main program flow.
- FIG. 37 illustrates a side view illustrating dual filters in the light path.
- FIG. 38 illustrates a top view of the drive mechanism for the rotatable linear polarized filter.
- FIG. 39 illustrates a top view of a second filter wheel.
- FIG. 40 shows the connectivity of another preferred embodiment.
- FIG. 41 shows an exploded view of another preferred embodiment of the present invention.
- FIG. 42 shows a perspective view of the preferred embodiment shown in FIG. 41.
- FIG. 43 shows a top view of an LED array.
- FIG. 44 shows another preferred embodiment of the present invention.
- FIGS. 45 - 50B illustrate the operation of the preferred embodiment shown in FIG.
- FIGS. 51 A - 5 IB show another preferred embodiment of the present invention.
- FIGS. 52 A - 53B illustrate the operation of the preferred embodiment shown in FIGS.
- FIGS. 54 A - 54D show another preferred embodiment of the present invention.
- FIGS. 55 A - 55D show preferred patterns.
- FIGS. 56A - 57B show photographs demonstrating the clarity of LED illumination.
- FIG. 58 shows another preferred embodiment of the present invention.
- FIGS. 59 and 60 show the Hampton Pin.
- FIG. 61 shows another preferred embodiment of the present invention.
- FIGS. 62 A and 62B show photographs demonstrating the clarity of UV LED illumination.
- FIG. 63 shows another preferred embodiment of the present invention.
- FIGS. 64 - 69 show photographs demonstrating the clarity of UV LED illumination.
- FIGS. 70 - 74 show another preferred embodiment of the present invention.
- FIGS. 75A - 75F show another preferred embodiment of the present invention.
- FIG. 1 shows a preferred embodiment of the present invention.
- Micro-well plates 125A - 125F are placed on fixture plate 129.
- each micro- well plate has 96 wells.
- Each well has a drop of liquid in which microscopic protein crystals may be growing.
- Computer 105 automatically controls linear actuators 115, 150 and 160.
- Linear actuator 115 moves fixture plate 129 along the x-axis.
- Linear actuator 150 moves moving base 154 along the y-axis, and linear actuator 160 moves moving plate 162 along the z-axis.
- the computer coordinates the movement of the linear actuators to properly position cameras 155 and 135 above each well of each micro-well plate 125A - 125F in sequence.
- cameras 155 and 135 are high-resolution 2/3 inch CCD cameras, with 1,300 horizontal pixel elements by 1,030 vertical elements. Each element is preferably 6.7 microns square with a sensing area of 8.7 mm by 6.9 mm. Cameras 155 and 135 take images of each well and transmit the images to computer 105 where they are digitized into a image data array of approximately 1,296 horizontal pixels by 1,000 vertical pixels by 8 bits of gray level representing the intensity (0 to 255) of each pixel. These digitized images are automatically recorded in the database of computer 105 and can be analyzed and scored by an operator via monitor 620.
- the digitized images may be further processed, analyzed, and the contents of the individual well scored by computer 105 executing program instructions to perform calculations on the image data array.
- computer 105 is connected via a communication/control line to a computer network. In this manner, the present invention can be controlled from a remote computer. Likewise, images and data can be transmitted to the remote computer.
- FIG. 1 As shown in FIG. 1, six 96-well micro-well plates 125 A - 125F have been placed (either by an operator or by an external loading robot) onto fixture plate 129.
- a 96 well micro-well plate 125A is shown in FIG. 2.
- Micro-well plate 125A has wells labeled Al through H12 and a bar code label 220.
- Micro-well plate 125A is available from Nalge-Nunc International, with U.S. offices in Rochester, NY.
- Fixture plate 129 will hold 24, 48, 96, 384, or 1536 well micro-well plates since the micro-well plate external width and length are fairly standard in the industry.
- the 96 well micro-well plate will be used to illustrate the present invention.
- FIG. 3 shows a top view of fixture plate 129 just prior to loading micro-well plates 125 A - 125F.
- Fixture plate 129 has six cutout sections 131 that are just slightly smaller in length and width than micro-well plates 125A - 125F.
- FIG. 4 shows a top view of micro-well plates 125A - 125F immediately after they have been placed on fixture plate 129.
- micro-well plates 125A - 125F onto fixture plate 129 as shown in FIG. 4, he enters the command into computer 105 (FIG. 1 and FIG. 6) to expand detents 510 and 520 (FIG. 5). The expansion of detents 510 and 520 firmly secures micro-well plates 125A - 125F against plate stops 530.
- FIG. 7 shows a display representing micro-well plates 125A - 125F on the screen of monitor 620.
- the operator has mouse clicked on bars 622A - 622F, which has caused them to turn green.
- bars 622A - 622F By clicking on bars 622A - 622F, the operator has selected corresponding micro-well plates 125A - 125F to "run".
- the operator sends the command to run the selected micro- well plates by clicking on run bar 623.
- the operator has given the command to run the selected micro-well plates.
- Micro-well plate 125 A has been moved to a position underneath cameras 155 and 135.
- plate sensor transmitter/receiver 186 and reflector 188 By the utilization of plate sensor transmitter/receiver 186 and reflector 188, information is sent to computer 105 reporting that micro-well plate 125A is in position underneath the cameras.
- Plate sensor transmitter/receiver 186 is fixed to support 189 and is aligned to sense whenever a micro- well plate breaks a beam of light emitted by transmitter/receiver 186 and reflected by a reflector 188.
- Reflector 188 is mounted on support 191 on the opposite side of the linear actuator 115.
- the plate sensor transmitter/receiver 186 and reflector 188 are preferably model # E3T- SR13 available from Western Switch Controls of Santa Ana, CA.
- Bar code reader 175 is also mounted to support 189 and is positioned to view barcode identity label 220 (FIG. 2) attached to micro-well plate 125A when it is positioned underneath cameras 155 and 135.
- Bar-code reader 175 is preferably model # BL601 available from Keyence Corporation of America of Newark, NJ. Bar-code reader 175 communicates with computer 105 via a communication line.
- Information encoded into label 220 preferably includes: the plate serial number, the plate type (i.e., 24-well, 48-well, 96-well, 384-well, or 1536-well micro-well plate), and the well type (i.e., square, or rounded, hanging drop, sitting drop, constrained sitting drop).
- plate sensor transmitter/receiver 186 and bar-code reader 175 is transmitted to computer 105 and stored for later use during the camera inspection and information acquisition phase.
- linear actuator 115 has moved fixture plate 129 so that micro-well plate 125B is underneath cameras 155 and 135.
- information is transmitted from plate sensor transmitter/receiver 186 and bar-code reader 175 to computer 105 and stored for later use during the camera inspection and information acquisition phase.
- micro-well plates 125A - 125F have been sensed and recorded by plate sensor transmitter/receiver 186 and bar-code reader
- linear actuator 115 moves micro-well plate 125A so that it is underneath lens 165 of camera 155.
- Motor 130 of linear actuator 160 moves moving plate 162 upward and/or downward as necessary to properly focus lens 165 on the drop of hanging liquid over well Al.
- lens 165 is set at a predetermined zoom.
- FIG. 11 shows a cross section side view of wells Al - El of micro-well plate 125 A.
- an attempt has been made to grow protein crystals in the hanging drops in each of the wells of micro-well plate 125A.
- FIG. 11 shows hanging drops ⁇ l, ⁇ l, ⁇ l, ⁇ l, and ⁇ l.
- the preferred method for protein crystal growth is the hanging drop method.
- the hanging drop method also known as vapor diffusion
- a drop of protein solution is suspended over a reservoir containing buffer and precipitant. Water diffuses from the drop to the solution leaving the drop with optimal crystal growth conditions.
- lens 165 of camera 155 is over well Al of micro-well plate 125A (FIG. 2, FIG. 11, and FIG. 13).
- FIG. 13 shows a top view of micro-well plate 125A positioned on fixture plate 129.
- FIG. 14 shows a side view of micro-well plate 125A positioned on fixture plate 129.
- Support 191 with embedded light source 194 is positioned to the side of fixture plate 129.
- Light from light guide 195 is directed upward through cutout 131 (also shown in FIG. 3).
- Light guide 195 is positioned between fixture plate 129 and plate 127 such that both plates can move around the light guide 195 without interference.
- fixture plate 129 has cutouts 131 (FIG. 3) that are smaller than the micro-well plates 125 and located under each well plate, such that light from light guide 195 can be projected through the well plates when they are brought into position for inspection.
- light source 194 is model # A08925 fiber-optic backlight available from Aegis Electronics of Carlsbad, CA.
- Camera 155 (FIG. 10) inspects well Al and transmits an image to computer 105 for digitization.
- camera 155 preferably (FIG. 10) inspects well Al at a IX magnification so that every 6.7 micron square pixel represents approximately 6.7 square microns on the object being measured, allowing for some small geometric distortions caused by the lens 165.
- Computer 105 has been programmed to digitize the camera image and then by utilizing vision software determines a position within well Al for the drop of liquid hanging from grease seal 361. The position of the drop of liquid is recorded for later use onto the hard drive of computer 105 and to a memory location within the computer.
- the vision software used to determine the position of the drop of liquid uses a software routine algorithm called mvt_blob_find from a collection of image processing tools called MVTools.
- MVTools is available from Coreco Imaging, US Office in Bedford, MA.
- linear actuator 115 moves fixture plate slightly to the left so that lens 165 is over well Bl (FIG. 13).
- the position of the drop of liquid hanging from grease seal 361 in well Bl is recorded on the hard drive of computer 105 and in computer memory.
- computer 105 will record that drop of liquid ⁇ l is towards the upper left-hand quadrant of well Al.
- the position of drop of liquid ⁇ l is recorded onto the database of computer 105 as being in the lower right-hand quadrant of well Bl.
- linear actuator 115 has moved fixture plate 129 so that well Hl is under lens 165.
- linear actuator 115 has moved fixture plate 129 to the left and linear actuator 150 has moved moving base 154 slightly rearward so that lens 165 is over well A2 of micro-well plate 125 A (FIG. 13).
- linear actuator 115 moves fixture plate 129 and linear actuator 150 moves moving base 154 so that cell Al of micro-well plate 125B is underneath lens 165 (FIG. 21).
- An operation to inspect each hanging drop is performed at a higher magnification using camera 135 with its zoom lens 145 capable of magnifications of 2.5X to 1OX corresponding approximately to digitized pixels representing 2.68 microns square ( at 2.5X) to 0.67 microns square (at 10X). This inspection is done for the purpose of determining whether protein crystals have grown.
- Zoom motor 192 controls the degree of zoom for zoom lens 145.
- computer 105 (FIG. 1) automatically transmits a signal to linear actuators 115 and 150 to position lens 145 directly over the drop of liquid within each well. For example, in FIG. 23 lens 145 is positioned over the top of well Al of micro-well plate 125A.
- lens 145 is precisely positioned so that it is able to zoom in on drop of liquid ocl (FIG. 13).
- FIG. 16 shows a magnified view of drop of liquid ocl.
- motor 130 of linear actuator 160 has moved moving plate 162 upward and/or downward as necessary to properly focus lens 165 on drop of liquid ocl.
- Zoom motor 192 has manipulated lens 165 to obtain the desired degree of zoom.
- Camera 135 inspects well Al and transmits a signal representing the magnified image of the hanging drop of liquid to computer 105. The images are stored on computer 105 temporarily in memory for immediate analysis and on hard disk for later analysis.
- linear actuators 115, 150 and 160 and zoom motor 192 operator to properly position and magnify zoom lens 145 over each hanging drop of liquid to obtain desired focus and magnification for image data storage.
- FIG. 17 shows a magnified view of hanging drop of liquid ⁇ l.
- micro-well plates 125B - 125F so that magnified images of the hanging drops of liquids are recorded for each cell Al - H12 for micro- well plates 125B - 125F.
- the sequence has ended for micro-well plates 125A - 125F.
- Linear actuator 115 has moved fixture plate 129 and linear actuator 150 has moved moving base 154 so that well H12 of micro-well plate 125F is under lens 145.
- monitor 620 will appear as shown in FIG. 26.
- FIG. 26 six images representing micro-well plates 125A - 125F appear on the screen. Above each image is a message "Run Comp” indicating that image data for hanging drops of liquid has been transferred into computer 105. Beneath each image are buttons 710 - 715 marked "S". By mouse clicking on any button 710 - 715, the operator may manually score for successful crystal formation each magnified image of each hanging drop of liquid. For example, in FIG. 26, the operator can mouse click on button 710 to score micro- well plate 125 A.
- FIG. 27 the operator has mouse clicked on the circle representing well Al of micro-well plate 125A. This has caused a magnified image to be displayed of drop of liquid ⁇ l in screen section 716. The operator has concluded that there are no crystals in drop of liquid ⁇ l and has therefore mouse clicked on button 717 for "NO CRYSTAL". On the display screen, this has caused the circle representing well Al of micro-well plate 125 A to turn red.
- FIG. 28 the operator has mouse clicked on the circle representing well Bl of micro-well plate 125A. This has caused a magnified image to be displayed of drop of liquid ⁇ l in screen section 716. The operator has concluded that there are crystals in drop of liquid ⁇ l and has therefore mouse clicked on button 718 for "CRYSTAL". On the display screen, this has caused the circle representing well B 1 of micro-well plate 125A to turn green.
- micro-well plates 125A -125F have all been scored, the operator has at his easy disposal a database that contains the identity of each micro-well plate that was inspected along with a score summarizing whether crystal formation occurred for each well in the micro-well plate.
- the depth of view of camera 135 is approximately 50 to 100 micrometers.
- the crystal in the drop of liquid may be larger than the depth of view or there may be crystals growing at various levels within the hanging drop of liquid, as shown in FIG. 29. Therefore, in the second preferred embodiment, lens 145 is focused at multiple different levels 721 - 724 and a set of images are recorded at the different levels so that the entire crystal may be analyzed.
- the third preferred embodiment of the present invention utilizes a specimen auto- focus subroutine 300 (FIG. 31).
- Subroutine 300 ensures that the specimen within the micro-well is in-focus at the desired zoom (or magnification ratio) of image lens 145.
- the present invention causes camera 135 to take a number of images defined by a Number_of_Z_Slices. Typically, there are between 5 and 10 slices separated in the Z-axis from one another by a Z_Step_Size. The typical step size is 0.05mm to 0.25 mm.
- the slices preferably start at a Z-Axis location defined by a Start_Z value, which is typically at the bottom of the cover-slip on the micro-well plate.
- input data 304 is received.
- the area of interest window within the images is further defined in the input data 304 by a X_Window_Center and a Y_Window_Center plus an X_Width and a Y_Width.
- Initial settings 306 for the routine 300 are the starting value of a counter N, a Best_Focus, a Best_Z, and a Focus_Error.
- the inspection device sets Z_Position(N) equal to the Start_Z location and moves camera 135 there in step 308.
- An image is acquired with camera 135 and digitized as previously described and stored as Image (N) in step 310.
- a second subroutine 312 extracts a focus value F (N) for Image (N) and is further described in the section for discussion for FIG. 32.
- a test is made between F(N) and the Best_Focus in step 316, such that if F(N) is greater than Best_Focus then Best_Focus is set to F(N) and Best_Z is set to N as shown in step 320 and the program flow goes onto step 322, if the test condition is not met in step 316 then the program flow skips step 320 and goes on to step 322.
- a test is made to determine of all of the slices have been taken as N is tested against the Number_of_Z_Slices.
- step 324 If N is equal to Number_of_Z_Slices then program flow goes onto step 324. If more slice images are needed, then the flow goes to step 318. In step 318, Z_Position(N+l ) is set to Z_Position(N) + Z_Step_Size and the Z- Axis is moved to Z_Position(N+l ) and the program flow goes on to step 314 where N is incremented by 1 (one). The program flow goes back to step 310 and completes the loop of step 310 to step 322 until all of the image slices have been taken and then moves onto step 324.
- step 324 Best_Z is tested against its initial value, and if it equals its initial value (meaning no focus was found in the focus value subroutine 312) then it is set to a default value of the Number_of_Z_Slices divided by 2 and Focus_Error is set to 1 (one) in step 326 and the program flow goes onto step 328. If Best_Z in step 324 has a value other than its initial value then program flow goes onto step 328 from step 324. In step 328 a Best_Image image is set to the image slice at best focus by setting Best_Image equal to Image(Best_Z). Also, a Best_Image_Z value is set equal to Z_Position(Best_Z) and the flow goes onto step 330 which is the RETURN part of the subroutine and program flow returns to the main software flow.
- an image(N) focus F(N) subroutine 312 is further detailed, starting at the step Start 401.
- a pointer_to_image(N) 404 is provided in step 402.
- the image(N) is convolved with a standard 3 x 3 Sobel_Horizontal filter 416 to produce an Image(N)_H wherein horizontal edges within the image are emphasized.
- the image(N) is convolved with a standard 3 x 3 SobelJVertical filter 418 to produce an Image(N)_V wherein vertical edges within the image are emphasized.
- both the horizontal edge emphasized image Image(N)_H and the vertical edge emphasized image Image(N)_V are summed pixel by pixel, during the summing process any resulting negative pixel values are set to zero and any resulting pixel values that are greater than 255 are set to 255, to produce an image Image(N)_Focus.
- a simple variance F(N) is calculated for the pixels within a window of interest defined in 422 by X-Window_Center, Y_Window_Center, X_width, and Y_Height. The resulting value of the variance is returned to the calling program as F(N) in step 424.
- the sobel processing and the variance calculation is performed with a collection of image processing software tools within MVTools.
- MVTools is available from Coreco Imaging, US Office in Bedford, MA.
- the inspection device automatically scores and classifies each drop specimen by executing computer software subroutines as shown in FIGS. 33, 34a, 34b, 34c, 34d, and 35a and 35b under control of the program flow shown in FIG 36.
- FIG. 36 illustrates the main program flow 840 starting at step 842.
- the software is initialized with parameters, inspection lists, and Type_of_Classification detailed in step 486.
- the flow continues onto step 844 where the system moves the micro-well of interest within the selected micro- well plate under the selected camera.
- step 851 if the drop needs to be located, the flow continues onto step 849 wherein an image is acquired by the camera and software operates on the image and determines the location of the drop. Then a test is made to determine if the last micro- well in the plate has been imaged, if not then the flow loops to step 844 and continues.
- step 853 the flow continues to step 856 where the system moves to the droplet within micro-well in micro-well plate under the high-resolution camera. Also in step 851 if the drop had been previously located, then the flow would continue from step 844 directly to step 856 without the need to re-locate the drop. From step 856, the flow continues onto step 857 wherein a high-resolution image of the drop is obtained. Then the flow goes onto step 850.
- step 850 a CALL is made to subroutine that automatically scores and classifies the drop depending on the Type_of_Classification required.
- the subroutine is detailed in FIG. 33 and starts at step 440 in FIG. 33.
- step 852 the subroutine returns to step 852 wherein the results are stored and reported.
- the program flow continues to step 858 where a test is made to determine if the last drop in the selected plate has been processed, if so then the flow goes onto step 848 wherein a test is made to determine if the last plate has been processed. If not, the flow loops back to 856 and continues. If the last plate has been processed, the flow goes onto step 854 and the program is done.
- FIG. 33 shows Micro-well Specimen Auto Score and Classify Subroutine 438 starting at step 440.
- Pointer_to_Best_Image 442 provides information to the initialization step 444 that allows access to the image that was found to be the best focus. Plus, the Type_of_Classification is passed into the routine.
- pointer 442 can point to an image that was taken at a z height value known to be the focus of the system.
- the subroutine 438 calculates in step 448 an average_gray_Background value normalized to allow for the variation present from various plate types, by using a first rectangular window defined by parameters shown in step 446 (X_Background _Center, Y_Background _Center, X_Background _Width, and Y_Background Height) and by summing all of the gray scale values of the pixels defined by the window 446 and dividing by the number of pixels within that window.
- the average_gray_background is normalized for well-type differences by multiplying the calculated value by a Micro-well normer value also found in step 446 and generally determined by measuring the various micro-well plate types under inspection and normalizing to the 96-well standard micro- well plate.
- This average_gray_background 448 is calculated in a window area of the image that is outside the area of the drop but generally within the well or within the bounding well walls.
- step 450 an average_gray_Classify_Window value is calculated in a similar manner as described above (except it is not normalized for micro-well type) using a second rectangular window defined by parameters shown in step 452 (i.e., X_Classify _Center, Y_Classify . Center, X_Classify _Width, and Y_Classify Height).
- This average_gray_classify_window value 450 is taken in a rectangular window area of the image that is inside the area of the drop and defined by being a fraction between 0.98 and 0.5 (with 0.8 preferred) of the width and height of the external bounding rectangular box from the blob utilizing subroutine mvt_blob_find.
- the subroutine mvt_blob_find defines the extent of the drop as previously discussed in the section "Determining the Position of the Drop of Liquid within Each Well".
- a diagonal difference image is calculated by stepwise subtracting pixel values from two pixel locations, defined by (x,y) and (x+Diff_Sep, y+Diff_Sep) within the Classify_window.
- the pixel values are separated in width and height by the value Diff_sep from step 452. This is repeated over all pixels within the Classify window defined by step 452 using X_Classify _Center, Y_Classify _Center, X_Classify _Width, and Y_Classify Height. For each value calculated the absolute value is taken of the subtraction result and compared to a threshold value Flag_Thresh defined in step 452.
- the pixel is set at the first location in x,y equal to a value defined by Setl in step 452, if the calculated value is equal to or less than Flag_Thresh, 452, then the pixel value is set to zero.
- Typical values for Diff_Sep are between 1 to 20 pixels with 7 preferred.
- Typical values for Setl are between 1 and 511 with 128 preferred.
- Typical values for Flag_Thresh are between 5 and 50 with 25 preferred.
- Step 456 a calculation similar to that performed in step 454 is performed on the Classify_Window except that the separation between the two pixels undergoing the calculation is defined by (x+Diff_Sep, y) and (x, y+Diff_Sep), as is shown in the mathematical calculation in 456 to generate a Diff_Image_2.
- This calculation uses definitions shown in step 452. Typical values for Set2 are between 1 and 511 with 200 preferred.
- Step 458 the Classify_Image, which is a combination of the images generated in step 454 and 456 is calculated as shown by the mathematical equations shown in step 458 using definitions shown in step 452. If the x,y pixel value in either Diff_Image_l or Diff_Image_2 (steps 454 and 456 respectively) has a value equal to Setl 452 then the pixel value is set at (x,y) in Classify_Image equal to Set2 452. Otherwise, the value is equal to zero(O) as shown in the mathematical equations in step 458. The calculations are repeated for all pixels within the window defined in 452.
- the Classify_Image is basically an image of the classify_image_window wherein edges present within the original Best_Image are detected.
- step 460 the value of number_Pixels_Set2 is set equal to the total number of pixels that are set equal to Set2 452 in step 458. Also, the value of Total_Pixels_in_Window is set to the total number of pixels in the Classify_window in step 458 .
- a Score_Gray is calculated by dividing the Average_Gray_Classify_Window determined in step 450 by the Average_Gray_Background found in step 448.
- a Score_Flag is also calculated by dividing the Number_Pixels_Set2 by Total_Pixels_in_Window from step 460. The Score_Gray and Score_Flag are normalized in this matter.
- step 464 the values of Score_Gray, Score_Flag, and Type_of_Classification are passed to a classify subroutine and a classification is returned for the Classify_image, effectively classifying the protein crystals within the window. Details of the Classify subroutine 464 are provided in FIG 34a, 34b, 34c,and 34d, plus FIG 35a and 35b.
- FIG 34a shows the Classify Subroutine 464. Start of Classify subroutine is shown in step 468 followed by initialization 470 whereby the initial classification CLASS value is set to 5, representing "unknown” and the flow goes onto step 480.
- a test is made to see of the type_of_classification is equal to 1, the first classification type wherein the drop is classified as simply clear (0) or unknown (5) as previously discussed. If the type_of_classification is equal to 1 then the flow goes on to step 483 and then returns to FIG. 33 step 466 with the results. If the type_of_classification is not equal to 1 then the flow goes onto step 476 for further classification.
- Step 476 utilizing threshold value parameters shown in step 478 (Lgt_Precip_Flag_LT, Lgt_Precip_Flag_GT, Lgt_Precip_Gray_GT), assigns the value of 1 to CLASS indicating that Light Precipitation is present in the Classify_Image.
- Step 476 utilizes the mathematical equation which states if Score_Flag is less than Lgt_Precip_Flag_LT and Score_Flag is greater than Lgt_Precip_Flag_GT and Score_Gray is greater than Lgt_Precip_Gray_GT then set CLASS to value 1.
- Step 492 continues the classification process in FIG 34b.
- Step 708 continues the classification process onto FIG 34c as 725.
- Step 726 continues from step 708 of FIG 34b.
- Step 727 calculates and generates a first set of additional image features for further use in classification taking as input 726 (X_Classify _Center, Y_Classify _Center, X_Classify _Width, Y_Classify Height, Pointer_to_classify_image, and mvt_blob_analysis_Params).
- MVT_Tools_Blob_Analysis with pointer to window in Classify_Image is called.
- Step 727 gets the num_found of blobs and, for each blob found, step 727 gets its area(m), height(m), width(m), perimeter(m), and location of each as location_X(m) ands location_Y(m). These values are recorded.
- Classify_Image which is the image that was formed in FIG 33 as step 458.
- Circle_Like_AHW for circular blobs has a value around 0.785. For square-like blobs the value will be closer to one (1). The program flow goes onto step 732.
- Step 732 determines whether to classify a drop as having a spherolite by utilizing the parameters found in 734 (Circle_Like_HW_lower, Circle_Like_HW_Upper, Circle_Like_AHW_Lower, and Circle_Like_AHW_Upper).
- Circle_Like_AHW(m) is less than Circle_Like_AHW_Upper and
- Step 736 tests whether all of the found blobs have been classified by testing m against num_found and, if equal, the subroutine goes onto step 738. If not the program loops back to step 730 and flows through as above.
- Step 738 goes onto FIG 34d in the classification subroutine as 740.
- FIG 34d continues from FIG 34c at 742 and goes to 744 where a second set of additional features is calculated.
- These features are generated with the original Best_Image generated in step 328 (FIG. 31) as the image of the best focus.
- the results are used from the blob analysis performed in FIG. 34c on the classify_image.
- Such features as shown in parameters 746 as num_found, area(m), height(m), width(m), perimeter(m), location_X(m), location_Y(m), num_spherolite, and spherolite(m). Plus it uses the average_gray background from FIG. 33 step 448, and the pointer_to_Best_Iimage in FIG. 31 step 328.
- "m” and num_phase_sep are set equal to zero.
- step 748 if the number of spherolites previously found (num_spherolite) is not greater than zero (0), then the flow goes to step 756 and the remaining steps shown in FIG 34d are bypassed. But if the num_spherolite is greater than zero (0), then the flow goes onto step 749 wherein m is incremented by 1. Then the flow goes on to step 751 to test whether to go to step 756 or to step 753 depending on comparing "m" to the num_found. In step 753, the value of spherolite(m) found in FIG. 34c step 732 is tested to see if any were classified as spherolite. If not, then the flow loops back to step 749. If spherolite(m) is equal to one (1) then step 750 is executed.
- step 750 the information location_X(m), location_Y(m), one half of height(m) and one half of width(m) is used to calculate the average gray value within this reduced image inside each spherolite.
- the Best_Image pixel data obtained from the previous blob analysis is utilized and phase_sep_Gray(m) is set equal to this value.
- Phase_sep_Gray(m) is then normalized by dividing it by the Average_Gray_Background.
- the program flow goes onto step 752, wherein phase_sep_Gray(m) is tested to see if it is greater than a parameter phase_sep_Gray_GT from parameter input step 754. If true, then CLASS is set equal to four(4) and the program loops up to step 749.
- step 751 the automatic program has classified the microdrop into one of the 10 primary classes, 0 through 9, detailed previously in Table 1. Then, a test is made to see if the type _of_classification is equal to 2. If so, then the flow goes onto step 755 wherein the flow is returned to FIG. 33. In step 466, a general classification is complete. If further classification into subcategories is required, then the flow goes onto step 758.
- step 758 the program flow goes onto FIG. 35a step 760.
- a crystal classification subroutine 762 further classifies any CLASS 9 crystal image into to additional subclasses, 9.2, 9.4, 9.6, 9.8, or 9.9, as previously discussed in Table 2.
- the subroutine begins at step 760 and goes onto step 764.
- the blob counter m is set to zero.
- a test is conducted to see if the CLASS is equal to 9 and, if not, the flow goes to step 778 and returns to FIG. 33 step 466.
- Step 764 uses input values as shown in step 766 (Average_Gray_Background, Pointer_To_Best_Image, num_found, area(m), height(m), width(m), perimeter(m), location_X(m), location_Y(m), num_spherolite, and spherolite(m)), as previously described. If the CLASS is equal to 9 another test is made to see if one or more than one blobs were previously found. If only one blob was previously found then the flow goes onto step 768. If more then one is found, the flow goes onto step 769 in FIG 35b. In step 768, a height to width ratio is calculated and compared to thresholds representing needle-like characteristics in step 770.
- step 771 the height to width ratio is compared to thresholds representing plate-like characteristics in step 772. If the conditions of the test are met, the CLASS is changed to 9.4 and the flow goes onto step 773. If not, no further classification is performed and the flow goes onto step 778 and the subroutine returns to FIG. 33 step 466.
- step 773 a normalized average gray value, Chunk_Gray(m), within the blob is calculated and the flow goes onto step 774.
- step 774 if Chunk_gray(m) is less than a threshold Chunck_Gray_LT (from step 775), then CLASS is set to 9.6 and the flow goes onto step 776.
- step 776 if Chunk_gray(m) is greater than a threshold Chunk_50_GT, (from step 777), then CLASS is set to 9.8 and the flow goes onto step 780.
- FIG 35b shows flowchart 781 illustrating the further classification of a CLASS 9 image having multiple blobs within the image into additional subclasses, 9.1, 9.3, 9.5, or 9.7, as previously discussed in Table 2.
- the subroutine begins at step 786 from step 769 in FIG. 35a and goes onto step 782.
- a loop begins at m equals 0 for each blob m using input values from step 784 (Average_Gray_Background, Pointer_To_Best_Image, num_found, area(m), height(m), width(m), perimeter(m), location_X(m), location_Y(m), num_spherolite, and spherolite(m)) as previously described.
- step 788 m is incremented and a height to width ratio is calculated and compared to thresholds representing needle-like characteristics in step 790. If the test conditions are met, then the CLASS is set to 9.1. If not, the flow goes onto step 794. In step 794, the height to width ratio is compared to thresholds representing plate-like characteristics in step 796. If the conditions of the test are met, the CLASS is changed to 9.3. If not, no further classification is performed and the flow goes onto step 812 to determine if all of the blobs have been tested. In step 798, a normalized average gray value (Chunk_Gray(m) as described before) within the blob is calculated and the flow goes onto step 802.
- Chunk_Gray(m) as described before
- step 802 if Chunk_gray(m) is less than a threshold Chunk_Gray_LT (from step 804) then CLASS is set to 9.5 and the flow goes onto step 806. In step 806, if Chunk_gray(m) is greater than a threshold Chunck_50_GT (from step 808) then CLASS is set to 9.7 and the flow goes onto step 812. In step 812, if all of the blobs have been tested, the flow goes onto step 810 and the subroutine returns to FIG. 33 step 466. If not all tested, then the flow loops back to step 788 for the next blob and the process loops until complete.
- Typical parameter and threshold values for use in classification in FIGS. 33, 34a, 34b, 34c, 34d, and 35a and 35b are given in Table 3 along with the preferred value. These values serve only as a guide, and other values may be used when circumstances justify. For example, different lighting conditions, variations in the transparency of micro-well plates, variations in the formulations of the protein growing media and drop, values may be used as well. One skilled in the art may adjust the parameter and threshold values to tune in the classification results specific to their setup. Table 3 Typical and preferred values for threshold and Classification parameters
- FIG. 30 shows the major components of the Applicant's prototype.
- Proteomic crystal verification and inspection system 100 has three axis of linear motion.
- Linear actuator 115 is preferably linear actuator model # 802-0763D available from Dynamic Solutions of Irvine CA., with 600 mm of travel driven by an enclosed 0.5 mm pitch ballscrew.
- Linear actuator 115 is driven by an intelligent self-contained servo motor 110 model # SM2320 SQ available from Animatics Corp. of Santa Clara, CA., with 38 oz-in of available torque.
- Servo motor 110 communicates with a Windows Based computer 105 through a serial connection routed through a central control unit 190.
- Linear actuator 115 has stationary part 116 fixed to a granite table top 170.
- Motor 110 moves moving part 117 along the x-axis.
- Granite top 170 is supported by a frame 180.
- Frame 180 has casters and adjustable legs.
- Plate 127 is attached to moving part 117.
- a spacing block 128 spaces fixture plate 129 from plate 127.
- fixture plate 129 is supported by spacing block 128.
- Fixture plate 129 provides for the mounting, removal, and positioning of multiple micro-well plates 125A - 125F.
- micro-well plates 125A - 125F are agar plates, micro-titer well plates of 96, 384, 1536 wells, or other sample plates.
- Support 191 is positioned adjacent to fixture plate 129 and contains a light source.
- the light source is model # A08925 fiber-optic back light available from Aegis Electronics of Carlsbad, CA.
- linear actuator 150 is model # 802-1384A available from Dynamic Solutions of Irvine CA., with 350 mm of travel driven by an enclosed 0.5 mm pitch ballscrew.
- Linear actuator 150 has stationary part 153 horizontally bridged above linear actuator 115 and supported by pillars 152.
- Moving base 154 provides a mounting base for components of linear actuator 160.
- linear actuator 150 is driven by driven by an intelligent self-contained servo motor 120 identical to motor 110.
- Servo motor 120 communicates with Windows Based computer 105 through a serial connection routed through central control unit 190.
- linear actuator 160 is model # 802-0756D available from Dynamic Solutions of Irvine CA., with 100 mm of travel driven by an enclosed 0.5 mm pitch ballscrew.
- Linear actuator 160 has with a stationary part 161 mounted perpendicular to and fixed to moving base 154 of linear actuator 150.
- Linear actuator 160 is driven by intelligent self-contained servo motor 130.
- Servo motor 130 is preferably identical to motor 110.
- Servo motor 130 communicates with the Windows Based computer 105 through a serial connection routed through the central control unit 190.
- Moving plate 162 provides a mounting base for camera 155.
- camera 155 is a high-resolution monochrome megapixel CCD camera model # CVMl from JAI America INC. of Laguna Hills, CA.
- Camera 155 has lens 165.
- lens 165 has a 0.75x to 3x zoom capability and is model # NT52-571 available from Edmund Industrial Optical of Barrington, NJ.
- Moving plate 162 also provides a mounting base for camera 135.
- camera 135 is a high-resolution monochrome megapixel CCD camera model # CVMl from JAI America INC. of Website.
- Camera 135 has lens 145.
- lens 145 has a 2.5x to 10x zoom capability and is model # NT54-396 available from Edmund Industrial Optical of Barrington, NJ.
- Moving plate 162 also provides a mounting base for a zoom lens motor 192, an intelligent self-contained servo motor model # SM2315D available from Animatics Corp. of Santa Clara, CA., with 20 oz-in of available torque.
- the servo motor communicates with the Windows Based computer 105 through a serial connection routed through the motion control box 190.
- the zoom lens motor operates the zoom lens 145 through a conventional belt drive.
- a bar-code reader 175 is mounted adjacent linear actuator 115 and is attached to support 172 fixed on granite top 170. Bar-code reader 175 is positioned to view a bar-code identity label attached to a micro-well plate when the micro-well plate is positioned under linear actuator 150.
- bar-code reader 175 is model # BL601 available from Keyence Corporation of America of Newark, NJ.
- a plate sensor transmitter/receiver 186 is also fixed to support 172 and is aligned to sense whenever a micro- well plate 125 breaks a beam of light emitted by transmitter/receiver 186 and reflected by a reflector 188.
- Reflector 188 is mounted on support 191 on the opposite side of the linear actuator 115.
- Plate sensor transmitter/receiver 186 and reflector 188 are preferably model # E3T-SR13 available from Western Switch Controls of Santa Ana, CA.
- FIG. 6 shows a block diagram illustrating the connectivity of Applicant's prototype.
- Linear actuator motors 110, 120 and 130, and zoom motor 192 receive DC power from 48 Volt DC power supply 680 through an electrical connection.
- Linear actuator motors 110, 120 and 130, and zoom motor 192 motors communicate with Windows-based computer 105 through common serial line 633.
- Bar-code reader 175 communicates with computer 105 through a communications line and plate sensor 186 communicates with the computer 105 through a communications line.
- Monitor 620 displays information to the operator transmitted from computer 105.
- Cameras 135 and 155 communicate with frame grabber 630 through communication lines.
- Frame grabber 630 is installed within computer 105 and is preferably PCVision from Coreco Imaging US Office in Bedford, MA..
- Frame grabber 630 digitizes the image from the camera to form the digitized image data array within computer 105.
- a 4-port Ethernet hub 640 provides for connectivity between computer 105, central control unit 190, and an external Ethernet 670. By providing for connectivity to the Ethernet, computer network communications are possible.
- the central control unit 190 controls light source 194 through an analog control line.
- Central control unit 190 receives 24 volt DC power from 24 volt DC power supply 660.
- Emergency stop button and switch (e-stop) 650 is connected to central control unit 190.
- the present invention is configured to record color images. It is desirable to be able to analyze color images in that certain precipitation products in the protein crystallization process have distinctive colors and a crystallographer or automated image analysis algorithm may use the color information to help discriminate crystallization results.
- FIG. 37 shows a side view of micro-well plate 125A positioned on fixture plate 129.
- Support 191 with embedded light source 194 is positioned to the side of fixture plate 129.
- Light from light guide 195 is directed upward through cutout 131.
- Light guide 195 is positioned between fixture plate 129 and plate 127 such that both plates can move around the light guide 195 without interference.
- Linear polarized filter 352 is rotationally mounted above light guide 195 such that light from light guide 195 can be polarized linearly at a programmable angle before it transits through micro-well plate 125A.
- Polarizer drive belt 356 (top view shown in FIG. 38) rotates polarized filter 352 about a vertical axis.
- Polarization drive belt 356 is driven by motor 358.
- Motor 358 is controlled by CCU 190 (FIG. 40).
- Second filter 354 (top view shown in FIG. 39) is positioned above micro-well plate 125A such that light transiting through micro-well plate 125A goes through second filter 354 before it goes into the camera zoom lens 145.
- Second filters 354 are mounted on filter wheel apparatus 355.
- Filter wheel apparatus 355 rotates the operator selected second filter 354 into position under the zoom lens 145.
- the selected second filter 354 is preferably either a red, green or blue dichroic filter.
- individual images taken through the red, green and blue second filters are combined to form a true color image.
- a false color (also called a pseudo color) image may be formed by taking three individual images using linear polarized filter 352 at three different polarization angles with respect to a second filter.
- a linear polarized filter is substituted for the dichroic second filters 354 discussed above.
- the polarized axis may be at 90 degrees to each other, and at plus and minus 45 degrees to each other. The three images are then called red, green, and blue and a false color image is produced. If the crystal exhibits any polarization rotation effects, then a very colorful image results. This pseudo color image is useful in detecting very small and fine crystals from the image background material.
- light guide 195 is model # A08925 fiber-optic backlight available from Aegis Electronics of Carlsbad. CA.
- Light source 194 is a Dolan-Jenner Model-PL-900 available from Edmund Industrial Optics, Barrington, NJ.
- First polarized filter 352, second filter 354 (including the linear polarized filters, and the dichroic red, green, blue filters) are available from Edmund Industrial Optics, Barrington, NJ.
- Filter wheel 355 is model FWl available from Integrated Scientific Imaging Systems, Inc of Santa Barbara, CA.
- LED Light Emitting Diode
- an array of LEDS is used to illuminate micro-well plates as they are positioned underneath lens 165 of camera 155.
- the utilization of an LED array provides several advantages, including: (1) a lifetime on the order of 10,000 hours compared to the Dolan-Jenner light source that uses a 150 watt halogen-filled incandescent bulb with a life on the order of several hundred hours, (2) an emission spectrum that is relatively constant as intensity varies over an order of magnitude compared to the incandescent source which has an emission spectrum which varies substantially with intensity, (3) switching on-off times measured in milliseconds or less compared to much longer on-off times for an incandescent bulb, (4) overall intensity at over 5 times that of a 150 watt halogen bulb and flat panel, and (5) a lower power consumption with the 600 LED array consuming less than 50 watts, and (6) more light directed in the direction perpendicular to the array compared to the diffuse flat panel driven by the incandescent bulb.
- FIG. 56A is a photograph showing a magnified image of wells El l and FI l of micro-well plate 125A.
- the photograph taken in FIG. 56A utilized a halogen filled incandescent bulb as a light source.
- the drop within well El l is approximately 1.2 millimeters across.
- FIG. 56B shows the same wells El l and FI l of micro-well plate 125A.
- the photograph taken in FIG. 56B utilized an LED array similar to that shown in FIG. 43 as a light source.
- the LED array provides a light source that is far more intense and directed. The result is better clarity in the image.
- FIG. 57A is a photograph taken at an even higher magnification than the photograph shown in FIG. 56A.
- FIG. 57A shows details of the drop containing microcrystals in well El l of micro-well plate 125A.
- the photograph taken in FIG. 57A utilized the halogen filled incandescent bulb used in FIG. 56A.
- FIG. 57B shows the same well El l as in FIG 57A.
- the LED array of FIG. 56B was used as a light source. By using the LED array light source, it was possible to achieve imaging of the fainter microcrystals in the drop.
- LED array light source 900 is mounted between fixture plate 129 and plate 127 and LED array light source 900 is providing illumination for micro-well plate 125A. LED array light source 900 is supported by side supports 901 and 902.
- FIG. 41 shows an exploded view of a preferred LED array light source 900 and FIG. 42 shows a perspective view of LED array light source 900.
- PCB (printed circuit board) 902 is mounted to bottom section 903 via screws 904.
- LEDS 905 are inserted into PCB 902 as shown. LEDS are available from many sources, however preferred LEDS 905 are white-light LEDS available from Brite-LED Optoelectronics, Valrico, FL, Part No. -BL- LBUW5B20C-NB.
- FIG. 41 shows only a small portion of the total number of LEDS 905 mounted to PCB 902.
- FIG. 43 shows a top view of a preferred array 915 of LEDS 905 mounted to PCB 902.
- LED array 915 includes 624 LEDS 905.
- Power connector 906 and control connector 908 are both mounted to extension 907.
- Fan 910 and filter 911 are both mounted to upper section 909.
- upper section 909 has cut-out section 912.
- Diffusing panel 913A and mounting frame 913B are mounted over cut-out section 912, as shown.
- diffusing panel 913A is a glass panel that has been sandblasted on one side with 120 grit spray so that the sandblasted side is facing the array of LEDS.
- Diffusing panel 913A diffuses the light from LED array 915 so that a more uniformly distributed light source is presented to the micro-well plate being inspected.
- LED array light source 900 is controlled by computer 105 (FIG. 44).
- LEDS 905 that are in the area below lens 165 are automatically turned “on” by computer 105.
- computer 105 automatically turns “off the lighted LEDS 905.
- the overall result is that as the position of lens 165 is changed relative to micro-well plate 125A, a pattern of lighted LEDS 905 underneath lens 165 follows the position of lens 165.
- the current to the LEDS in the array is controlled by computer 105 so that the intensity of the array is programmable. By varying the intensity, customized lighting applications can be programmed. In the preferred embodiment, the intensity of the lighted LEDS is varied by appropriately programming computer 105. Computer 105 then controls programmable driver 973 (FIG. 44) to drive the LEDS so that the desired intensity is displayed. As the current to the LEDS increases, the intensity of light generated by the LEDS increases. The illumination level can vary from "off to full current "on”.
- FIG. 45 shows micro-well plate 125 positioned on fixture plate 129.
- LED array 915 is positioned underneath and to the left of micro-well plate 125A.
- diffusing panel 913A FIG. 42
- FIGS. 46 - 5OB other elements of this preferred embodiment, such as fixture plate 129, are not shown so that the operation of LED array light source 900 can better be explained.
- FIG. 46A computer 105 has positioned lens 165 so that it is above the lower, left- corner well of micro-well plate 125A. Also, computer 105 has turned on a group of LEDS 905 underneath lens 165.
- FIG. 46B shows more clearly pattern 916 of the LEDS that have been turned on. LEDS that are "on" are shown in FIGS. 46 A and 46B as being all black.
- a preferred pattern 916 has the approximate shape of a hexagon.
- the center of pattern 916 is positioned underneath the center of lens 165 at the lower, left- corner well of micro-well plate 125A.
- FIG. 47 A computer 105 has moved lens 165 one position upward so that it is above the next well up from the lower, left-corner well of micro-well plate 125A. Also, computer 105 has turned on a group of LEDS 905 underneath lens 165 at its new position. FIG. 47B shows more clearly the new position of pattern 916.
- computer 105 continues to move lens 165 and pattern 916 upward from well to well. Until, as shown in FIG. 48 A, computer 105 has moved lens 165 upward so that it is above the upper, left-corner well of micro-well plate 125A. FIG. 48B shows more clearly the respective position of pattern 916 as shown in FIG. 48A.
- FIG. 49 A computer 105 has moved lens back to the position shown in FIG. 46A. Also, fixture plate 129 (FIGS. 44 and 45) has moved micro-well plate 125A one position to the left. Therefore, lens 165 is now above the well just to the right of the lower, left- corner well of micro-well plate 125A. Also, computer 105 has turned on a group of LEDS 905 underneath lens 165 at its new position. FIG. 49B shows more clearly the new position of pattern 916.
- computer 105 continues to move lens 165, pattern 916 and fixture plate 129 so that lens 165 and pattern 916 go from well to well.
- lens 165 is above the upper, right-corner well of micro-well plate 125 A.
- FIG. 5OB shows more clearly the new position of pattern 916.
- FIG. 51A shows a top view of another preferred embodiment of the present invention.
- Micro- well plate 125 A remains stationary on platform 920.
- Lens 165B is robotically controlled via computer 105 and is capable of vertical movement and movement in the x and y directions.
- LED array light source 900 is positioned under micro-well plate 125A and is also stationary.
- computer 105 controls the movement of lens 165.
- Computer 105 also moves pattern 916 under micro-well plate 125A by turning LEDS 905 "on” and "off in a fashion similar to that described above.
- FIG. 52A computer 105 has moved lens 165 so that it is positioned above the lower, right-corner well of micro-well plate 125A.
- Pattern 916 of LEDS 905 is centered underneath lens 165.
- FIG. 52B shows more clearly the new position of pattern 916.
- FIG. 53A computer 105 has moved lens 165 in the "x" and "y” direction so that it is positioned above a well more towards the center of micro-well plate 125A.
- Computer 105 has turned certain LEDS “on” and other LEDS “off so that the effect is that pattern 916 is underneath lens 165.
- FIG. 53B shows more clearly the new position of pattern 916.
- computer 105 can move lens 165 so that it is over any well of micro- well plate 125A.
- computer 105 can turn certain LEDS "on” and other LEDS “off so that the effect is that pattern 916 will always follow the movement of lens 165 so that it is underneath lens 165.
- FIGS. 54A - 54E show another preferred embodiment of the present invention.
- Robotic arms 930 and 931 grip platform 932.
- Micro-well plate 125A rests on platform 932.
- Lens 165C is positioned above micro-well plate 125A and is preferably capable of being raised and lowered. However, it remains stationary with respect to the x and y axis.
- Robotic arms 930 and 931 are controlled by computer 105 and are capable of moving platform 932 along the x and y axis.
- LED array light source 940 is stationary and is positioned below lens 165C. Light from LED array light source 940 provides illumination upward through cutout portion 941 of platform 932.
- LED array light source 940 is very similar to LED array light source 900 shown in FIGS.
- array 943 is much smaller than array 915 shown in FIG. 43.
- Array 943 can be smaller because its position is fixed with respect to the x-y position of lens 165C.
- a preferred array 943 has approximately 100 LEDS 905 compared to the 624 LEDS 905 shown in FIG. 43.
- FIG. 54B lens 165C is above a well towards the center of micro-well plate 125A.
- robotic arms 930 and 931 have moved platform 932 in the negative "y” direction and in the negative "x" direction so that lens 165C is above a well towards the upper, right corner of micro-well plate 125A.
- FIG 54C shows a side view while FIGS 54D and 54E show top views.
- FIG. 54E shows a partially cut away top view of lens 165C above array 943.
- computer 105 can alter the x-y position of platform 932 so that any well in micro-well plate 125A can be positioned between array 943 and lens 165C.
- pattern 916 was shown as being a hexagon (FIG. 46B).
- computer 105 can be programmed to create a variety of pattern shapes.
- FIG. 55A shows pattern 960, which is roughly in the shape of a circle.
- FIG. 55B shows pattern 961, which is roughly donut shaped.
- Pattern 961 is used for providing dark field illumination. Dark field illumination is used to illuminate the well from the side. Dark field illumination will cause the edges of the crystal being observed to appear light in comparison to the full image.
- computer 105 could be programmed to create a pattern in the shape of a square, rectangle, octagon, pentagon, triangle, or a variety of other shapes.
- FIG. 55C shows a top view of LED array 943.
- computer 105 is programmed to individually control the intensity of the LEDS arranged in four approximately circular zones 936A - 936D centered about the center of the array. This provides for patterns that have a symmetrically arranged intensity levels. For example, zone 936A has the lowest level of intensity, zone 936B has an intensity level higher than, zone 936A, zone 936C has an intensity level higher than zone 936B, and zone 936D has the highest level on intensity.
- FIG. 75A shows an ON/OFF/INTENSITY key that refers to FIGS. 75B - 75 A.
- the array is small and the center of the lens is always directly above the center LED.
- Each LED can be varied from OFF to full intensity ON or any degree of intensity between ON and OFF to create patterns under the lens. Sample preferred patterns are shown in FIGS. 75B - 75F.
- the system includes an ultraviolet microscope with at least one ultraviolet light emitting diode providing illumination for the microscope.
- ultraviolet LED's are mounted on a circle around the end of the lens of the microscope with each of the LED's focused at the focal region of the microscope.
- the wavelength of the LED is matched to the absorption peak of one of the fluorescing amino acids which in most cases will be tryptophan at 280 nm.
- the bandwidth of this LED is narrow so no filtering of the LED light is required.
- Light fluorescing from the amino acid is filtered to block primarily any reflected 280 nm light and the filtered fluorescing light is imaged by a camera with a charge couple device (CCD) sensor.
- CCD charge couple device
- Images are recorded digitally and preferably are stored for subsequent analysis.
- the system includes robotic components adapted to monitor large numbers of hanging drops or sitting drops for protein crystal growth.
- the robotic equipment includes computer processor components programmed with pattern recognizing algorithms for narrowing down interesting specimens for closer analysis.
- zoom visible light camera 135 is replaced with ultraviolet camera 535 as described below. Details of that camera are shown in detail in FIG. 61.
- Camera 535 includes zoom lens unit 508 (part number UV T58-946) supplied by Edmunds Scientific.
- the unit also includes eight ultraviolet LED's 510. These LED's are designed to produce narrowband ultraviolet light at about 280 nm.
- the eight ultraviolet LED's are arranged in a circle in around ring fixture 512 as shown in FIG. 61 and each of them are focused by a small lens included in the ring fixture at a region enclosing the depth of focus of lens unit 508.
- the unit includes a band pass filter (not shown) inside the lens unit that blocks substantially all light at below about 300 nm including substantially all of the 280 nm light that is reflected from the region of the drop being inspected.
- the camera unit also includes an ultraviolet CCD sensor unit not shown for producing digital images of the hanging drops.
- the sensor unit is part number SONY XCD-SX910UV supplied by Sony.
- FIGS. 62A and 62B compare hanging drop images obtained with a visible camera and a prototype version of the present invention built by Applicants.
- the comparison of the ultraviolet images (FIG. 62B) with the visible images (FIG. 62A) demonstrates that the present invention can distinguish salt crystals from protein crystals which prior art systems with only visible light units typically cannot do.
- the crystals are protein crystals located on or within horse hair loops of a Hampton Pin shown in FIGS. 59 and 60 and the robotic system is an ACTOR system used for x-ray crystallography both of which are discussed in the background section.
- This preferred embodiment utilizes an ultraviolet microscope having eight ultraviolet diode lasers in order to precisely locate one or more crystals on or within the horse hair loop of a Hampton Pin so that the crystal can be illuminated with an x-ray beam in an ACTOR x-ray crystallography system.
- the microscope utilized is preferably microscope/zoom lens 508 described above under heading "Ultraviolet Microscope” and is also shown in FIG. 61.
- FIG. 63 A and 63B show the test setup used by Applicants to demonstrate the present invention.
- FIGS. 63 A and 63B show microscope 508 and UV LED Array 510 directed at "the loop" 530 of a Hampton Pin.
- FIG. 64 shows the UV image of crystals on the loop as compared to a visible light image of the same loop.
- 65 - 69 are additional images demonstrating how clearly and precisely the protein crystals are located in the ultraviolet images of the present invention.
- the ACTOR system is programmed to recognize the crystals and to direct its x-ray beam at them automatically.
- FIG.70 shows an exploded view
- FIG. 71 shows a simplified view
- FIG. 72 shows a perspective view of another preferred embodiment of the present invention.
- ultraviolet LED 551 (FIG. 71) is utilized to illuminate the wells of micro-well plate 552.
- FIG. 70 shows an exploded view of some of the components of this preferred embodiment.
- the components preferably included: disc longpass filter 581, longpass filter 582, microscope lens unit 508, cube mirror mount 583, 3mm dowels 584, mirror mount 585, and beamsplitter 553.
- UV LED 551 is mounted to LED mount 590 (FIG. 71).
- the UV light from LED 551 is reflected off beamsplitter 553 so that it illuminates crystals growing in micro-well plate 552.
- Beamsplitter 553 is constructed such that it reflects UV light but allows fluorescing light to pass through.
- UV light is directed sideways at beamsplitter 553 and is then reflected downward onto a well of micro-well plate 552.
- the crystal in the micro-well plate fluoresces light upwards.
- the fluorescing light passes through beamsplitter 553, through lens unit 508 to camera 535.
- FIG. 73 shows another preferred embodiment of the present invention similar to the embodiment shown in FIGS. 70 - 72.
- UV LED 551 has been replaced with UV LED array 571.
- UV LED array 571 directs UV onto micro-well plate 552 in a fashion similar to that described above. However, because an array of UV LEDS is utilized, the UV light intensity is increased which allows for greater fluorescing and more precise imaging of the crystal. Substituting UV LEDS
- UV LEDS described above can be substituted as desired to accommodate a variety of protein crystals.
- a specific protein crystal fluoresces at its own wavelength when excited by UV light of a specific wavelength.
- the wavelength of the UV LED is matched to the absorption peak of the fluorescing microscopic crystals.
- Tyrosine fluoresces at wavelengths of about 303 nm when illuminated at wavelengths of about 274 nm and Phenylalanine fluoresces at wavelengths of about 257 nm when illuminated at wavelengths of about 282 nm.
- Phenylalanine fluoresces at wavelengths of about 257 nm when illuminated at wavelengths of about 282 nm there are other protein crystals that fluoresce at specific wavelengths. To image these protein crystals an appropriate UV LED with the appropriate wavelength should be utilized.
- an array of LEDS can be advantageously used with specific color output such as Red, Green, or Blue LEDS in addition to the white light LEDS described above.
- individual LEDS containing red, green, and blue emitters can be utilized to provide any color output as desired when controlled by the control computer.
- the LEDS can be strobed so that synchronization between the light and imaging apparatus may be obtained.
- lens 165 it is possible for lens 165 to be attached to either a camera (such as cameras 135 or 155 shown in FIG.
- an indexing device in which linear actuators 115, 150, and 160 operate in conjunction to sequentially position protein crystals under cameras 155 and 135, there are a variety of other types of robotic indexing devices that could also be utilized for the same purpose.
- an indexing device could be built in which the plurality of micro-well plates are kept in a stationary position.
- the camera lens would be attached to an indexing device that is preferably capable of unrestricted movement in the horizontal plane.
- the camera lens would be moved sequentially from micro-well to micro-well in the horizontal plane. Once in position over a micro-well, the lens could be raised or lowered in the vertical direction to achieve proper zoom and focus.
- an indexing device could be built in which cameras 155 and 135 are kept stationary with respect to horizontal movement.
- the plurality of micro-well plates would be preferably placed on a positioning platform that is capable of unrestricted movement in the horizontal plane. In this fashion, the positioning platform could be moved so that each micro-well is sequentially positioned underneath the appropriate camera.
- the lens once in position over a micro- well, the lens could be raised or lowered in the vertical direction to achieve proper zoom and focus.
- FIG. 12 shows protein crystal growth as a result of aqueous drop in oil protein crystallization.
- Cameras 135 and 155 focus on the crystals in drop 362.
- the present invention could also be utilized to inspect other types of microscopic specimens.
- the present invention could be utilized to inspect typical micro- well micro titer plate reactions wherein the quality of the reaction can be judged by the amount and wavelength of fluorescence emitted by the specimen by configuring the system with appropriate light sources, filters, and sensitive cameras as is typical for fluorescence detection. .
- the above preferred embodiments disclosed the utilization of two cameras 135 and 155, it would also be possible to have just one camera that is capable of zooming out so that it can focus on the entire well and zooming in so that it can focus on the drop of liquid containing the crystal.
- an area CCD camera is shown, a linear CCD camera combined with moving of the micro-well plate would also work in the present invention.
- the detents 510 and 520 can be simply spring loaded and not controlled by the computer 105.
- the invention could likewise be practiced with either the micro-wells moving in two orthogonal axes (such as X and Y) while the camera moves only in the Z-axis or the motion of all three axes be done with the camera system, wherein the micro-well plates are stationary and the system moves above them.
- These other variations of system design could also require rearrangement of the light source or multiple light sources.
- other filter types may be substituted for second filter 354. For example, a linearly polarized filter would be very effective.
Landscapes
- Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Immunology (AREA)
- General Health & Medical Sciences (AREA)
- Pathology (AREA)
- General Physics & Mathematics (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Biochemistry (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Molecular Biology (AREA)
- Biomedical Technology (AREA)
- Engineering & Computer Science (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Investigating Or Analysing Materials By Optical Means (AREA)
- Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
- Microscoopes, Condenser (AREA)
- Analysing Materials By The Use Of Radiation (AREA)
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US96172207P | 2007-07-23 | 2007-07-23 | |
US99783907P | 2007-10-05 | 2007-10-05 | |
PCT/US2008/070895 WO2009015209A1 (en) | 2007-07-23 | 2008-07-23 | Computer controllable led light source for device for inspecting microscopic objects |
Publications (1)
Publication Number | Publication Date |
---|---|
EP2188588A1 true EP2188588A1 (en) | 2010-05-26 |
Family
ID=40281798
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP08796485A Withdrawn EP2188588A1 (en) | 2007-07-23 | 2008-07-23 | Computer controllable led light source for device for inspecting microscopic objects |
Country Status (3)
Country | Link |
---|---|
EP (1) | EP2188588A1 (en) |
JP (1) | JP2010534837A (en) |
WO (1) | WO2009015209A1 (en) |
Families Citing this family (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR101191856B1 (en) | 2010-09-20 | 2012-10-16 | 주식회사 프로이천 | LED test system and test method thereof |
US9182216B2 (en) | 2010-09-22 | 2015-11-10 | Osaka University | Method for observing protein crystal |
KR101856533B1 (en) | 2011-03-28 | 2018-05-14 | 삼성전자주식회사 | Apparatus for inspecting light emitting device and inspecting method using the same |
JP2015132550A (en) * | 2014-01-14 | 2015-07-23 | 高電工業株式会社 | Specimen inspection device |
JP6320436B2 (en) * | 2016-01-15 | 2018-05-09 | 株式会社Screenホールディングス | Imaging apparatus and imaging method |
US11815446B2 (en) | 2016-01-28 | 2023-11-14 | Siemens Healthcare Diagnostics Inc. | Methods and apparatus for characterizing a specimen container and specimen |
JP6698451B2 (en) * | 2016-07-11 | 2020-05-27 | オリンパス株式会社 | Observation device |
EP3769082A4 (en) * | 2018-03-20 | 2021-12-15 | Sanofi Pasteur Limited | Methods for determining adjuvanted protein concentration and percent adsorption using intrinsic fluorescence |
WO2021232135A1 (en) * | 2020-05-19 | 2021-11-25 | S.M. Research Inc. | A random access real-time quantitative polymerase chain reaction (qpcr) reactor system |
Family Cites Families (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6985616B2 (en) * | 2001-10-18 | 2006-01-10 | Robodesign International, Inc. | Automated verification and inspection device for sequentially inspecting microscopic crystals |
WO2004077023A2 (en) * | 2003-02-27 | 2004-09-10 | University Of Georgia Research Foundation, Inc. | High-throughput structure and electron density determination |
US7162888B2 (en) * | 2003-03-31 | 2007-01-16 | Uchicago Argonne Llc | Robot-based automation system for cryogenic crystal sample mounting |
US20060291042A1 (en) * | 2005-05-17 | 2006-12-28 | Alfano Robert R | Optical scanning zoom microscope with high magnification and a large field of view |
-
2008
- 2008-07-23 EP EP08796485A patent/EP2188588A1/en not_active Withdrawn
- 2008-07-23 WO PCT/US2008/070895 patent/WO2009015209A1/en active Application Filing
- 2008-07-23 JP JP2010518356A patent/JP2010534837A/en active Pending
Non-Patent Citations (1)
Title |
---|
See references of WO2009015209A1 * |
Also Published As
Publication number | Publication date |
---|---|
JP2010534837A (en) | 2010-11-11 |
WO2009015209A1 (en) | 2009-01-29 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20090080611A1 (en) | Computer Controllable LED Light Source for Device for Inspecting Microscopic Objects | |
US7406189B2 (en) | Computer controllable LED light source for device for inspecting microscopic objects | |
WO2009015209A1 (en) | Computer controllable led light source for device for inspecting microscopic objects | |
US6637473B2 (en) | Automated storage and retrieval device and method | |
US7352889B2 (en) | Automated storage and retrieval device and method | |
EP1742064B1 (en) | Methods and apparatus for imaging and processing of samples in biological sample containers | |
US20050051723A1 (en) | Examination systems for biological samples | |
US20120126142A1 (en) | Fluorescent analysis method | |
US20070207450A1 (en) | System and method for process automation | |
US20050222777A1 (en) | System, method, and computer software prorduct for specifying parameters associated with a biological probe array | |
ES2610820T3 (en) | Method to automatically judge an agglutination image using MT system, device, program and recording medium | |
US20040241832A1 (en) | Cell culture detection apparatus, cell culture observation apparatus, and cell culture observation method | |
JP7082619B2 (en) | Transmitted illumination imaging using an array of light sources | |
US7403647B2 (en) | Method for identifying an image of a well in an image of a well-bearing component | |
US7372626B2 (en) | Device for sequential observation of samples and methods using same | |
US7326938B2 (en) | Optical system and method for inspecting fluorescently labeled biological specimens | |
US20040002154A1 (en) | Methods for preparing libraries of unique tags and related screening methods | |
JP3966220B2 (en) | Protein crystal observation device | |
US10591501B2 (en) | Automatic structure determination | |
JP2005030919A (en) | Light detector | |
US20070201734A1 (en) | Automated multicolor fluorescent microscope with scanning and thermostatic ability |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20100223 |
|
AK | Designated contracting states |
Kind code of ref document: A1 Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MT NL NO PL PT RO SE SI SK TR |
|
AX | Request for extension of the european patent |
Extension state: AL BA MK RS |
|
RIN1 | Information on inventor provided before grant (corrected) |
Inventor name: COSAND, ANDREW Inventor name: WILLIS, MICHAEL Inventor name: BORKENHAGEN, JAMES Inventor name: ROSSMAN, CHRIS Inventor name: KRANE, KEITH |
|
DAX | Request for extension of the european patent (deleted) | ||
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20160202 |