Classifications

• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
  • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
  • G01N21/64—Fluorescence; Phosphorescence
  • G01N21/645—Specially adapted constructive features of fluorimeters
  • G01N21/6456—Spatial resolved fluorescence measurements; Imaging
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
  • H04N23/50—Constructional details
  • H04N23/51—Housings
• • G01N15/1433—
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N15/00—Investigating characteristics of particles; Investigating permeability, pore-volume, or surface-area of porous materials
  • G01N15/10—Investigating individual particles
  • G01N15/14—Electro-optical investigation, e.g. flow cytometers
  • G01N15/1468—Electro-optical investigation, e.g. flow cytometers with spatial resolution of the texture or inner structure of the particle
  • G01N2015/1472—Electro-optical investigation, e.g. flow cytometers with spatial resolution of the texture or inner structure of the particle with colour
• • G—PHYSICS
  • G01—MEASURING; TESTING
  • G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
  • G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
  • G01N21/62—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
  • G01N21/63—Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
  • G01N21/64—Fluorescence; Phosphorescence
  • G01N21/6428—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes"
  • G01N2021/6439—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" with indicators, stains, dyes, tags, labels, marks
  • G01N2021/6441—Measuring fluorescence of fluorescent products of reactions or of fluorochrome labelled reactive substances, e.g. measuring quenching effects, using measuring "optrodes" with indicators, stains, dyes, tags, labels, marks with two or more labels

Definitions

• the invention relates to a color imaging system, and is particularly relevant to the production of a color image using an integrating, low light level, color camera for obtaining multi-color images of biological samples.
• U.S. Patent 5,008,185 to Bacus describes the use of an imaging system to visually analyze human breast cancer specimen cells which have been marked with an optical enhancement factor using a PAP-staining technique.
• the quantity of a particular protein in the stained cells is determined from a digital grey scale image of the specimen cells.
• a significant limitation of grey-scale analysis is that only a single color can be analyzed at a time. If the analysis is to employ more than one- color, each color necessitates a compound step consisting of staining followed by imaging.
• fluorescent dyes can be carried by known DNA probe techniques into very small structures in human biological specimens. Indeed, fluorescent markers can be carried by probes down to the molecular level of a cell. Ordinary dyes will not stain such small structures. However, because of the small size of fluorescent markers and the nature of their light-generating processes, the image of a specimen stained with fluorescent dyes is very dim.
• color cameras have been utilized for display and analysis of colored cell images.
• color cameras have not been employed to acquire colored images of cells stained by fluorescent materials because of the cameras' limited sensitivity and because of the low level of chromatic intensity provided by staining techniques.
• Proposals have been made to image specimens marked with fluorescent stains using image intensifiers. However, this would result in a monochromatic image and would surrender the significant advantages which a colored image gives.
• the need exists to provide an imaging system for the display, analysis, and archiving of low light level, color images.
• the need is particularly acute when the image to be analyzed is a magnified visualization of a cellular specimen.
• the invention is directed to means and methods for display and analysis of low light level color images for microscopic or macroscopic applications.
• the objective of the invention is to provide a color camera for acquisition of low light level color images.
• an imaging system which includes an electronic camera which is capable of low light level color imaging.
• the camera includes an integrating opto-electronic imaging assembly, a cooling means, and a lens adapter frame.
• the integrating opto-electronic imaging assembly scans an image and produces image signals representative of intensity distributions of colors in the scanned image.
• the cooling means is in thermal contact with the imaging assembly and reduces imaging assembly noise in the image signals.
• the cooling means is mounted in a open end of a camera frame.
• the lens adapter frame mounts a lens adjacent to the imaging assembly.
• the lens adapter frame is mounted in thermal contact with the cooling means.
• the camera has a means acting between the imaging assembly and the lens adapter frame to prevent condensation from forming on the imaging assembly.
• the electronically cooled integrating camera forms a major component of a system for low light level color imaging.
• the system includes a means for holding a biological sample, a filtered light source, an integrating electronic color camera, and an image analyzing means.
• the holding means holds a biological sample which is stained with a plurality of fluorescent dyes.
• the filtered light source is arranged to illuminate the stained sample.
• the integrating electronic color camera is arranged to receive light from the biological sample and includes an integrating opto-electric imaging device, a cooling means, and a conversion means.
• the integrating opto-electronic imaging device scans an image of the sample and produces image signals representative of intensity distributions of colors in the scanned image.
• the cooling means is positioned in thermal contact with the imaging device for cooling the imaging device and reduces imaging device noise in the signals.
• the conversion means is connected to the camera and converts the image signals into a plurality of mutually distinct color component signals which correspond to respective colors in the scanned image.
• the image analyzing means includes a color element means and an image reconstruction means.
• the color element means which is coupled to receive the color component signals and is responsive to color selection signals identifying at least two colors, produces color identification signals that identify colors among the colors in the scanned image.
• the image reconstruction means receive the color identification signals and produces a representation of the image in the plurality of colors contained in the image.
• Figure 1 is an exploded view of a cooled, low light level color camera according to the invention.
• Figure 2 is an exploded view of a cooled imaging assembly on the camera illustrated in Figure 1.
• Figure 3 is a magnified, partial side sectional view of the front end of the camera illustrated in Figure 1.
• Figure 4 is a schematic diagram of a circuit employed in the invention for controlling an electronic cooler in the camera of Figure 1.
• Figure 5 is a block diagram illustrating a low light level color imaging system including the camera of Figure 1.
• Figure 6 is a block diagram illustrating in greater detail a multi-color image analyzer included in the system of Figure 5.
• Utilization of an integrating opto-electronic device improves the sensitivity of the camera by extending image acquisition time in order to collect more photons.
• integrating opto-electronic devices such as CCD's exhibit a particular noise characteristic in the form of a dark current (i D ) which leaves traces ("artifacts") in an image in the form of pixellation noise which is visualized as a random scattering of color dots across an image which are not produced by any feature of the image but, rather, by the device's response to ambient heat.
• i D dark current
• pixellation noise Typically, a magnified visualization of a cell specimen which is magnified through a microscope must be acquired in an integrating color camera over a relatively long integration time. The pixellation noise in the image is proportional to the integration time.
• the invention is based on the inventors' critical realization that thermal noise effects of an integrating opto-electronic device used in a color camera to acquire low light level cell images colored by fluorescent markers might be reduced significantly by cooling the device during the integrating period. Cooling the device reduces its thermal activity and, consequently, the thermally-induced artifacts in the image acquired by the device.
• the inventors encountered a significant challenge in devising a means and method for cooling the imaging device which would not interfere with operation of the camera and degrade the quality of the acquired image.
• the resulting color camera is illustrated in Figures 1, 2, and 3.
• Figure 1 is an exploded view of a cooled, low light level, integrating color camera to be used in the invention.
• the camera assembly includes a camera frame 12 with a generally rectangular cross-section, five closed sides, and an open front end 14.
• a color camera chassis 16 is received and mounted inside the camera frame 12.
• the camera chassis contains a plurality of circuit boards which contain conventional circuitry thereon, for example, a Pulnix Model TMC-74 with integration connector 12P-02, which receives a signal from a CCD and provides an electronic output (not illustrated) preferably in a form which includes separate RGB components.
• the invention is not limited to circuitry which produces such signals, and it will be apparent to one having ordinary skill in the art that signals in other formats could serve as well.
• On the forward end of the camera frame 12 is mounted a cooled imaging assembly 20.
• the cooled imaging assembly is shown in an exploded view in Figure 2.
• a standard electronic plug or receptacle 18 is mounted on the forward end of the camera frame 12; the receptacle receives the metallic connectors of a chip carrier 24.
• An integrating opto-electronic device 28 ("CCD") is mounted in a recess in the chip carrier 24, which is covered by a thin sheet of transparent material (not shown).
• a single CCD chip is used to record an image.
• the chip carrier's metallic connectors are conventionally connected to the CCD 28 and are conventionally extended for this invention.
• a cover assembly 36 includes a bezel 40 which attaches a transparent block 34 over the CCD 28 on the chip carrier 24.
• a lens adaptor frame 44 is positioned over the imaging device 28.
• the lens adaptor frame includes a threaded portion 46 for receiving a conventional imaging lens 48 shown in dashed outline.
• the integrating opto-electronic device 28 is cooled by direct contact with an electronic thermal cooler assembly 30.
• the electronic cooler assembly 30 can include a Melcor Model Thermoelectric Module Peltier cooler, No. FCO-6-66-05L.
• the electronic thermal cooler assembly operates conventionally to conduct heat away from the CCD 28 to a heat sink 50.
• the heat sink 50 includes two parallel, elongate slots 58 through which the pins 26 of the chip carrier 24 extend when they are received in the plug receptacle 18.
• the electronic thermal cooler assembly 30 is sandwiched between the chip carrier 24 and the heat sink 50.
• the heat sink 50 is constructed from aluminum or copper and has a substantially rectangular shape which corresponds to the rectangular cross-section of the camera case 12.
• the heat sink includes a central recess 54 shaped to receive the lens adaptor frame 44 and in which the two slots 58 are disposed.
• Sets of convection ports 56 are disposed on either side of the recess 54 to enhance cooling of the heat sink by convection.
• Fans 22 (shown in Figure 1.) are mounted in the camera case 12 adjacent the heat sink in the vicinity of the convection ports 56. The fans operate to increase the flow of air through the convection ports 56 and to increase the rate of cooling of the heat sink 50.
• Threaded screws are used conventionally to attach the camera chassis 16 to the interior of the camera case 12, the heat sink 50 to the front end 14 of the camera case 12, the fans 22 to the camera case 12, the lens adaptor frame 44 to the heat sink 50 and the bezel 40 to the chip carrier 24.
• Figure 3 shows a cross-section of the cooled camera of the invention from the heat sink 50 forward to the lens adaptor frame 44. The view is a sectional one, just to the side of the chip carrier 24.
• cooling of the chip carrier 24 and the CCD 28 can induce condensation on or over the device, which can lead to significant deterioration in the quality of the image produced by the CCD 28 and eventually to failure of the device. Further, since the chip carrier 24 is also being cooled, condensation can form on it and its pins 26 potentially interfering with the operation of the camera.
• the invention provides for the maintenance of a dry atmosphere for the CCD 28 and the pins 26. The means for maintaining the dry atmosphere are illustrated in Figure 3.
• the bezel 40 and transparent block 34 when attached to the chip carrier 24 form a space 60 over the CCD 28.
• This arrangement is normally provided to protect the CCD 28 from contamination and damage.
• the invention utilizes this arrangement to provide a dry atmosphere for the CCD 28 which is retained in the space 60 by a gasket 32 around the periphery of the bezel 40 between the bezel 40 and the chip carrier 24.
• the interior space 60 between the transparent block 34 and the CCD 28 contains a dry atmosphere consisting of, for example, dry nitrogen.
• the dry atmosphere contains no water vapor and, thus, prevents condensation on or over the CCD 28 when it is cooled.
• the pins 26 are protected in the same manner by provision of a continuous seal 62 contacting and acting between an inner periphery of the lens adaptor frame 44 and an outer periphery of the bezel 40 substantially above the periphery defined by the gasket 32.
• the seal 62 can be in the form of, for example, an adhesive such as a clear silicone rubber cement.
• Additional seals 64 composed of an adhesive are provided between the pins 26 and the interior surfaces of the slots 58 of the heat sink 50.
• the seals 62 and 64 seal a space 66 formed by inner surfaces of the lens adaptor frame 44, the heat sink 50, and the slots 58.
• the space 66 contains a substantial length of the metallic connectors 26.
• the space 66 is filled with dry nitrogen, which prevents condensation on the metallic connectors 26 when the chip carrier 24 is cooled by the electronic thermal coolers 30.
• the lens adaptor frame 44 is mounted in direct thermal contact with the heat sink 50, with the thermal conductivity between the lens adaptor 44 and heat sink 50 being enhanced by, preferably, a silicone grease and very tight contact between these pieces by highly torqued screws. Similarly, a highly conductive thermal path is provided from the chip carrier 24 through the electronic thermal coolers 30 to the heat sink 50 by tight contact and, if necessary, a silicone grease.
• the camera illustrated Figures 1-3 is preferably assembled in a closed and controlled environment, for example, in a glove box containing an atmosphere of dry nitrogen.
• a closed and controlled environment By assembling the camera in such a closed and controlled environment, the dry nitrogen is trapped in the spaces 60 and 66 shown in Figure 3 where it is remains even when the camera is removed from the glove box and operated in the manner contemplated by the invention.
• the cross-sectional representation in Figure 3 shows highly conductive thermal pathways between the CCD 28 and the heat sink 50, and between the heat sink 50 and the lens adaptor frame 44.
• the electronic thermal cooler assembly 30 When the camera is operated to acquire an image on the CCD 28, the electronic thermal cooler assembly 30 is turned on to conduct heat away from, and thereby cool, the CCD 28.
• the heat sink 50 conducts the heat laterally to the left and right in Figure 2 toward the sets of convection ports 56. Operation of the fans 22 convects the heat from the heat sink 50.
• some of the heat conducted from the chip carrier 24 and CCD 28 is returned through the lens adaptor frame 44 and radiated therefrom to the outside surface of the transparent window 38 in the bezel 40. Since this thermal pathway is relatively long, the amount of heat radiated will be relatively low, but will be of a magnitude sufficient to keep the outside of the transparent window 38 above the dew point, thereby preventing condensation on the window 38.
• a three-CCD chip arrangement is used to record an image.
• a three-CCD chip arrangement is illustrated in Figure 7.
• a filter 130 can be placed in front of the CCD chips 132, 134, 136 to split the image into specific color components. The specific color components can then be sent to different CCD chips.
• a single filter may be used or a three filter configuration can be used to divide the visible color spectrum into three regions.
• the present invention need not be limited to only the visible portion of the spectrum but may also include electromagnetic radiation in the ultraviolet and infrared portions of the spectrum. The term "light" will be used to include all of these portions of the spectrum.
• FIG 4 illustrates, in the form of a block diagram, a power circuit for controlling the operation of the cooling elements.
• a plug 70 receives an AC input which, when switch 72 is closed, is conducted to an AC socket 74 into which a main power supply (not shown) for the camera chassis shown in Figure 1 is plugged.
• AC power provided to the socket 74 is also provided to a second switch 76 that controls power for fans 22 and the electronic thermal cooling assembly 30 shown in Figures 1-3.
• the switch 76 When the switch 76 is closed, AC power is provided for conversion by a linear power supply 78.
• the lamps 75 are power indication lamps.
• the power supply 106 converts and filters the AC power, providing a +6 volt DC output.
• the DC output is fed into a plurality of connections 84, 86, and 88 in an output plug 82.
• the connections can then be used to provide ⁇ » ower to the electronic components of the camera.
• the connec ⁇ on 84 can provide power to the cooling fans 22 illustrated in Figure 1, while the connection 88 can provide power to the electronic thermal cooler 30 illustrated in Figure 2.
• a switch 80 when opened, interrupts the power supplied to the fans 22, but does not interrupt the power supplied to the electronic thermal cooler assembly 30.
• the switch 80 thus eliminates vibration caused by the fans, without interrupting the thermo-electric cooling of the CCD 28 when a very highly magnified image is being acquired.
• the switch 76 shuts off the active cooling elements of the cooled camera, while permitting the camera chassis electronics to warm up.
• Figure 5 is a functional block diagram of an image analysis system which employs the cooled low light level color camera described and illustrated above to produce a single, multi-colored electronic image corresponding to a microscopic visualization of a specimen stained with fluorescent dyes.
• General details of the system of Figure 5 relating to instrument control and mechanization, and specific details of image acquisition and controlling can be understood with reference, for example, to the above-referenced Bacus patent.
• Figure 5 assumes that a biological specimen on a laboratory slide has been stained by a procedure which selectively marks cell characteristics in the specimen with fluorescent molecules, using methodology known in the art. Many well-known fiuorochromes are employed in specimen staining.
• the system of Figure 5 includes a filtered light source 90 which emits light only in the frequencies necessary to excite the fluorescent dyes used to stain biological structures in the specimen.
• a condenser lens " 92 focuses the stimulating light onto a laboratory slide 94 which carries the biological specimen 96 to be analyzed.
• the biological specimen 96 has been stained with one or more fluorescent dyes, chosen to fluoresce in different colors when stimulated by light of the appropriate frequency produced by the light source 90.
• the staining compounds may be chosen from a wide variety of known stains, or stains to be discovered.
• the stains are linked to fluorescent molecules using methodology known in the art.
• the stains can, for example, be known stains which preferentially stain the nucleus, cytoplasm, or other structures in cells, or more recently-discovered stains based on monoclonal antibodies, which attach to one particular structure for which they are designed, or recently discovered antisense DNA stains which will attach to one particular sequence of DNA, thus uniquely identifying either a known organism or known genotype.
• a probe can be labelled with two different colors. For example, location of cells might be identified by staining the cytoplasm of the cells with a cytoplasmic stain attached to a certain fluorescent color and then staining one particular structure, for example, the receptor for a given hormone, with an antibody stain attached to a different fluorescent color.
• the specimen, thus stained is illuminated and the image is acquired, the analysis of the image would recognize cells with receptors by contact of the two colors, whereas a cell without receptors would have the base color and not the receptor color.
• chromosomes may be detected by staining different chromosomes with DNA probes attached to different colors. Chromosome crossover would appear to image analysis as a strand in which color changed part way through.
• a lens 98 focuses a magnified image of the stained specimen 96 on the lens 48 of a cooled, integrating low light level color camera 10 constructed as described above.
• Lens 98 may be a microscope or a macro lens.
• the cooled color camera 10 produces a video output consisting of RGB signals 100 and a synchronization signal 102.
• the preferred format of the image represented by the RGB signals is the well-known 525 line
• the format could be, for example, a standard HDTV format.
• the image signals 100 and synchronization signal 102 are fed to a multi-channel color image analyzer 104 comprising available color identification technology.
• the image analyzer 104 operates to recognize or identify colors in a scanned image.
• the image analyzer 104 is preferably programmable to permit selective definition of colors.
• An exemplary color identification apparatus is described in detail in U.S. Patent 5,087,965, assigned to the Assignee of this application, and incorporated herein by reference.
• the image analyzer 104 includes a color frame store into which the image produced by the camera 10 is stored.
• the analyzer includes timing electronics which determine the integration time of the camera.
• the color identification element of the image analyzer 104 is a multi-channel apparatus of the type taught in the incorporated patent in which each channel is trained to recognize a different one of the fluorescent colors resulting from the different fluorescent dyes in the biological structures of the specimen 96.
• Figure 6 illustrates an image analyzer for "training" of the image recognition channels.
• the RGB signals 100 are provided, together with the synchronization signal 102 to a conventional analog-to-digital converter 110 which converts their instantaneous analog values into digital values.
• the digitized image produced by the camera 10 is switched by switch 112 to a frame buffer 114 where an image in the form of a scanned representation of the specimen 96 is stored in conventional format and fed through a interface 116 to a conventional multi-channel display 118.
• the frame buffer 114 at this point contains a multi-color image of the stained specimen 96 which enables an operator to observe the colors which are present.
• the operator then provides color select signals on a signal line 122 to a color selection apparatus 120 corresponding, for example, to the apparatus illustrated in Figure 6 of the incorporated patent.
• the color select signals 122 are used to store compound color component information in memories which define colors of interest to the color selection apparatus 120. These memories are not shown in Figure 6, but are described in detail in the incorporated patent.
• the colors of interest are selected by a user, for example, by moving a pointer across the displayed image and selecting various colors from the image.
• the selection switch 112 is then set to provide the converted, digitized scanned image to the now-programmed color selection apparatus 120.
• the color selection apparatus 120 provides an output in the form of a pixellated representation of the image acquired by the cooled camera 10 which includes only the colors indicated by the color selection signals.
• the image including only the colors selected in the color selection circuit 120 is fed to a frame buffer 124 and output through the interface 116 to the display 118.
• CML Chronic Myelogenous Leukemia
• Chronic Myelogenous Leukemia is characterized by the fusion of a bcr gene on chromosome 22 and an abl oncogene on chromosome 9.
• a reciprocal translocation frequently referred to as the Philadelphia chromosome, is found in approximately 90% of CML cases.
• a bcr/abl translocation probe can be introduced into the nucleus.
• the bcr/abl translocation DNA probe contains a hybridized bcr probe labelled with rhodamine (red), a hybridized abl probe labelled with fluorescein (green), and a counterstain DAPI (blue) for counterstaining the remainder of the DNA. If the nucleus does not contain a reciprocal translocation, the system produces a single multi-colored image which contains a plurality of random positioned red and green spots. However, if the nucleus contains a reciprocal translocation, the system produces a single multi-colored image which contains yellow or combined red/green spots which indicate the position of the reciprocal translocations.

Applications Claiming Priority (3)

Publications (2)

Family

ID=25429219

Family Applications (1)

Country Status (5)

Families Citing this family (6)

Citations (10)

Family Cites Families (7)

• 1993
  • 1993-07-08 CA CA002139754A patent/CA2139754A1/en not_active Abandoned
  • 1993-07-08 AU AU46695/93A patent/AU4669593A/en not_active Withdrawn
  • 1993-07-08 WO PCT/US1993/006442 patent/WO1994001968A1/en not_active Application Discontinuation
  • 1993-07-08 JP JP6503513A patent/JPH07509819A/ja active Pending
  • 1993-07-08 EP EP93917039A patent/EP0691064A1/de not_active Withdrawn

Patent Citations (10)

Non-Patent Citations (4)

Also Published As

Similar Documents

Legal Events