JP2004535569A - Computer techniques for image pattern recognition of organic matter. - Google Patents

Abstract

A specialized apparatus for performing various functions performed by a histologist and / or pathologist in an automated manner for a tissue specimen in which features associated with a pattern are detected.
A specialized apparatus and software method for image recognition optimized for repetitive patterns specific to organic matter. Such a method is achieved by calculating a parameter across a two-dimensional grid of pixels having an intensity value with a precision of eight valid bits for each pixel (as opposed to a one-dimensional scan). The parameters are the neural networks for each parameter, wherein each neural network is trained with images representing the tissue, structure or cell nucleus to be recognized and is described as containing no material to be recognized. Provided to multiple neural networks trained with potential images. Each neural network then outputs a measure of the similarity between the unknown substance and the known substance from which the neural network was trained. The outputs of the plurality of neural networks are collected by an association decision matrix. The neural sub-network is used for data reduction of each identified mode in the input data.
[Selection diagram] FIG.

Description

【０１３３】
【表１−２】

【０１３４】
【表１−３】

【０１３５】
【表１−４】

【０１３６】
（２） 伝染性、炎症性及び自己免疫性の病気
【０１３７】
本発明に係る方法は、伝染性、炎症性及び自己免疫性の病気を含む、免疫系を含む病気を確認するために用いることができる。これらの病気においては、炎症性細胞は、活性化され、本発明に係る方法によって検出される特性を含む定められた数の組織に浸透し、また、組織内に出現する内在細胞型内の細胞の損傷又は修復の結果である組織構造の特性変化が生じる。炎症性細胞は、好中球、肥満細胞、形質細胞、リンパ球の免疫芽芽球、好酸球、組織球及び大食細胞を含む。
【０１３８】
炎症性疾患の例として、類肉腫症及びクローン大腸炎、バクテリア性（細菌性）疾患、ウィルス性疾患、菌類による疾患のような肉芽腫性疾患、又は結核、ヘリコバクタ幽門誘発性潰瘍、髄膜炎及び肺炎のような他の生体伝染病が含まれる。アレルギー性疾患の例として、喘息、アレルギー性鼻炎（花粉症）もしくは小児脂肪便、慢性関節リウマチのような自己免疫性疾患、乾癬、Ｉ型糖尿病及び潰瘍性大腸炎、多発性硬化症、移植拒絶反応のような過敏性反応、及び、浸潤性免疫細胞又は病気の特徴となる現存の組織型の改変形態の存在を含む固有パターンを生じる他の免疫系又は炎症性状態（心内膜炎もしくは心筋炎、糸球体腎炎、膵臓炎、気管支炎、脳炎、甲状腺炎、前立腺炎、歯肉炎、胆嚢炎、子宮頚炎、甲状腺炎又は肝炎のような）の不調又は障害が含まれる。炎症性細胞の存在と動脈の裏うち又は内壁の細胞内の特性構造変化の存在とを含む粥状硬化症もまた、この方法によって認識することができる。
【０１３９】
（３） 変性疾患及び無酸素傷害又は化学的傷害
【０１４０】
本発明に係る方法は、特定の細胞型の喪失又は損傷かつ悪化した細胞型の出現を含む病気を検出するために有用である。神経変性疾患の例として、神経の喪失と損傷した神経内の特性変化とを含むアルツハイマー病、パーキンソン病、筋萎縮性側索硬化症が含まれる。虚血性損傷（血液供給の喪失）による細胞型への傷害を含む病気の例として、発作、心筋梗塞（心臓発作）、及び器官への血栓性又は塞栓性の傷害が含まれる。特定の細胞型の喪失又は改変を含む病気の例として、関節の骨関節炎が含まれる。傷害の慢性形態の例として、高血圧症、硬変症及び心不全が含まれる。細胞死の特性を生じる化学的又は中毒性の傷害の例として、腎臓の急性管系組織壊死が含まれる。器官内で生じる老化の例として、皮膚又は毛髪に生じる老化が含まれる。
【０１４１】
（４） 代謝性疾患及び遺伝病
【０１４２】
特定の遺伝病をもまた、この方法で認識することができる細胞数の特性変化を生じる。そのような病気の例として、嚢胞性繊維症、網膜炎、神経線維腫症、及びゴーシェ病やティ−サックス病のような蓄膿症が含まれる。骨髄又は末梢血球の特性変化を生じる病気の例として、貧血症又は血小板減少症が含まれる。
【０１４３】
組織及び構造の認識
【０１４４】
いくつかの実施例において、既知の組織／構造型の望ましい組の画像は前記パラメータ抽出に従属され、また、多くの種類の１つに分級器としてではなく、認識されるべき組織又は構造のための単一のテンプレートへの構造パターンの基準として用いるための人工的神経ネットワークを用いて、個々の関連的種類テンプレートが形成される。これらの基準は基準と検査の組織もしくは構造との間の「類似性の程度」を示しており、また、同時に、認識確率（信頼度）を評価するものであってもよい。次いで、各ネットワークは、図８に示す「関連付け行列」を作成する関連評価の表に寄与する。図８に示す実施例においては、認識される各組織又は構造のための各パラメータのための特定のテンプレートを有するサブネットワーク６１〜６３が設けられている。したがって、図８に示すように、組織型１の認識のために、各パラメータのためのｎのサブネットワーク６１が設けられている。同様に、組織型２のためにｎのサブネットワーク６２が設けられており、組織型ｍのためにｎのサブネットワーク６３が設けられている。前記したように、これらのサブネットワークのそれぞれは、付加的なサブネットワーク、例えば、訓練セットのデータ劣化の各モードのためのサブネットワークを含むことができる。
【０１４５】
この方法によって、本発明に係る装置は、以下の表２を含む、顕微鏡を用いて病理学者によって認識することができる組織型及び構造と同じ有用な多くの組織型及び構造である十分な確実性を有して認識することができる。前記装置の作動において、構造と基礎構造との間には機能的な差異がない。それらはいずれも、同じ方法によって認識することができる。基礎構造は、単に、より大きな構造形式内で見いだすことができる形式である。しかし、この相対的階層は、該階層が病理学者によって用いられかつ以下の表がさらに小型又は簡潔になることを可能にするので、正常の細胞型をも列挙する以下の表２に示される。
【０１４６】
【表２−１】

【０１４７】
【表２−２】

【０１４８】
【表２−３】

【０１４９】
【表２−４】

【０１５０】
【表２−５】

【０１５１】
【表２−６】

【０１５２】
【表２−７】

【０１５３】
【表２−８】

【０１５４】
【表２−９】

【０１５５】
【表２−１０】

【０１５６】
【表２−１１】

【０１５７】
【表２−１２】

【０１５８】
【表２−１３】

【０１５９】
【表２−１４】

【０１６０】
【表２−１５】

【０１６１】
脳は、人体で最も複雑な組織である。それらは、無数の脳構造、及び、前記のように列挙されない、脳走査で画像化されかつこの装置によって認識することができる他の構造、細胞型、組織及びその他である。
【０１６２】
いくつかの病気は、該病気の証明（ｈａｌｌｍａｒｋ）として用いられる組織内の物質の蓄積又は累積によって確認することができる。これらの物質蓄積は、しばしば、組織内に異常の構造を形成する。そのような蓄積は、細胞（例えば、パーキンソン病の黒質のドーパミン作動性ニューロンのレービー小体）内で位置することができる、又は、細胞外（例えば、アルツハイマー病の神経炎斑）で発見することができる。それらは、例えば、糖蛋白質、蛋白質、脂質、クリスタリン、グリコゲン及び／又は核酸の蓄積とすることができる。いくつかのものは、付加標識物なしで画像において確認することができ、他のものは、該他のものに付着される選択的な標識物を必要とする。
【０１６３】
特定の病気の診断のために有用な蛋白質の蓄積物（糖蛋白質の蓄積物を含む）の例として、アルツハイマー病の神経炎斑及びもつれ、多発性硬化症の斑、海綿状脳障害のプリオン蛋白質、強皮症のコラーゲン、ヒアリン病のヒアリン沈着物及びマロリー小体、キンメルスティルウィルソン病の沈着物、パーキンソン病のレービー小体及びレービー小体病、複数系統無栄養症のグリア細胞のアルファ瘢痕形成封入体、粥状硬化症のじゅく状斑、ＩＩ型糖尿病のコラーゲン、結核の乾酪化変性肉芽腫、及び封入体筋炎のアミロイドベータ前駆体蛋白質が含まれる。脂質の蓄積物（脂肪の蓄積物を含む）の例として、栄養学的肝疾患の沈着物、粥状硬化症のじゅく状斑、肝臓の脂肪変化、粥状硬化症の泡大食細胞、黄色腫、他の脂質蓄積疾患、及び粥状硬化症の脂肪線条が含まれる。クリスタリンの蓄積物の例として、腎結石の尿酸及びシュウ酸カルシウムの結晶、痛風の尿酸結晶、動脈硬化性斑のカルシウム結晶、腎結石症の石灰沈着、弁膜性心疾患の石灰沈着、及び乳頭癌の砂腫体が含まれる。核酸の蓄積物又は封入体の例として、ヘルペス（疱疹）のウィルスＤＮＡ、サイトメガロウイルスのウィルスＤＮＡ、ヒト乳頭腫ウイルスのウィルスＤＮＡ、ＨＩＶ（免疫不全ウイルス）のウィルスＤＮＡ、ウィルス性肝炎のカウンシルマン体、及び伝染性軟疣の軟属腫小体が含まれる。
【０１６４】
高確実性の認識に基づく装置自己教示
【０１６５】
現存の訓練された組織／構造型の経験のための関係付けテンプレート評価の蓄積された評量値の評価は、分類／認識の決定を規定する。このこと及び／又は他の理由のために、本発明に係る方法は、動的装置適応性及び自己統合化発展性を含むことができる。検査組織／細胞構造の基準評価は、定められた境界限界内（容認可能の確率帯域幅内）に入るとき、本発明に係る装置は、現試料検査のわずかな変化を含むように、パラメータ基準テンプレートの認識包絡線のそれぞれの訓練を自動的に向上させることができる。前記装置は、該装置の訓練された経験を動的かつ自動的に増加させる。基準評価が先の経験の外側にあるとき、その逸脱性の特性が、訓練された型（自己教示）のそれぞれへの関連から及び類似の分散型の有効な統計学的再発生下で出現し、新たな基準を自動的に生成し、関係付け行列に動的に付加することができる。
【０１６６】
弁別的な分子を含む位置及び量の決定の成分
【０１６７】
既知の方法を用いて、標識物又は標識物上の標識札によって発した色を示すか又は他の弁別可能の波長である画素を確認することができ、前記色の強度を、標識化された成分の量に関係づけることができる。同様に、いくつかの組織成分は、標識物の使用により画像から直接弁別することができる。成分又は標識物もしくは標識札によって発した主要信号の関係付け又は関連性の程度が決定され、構造、細胞型等に局所的に限定される。いくつかの適当な方法がある。
【０１６８】
１つの方法においては、まず、被探求成分の存在を示す弁別可能の標識又は識別特性を示す１つの画素又は複数の隣接画素を確認し、該画素が細胞核内に又は細胞核の付近に存在するか否かを決定するために調査し、該画素が細胞核内に又は細胞核の付近に存在するとき、細胞核型を確認する。成分が細胞核内に出現したとき又は細胞内に成分が必ず存在するような小さな半径範囲内に成分が出現したとき、前記した方法は、細胞型を決定し、また、成分が出現した箇所で細胞核が正常であるか又は異常であるかを決定することができる。本発明に係る装置は、また、組織型を確認することができる。組織型は、該組織型内に、限られた数の構造を有し、各構造は、限られた数の細胞型を含む。確認された細胞型が、前記組織型内のただ１つの構造型に生じたとき、前記構造が識別される。
【０１６９】
いくつかの場合において、まず、構造（該構造より大きな構造の基礎構造であってもよい。）を発見し、被探求成分が前記構造内に含まれているか否かを決定することが望ましい。この方法において、多くの試料窓であって該試料窓の一部が一致する又は重なる多くの試料窓、典型的には、前記組織の構造型のための少なくとも１つの候補を得るために十分の大きさを有する多くの試料窓が画像から得られる。各例は、前記したように神経ネットワークを用いて、構造型のためのテンプレートと比較される。次いで、構造を示すとして確認された試料窓は、寸法減少が認識の確実性を低減するまで、順に、各端部で寸法が減少される。
【０１７０】
いくつかの実施例において、成分が生じる構造が既知の基礎構造を有する構造であるとき、一部分において一致する又は重なる多くのさらに小さい窓は、減少された窓から標本化され、基礎構造のためのテンプレートに比較することができる。基礎構造が発見されたとき、小さい窓は、認識の確実性がなくなるまで、順に各端部で再び減少される。
【０１７１】
構造又は基礎構造が、画素強度の変化によって決定することができる境界を有するとき、窓又は該窓より小さい窓内の構造又は基礎構造の境界は画素のループとして確認することができ、成分を示す各画素は、該画素がループ上、ループ内又はループの外側にあるか否かを決定するために調査することができる。ループ上又はループ内のすべての画素のための成分強度は、被探求成分の存在の量を定めるために合計することができる。
【０１７２】
いくつかの場合において、前記した方法は、閾値を超える成分の存在を示す各組の１以上の隣接画素から開始するように逆転することができる。次いで、各組の画素を取り囲む窓は、前記組織に生じることが既知の構造の出現のために取得され、調査される。次いで、構造が発見されないとき、窓は拡大され、前記した過程を、構造が発見されるまで繰り返す。したがって、構造の境界を確認することができ、成分を示す前記組の画素を前記境界が含むか否かの決定がされる。
【０１７３】
特許請求の範囲に関し、前記した実施例は本発明を実施する特定の方法を説明しているが、本発明は、任意の前記した特定の例によって限定されると解釈されるべきではなく、特許請求の範囲によってのみ解釈されるべきである。
【図面の簡単な説明】
【０１７４】
【図１】全体系を示す図。
【図２】対象物の分割を示す図。
【図３】対象物から得られた試料分析窓を示す図。
【図４】６のパラメータ計算（特徴抽出）方法を示す図。
【図５】ＩＤＧパラメータ抽出方法を示す図。
【図６】認識のために用いられるサブネットワークの典型的な神経ネットワークを示す図。
【図７】細胞核認識のための決定行列を示す図。
【図８】組織又は構造の認識のための決定行列を示す図。
【符号の説明】
【０１７５】
１ 画像
２、３、４、５、６、７、８、９ 画像化構造（対象物領域）
１０ 中心線
１１、１２、１３、１４、１５、１６ 試料分析窓
１７、１８、１９ 強度レベル
２０、２１、２２ 強度ピーク
２３ 窓
３１ 入力ノード
３２ ニューロン
３３ 出力ニューロン
３６ 入力
３７ 重み付け係数
３８ 重み付けの数
３９ 最後の数
５１ 対象物分割モジュール
５２ パラメータ計算モジュール
５３ 構造パターン認識モジュール
６１、６２、６３ サブネットワーク【Technical field】
[0001]
Cross-reference of related applications
This application claims priority from US Provisional Application No. 60 / 282,677, filed provisionally on April 9, 2001, and US Provisional Application No. 60 / 310,774, provisionally filed on August 7, 2001. It asserts its rights. These and all other references mentioned herein, for all teachings and disclosures thereof, regardless of where the references appear in the present specification, are set forth as such. Incorporated herein by reference.
[Background Art]
[0002]
The human brain functions as an excellent image processing device. As a result of extensive training and experience, histologists recognize hundreds of different types of tissue, or distinguishable features of histological types, either through a microscope or from images or video (hereinafter "images"), and Learn to identify distinguishable features of structures, substructures, cell types and cell nuclei that are elements or components of tissue. By repeatedly observing these unique or characteristic patterns of tissue, the human brain generalizes the knowledge gained from the observations, and in new specimens or images, the tissue type, tissue structure, tissue substructure, cells Classify type and cell karyotype accurately.
[0003]
In addition, the human pathologist may determine the appearance of normal tissue and the appearance of one or more diseases or disorders that alter the appearance of particular cells, structures or substructures within the specimen, or the overall appearance of the tissue. And learn to identify. With extensive training and experience, human pathologists learn to identify and classify many types of diseases or disorders associated with each tissue type.
[0004]
Also, when a particular tissue component contains a visible molecule or contains a molecule that has been characterized using a chemical technique that represents a color that can be distinguished from a microscope or from an image, one can determine the presence of this particular tissue component. And the cell type or other tissue component in which the component appears can be identified.
DISCLOSURE OF THE INVENTION
[Means for Solving the Problems]
[0005]
Computer techniques according to the present invention may be used to automate various functions typically performed by histologists and / or pathologists as described above for tissue specimens in which pattern features are detected, in an automated manner (hereinafter, referred to as an automated method). Includes specialized equipment that runs in "automated methods." The specialized device is a device and method for analyzing an image of the tissue specimen, wherein (1) classifying the type of the sample, and (2) where a designated tissue structure, tissue substructure or cell karyotype exists. (3) confirming the tissue structure, tissue substructure or cell nucleus in the image by visible labeling or pixel coordinates, (4) the structure type, base at a specific position in the image Devices and methods for classifying cell nuclei of structural types, cell types and tissue components. In addition, this automated device (hereinafter "automated device") and method can be used to alter the appearance of cell nuclei or specific cell types, alter the appearance of tissue structures or tissue substructures, or alter the overall appearance of tissue. The tissue component can be classified as normal or abnormal (eg, disease or disease) based on the change. Further, the apparatus and the method classify a position where a sought component (a sought element) including a discriminable molecule appears in the sample, and classify a cell type including the sought component and the component is present in the cell nucleus. Not only the classification of the position, but also the organization type, organization structure and organization basic structure can be classified.
[0006]
In addition to the benefits of reducing the costs associated with paying histologists and / or pathologists, the devices and methods of the present invention can be tailored to perform as many of such analyzes per hour. Thereby, the device and method can be used, for example, for organisms where drugs or other bound compounds are present, organisms where a particular gene sequence product is present, or organisms where a particular tissue component is localized. It is feasible to identify the tissue components. The devices and methods according to the present invention can be adjusted to select tens of thousands of compounds or gene sequences of an organism having a set of tissue samples. Such information can be gathered with the help of histologists and / or pathologists, but the costs are high and, even at no cost, the time required for such analysis should be within the time allowed. Will be impeded.
[0007]
The apparatus and method according to the present invention uses the ability or function of image pattern recognition to learn information about an image that is characteristic of many cells fixed in relation to each other as part of the tissue of an organism. It can also recognize two-dimensional traversing patterns in the appearance of the cell nucleus surface for cells fixed to the tissue or cells separated from the original tissue within the cell. The devices and methods according to the invention can be used for cells from any type of organism, including plants and animals. One value of this device and method is in automated human tissue analysis. Such devices and methods provide the ability to automate, with an image capture device and a computer, a technique for identifying and classifying tissue type, tissue structure, tissue substructure, cell type and cell nucleus properties within a specimen. The image capture device may be any device or technique that includes scanning a sample in two or three spatial extents and captures high-resolution images that are characteristic of a tissue sample.
[0008]
Automated biohistology
[0009]
Techniques used by histologists include staring at a tissue sample containing many cells in a fixed relationship to each other, and identifying patterns that occur in the tissue. Different tissue types produce a discriminable pattern that includes many cells, groups of cells and / or many cell types. Various tissue structures and tissue substructures also result in discriminable patterns involving many cells and / or many cell types. Intercellular patterns are used by the specialized device as used by histologists to identify the tissue type, tissue structure, and tissue substructure within the tissue. Recognition of these properties by automated devices and methods does not require identification of individual cell nuclei, cells or cell types in the sample, but identification can be assisted by the simultaneous use of the method.
[0010]
The automated apparatus and method identifies individual cell types in a sample from the relationship of cell types across many cells, from the relationship between cell types and other cell types, or from the appearance of the cell type nuclei. can do. By means of methods similar to those used to identify tissue types, tissue structures and tissue substructures, the device according to the present invention provides at least two spatial images of a cell nucleus to identify individual cell types in a sample. Analyze patterns that cross target areas.
[0011]
Many cellular features that occur in tissue must be detectable from the image so that the computer device and method can recognize the tissue component by repeating the analysis of multiple cell patterns. To recognize the cell karyotype, the device examines the pattern across the image of the cell nucleus. Depending on the tissue type, the cell type being investigated, and the method of forming the image, staining of the sample may or may not be desired. Some tissue components are fully detectable without staining.
[0012]
Visible light received through an optical lens is one way to form an image. However, any other method of capturing a sufficiently large image with a sufficiently high resolution, including methods using other frequencies of electromagnetic radiation, scanning with a highly focused beam such as an x-ray flux, or using an electron microscope. Can be used.
[0013]
In one example, a tissue sample is sliced and mounted on a microscope slide by conventional methods. Alternatively, images of a plurality of cells in the tissue may be formed without removing the tissue from the specimen. For example, there are microscopes that can represent the cellular structure of human skin without removing skin tissue. There are also endoscopic microscopes that can represent the cellular structure of the walls of the gastrointestinal or gastrointestinal tract, lungs, blood vessels and other internal areas accessible to the endoscope. Similarly, an invasive probe (hereinafter "probe") can be inserted into human tissue and used to form an in vivo image. The same methods of image analysis can be applied to images acquired using these methods. For other in vivo image forming methods, if the method can identify features of images of a plurality of cells or if a pattern on the surface of a cell nucleus can be identified with a sufficient resolution, the method may be used. it can. Such methods include a method using CT scan, a method using MRI, a method using ultrasound, or a method using PET scan.
[0014]
As images are formed from the tissue, a set of data for each image is typically stored on a computing device. In one example, at least about one million pixels per image and 256 intensity levels for each of the three colors of each pixel are stored for each image to obtain at least 24 bits of information per pixel. Is done. In order to use the computer to ascertain tissue type, tissue structure and cell karyotype from this amount of data, using the full range of 256 intensity values for each pixel, the data across at least two spatial ranges A parameter is calculated from the data to reduce the amount of the data by searching for the pattern. Once the parameters have been calculated, the amount of data required to represent the parameters of the image is extremely small compared to the content of the original image. Therefore, the target information is maintained by the parameter calculation method, and the remaining information included in the image is discarded.
[0015]
Many parameters are calculated from each image. Using this technique, identification or signature can be made for each tissue type, tissue structure, tissue substructure, and cell karyotype, and this information is used as a basis for knowledge for use by specialized equipment. Preferably, a set of neural networks is used. Using this specialized device, the data contained within each parameter from the unknown image was pre-calculated from the tissue type, tissue structure, tissue substructure, cell type or other image of which cell nucleus properties are known, It is compared with the corresponding parameter. The specialized device calculates a similarity between an unknown image and a known image previously supplied to the specialized device. Similar probabilities are calculated for each comparison.
[0016]
Automated histopathology
[0017]
Normal tissue includes certain cell types that represent the unique morphological characteristics, function, and / or placement of other cells with the genetic program of the cell. Normal tissues have a precise spatial relationship with each other and contain a certain number or percentage of certain cell types. These features tend to be within a significantly narrower range within similar normal tissues among different individuals. In addition to cell types that confer on certain organs the ability to perform their intrinsic functions (eg, epithelial cells or parenchymal cells), normal cells also include blood cells, blood vessels, nerve cells, nerves including Schwann cells, and the brain. Provides the ability for the movement or contraction of tissue cells, such as fibroblasts (stromal cells) outside the central nervous system or glial cells (stromal cells), and some inflammatory cells, and organs (eg, smooth muscle) It has cells that perform common functions across organs such as cells. Combinations of cells containing these specific functions include patterns that are regenerated between various individuals for specific organs or tissues, and the like, and are described as "normal" for specific tissues by the methods described herein. "Can be recognized.
[0018]
In abnormal conditions, changes in tissue detectable by this method occur in one or more of several ways. (2) morphological changes in nuclear properties of cells; (3) changes in the proportion of specific cells; and (4) changes in the appearance of cells that are not normal components of the organ. Changes, (5) changes related to the loss of cells that should normally be present, or (6) changes due to accumulation of abnormal substances. Whether the damage is due to genetic, environmental, chemical, toxic, inflammatory, autoimmune, developmental, infectious, reproductive, neoplastic, incidental or trophic, changes in properties within the organ Occurs outside the boundaries of the normal features, which can be recognized and classified by the method according to the invention.
[0019]
By collecting images of normal and abnormal tissue types, identification or marking is performed for each normal tissue type and each known abnormal tissue type. The specialized device can replace the pathologist to determine whether the new tissue sample is normal or applies to a known abnormal tissue type. The calculated parameters can also be used to determine what each structure exhibits anomalies or what cells represent an abnormal cell nucleus and calculate the extent of the anomalies.
[0020]
Automated tissue component detector
[0021]
While the ability or ability to replace histologists and / or pathologists with automated devices is an important aspect of these devices and methods, other useful aspects include tissues, including human tissue, that can be identified from images. Function to determine the location of internal structures or other components. One of the valuable applications for this aspect of the invention is to find cellular components associated with a particular gene.
[0022]
Scientists have sequenced the genome of humans and the genome of living organisms. However, even if knowledge of the nucleic acid sequence or protein sequence of a gene is obtained, it is not always known whether or not the gene appears or appears in a living body (hereinafter referred to as "appears"). Genes can exhibit numerous patterns of appearance or appearance across tissues (hereinafter "appearances"). Some genes will be extensive, while others will show very discrete localized patterns. Gene products, such as messenger ribonucleic acid (mRNA) and / or proteins, may appear in one or more cell types or one or more tissue structures or tissue substructures in one or more tissues. Some genes do not appear in normal tissue but may appear during development or as a result of disease. Finding the cell type, tissue structure, tissue substructure and tissue type in which the gene appears and generating the gene product is extremely valuable. Currently, little is known about when and where genes appear in human or other biological tissues. Mapping the localization of single gene manifestations across the human body is a time consuming task for histologists and / or pathologists. Mapping the pattern of manifestations of a large number of genes across the human body is a tremendous task. The specialized device and method according to the invention automate this task.
[0023]
In addition to localizing the gene product, the device according to the invention can be used to find any localized component by an identifiable unique structure or an identifiable molecule, including metabolic by-products. The device is capable of recognizing substances produced by cells and / or substances associated with the exterior of cells, such as proteins, fatty acids, carbohydrates and lipids having discriminable structures or recognizable molecules that can be placed on images. Can be used to find. The sought component need not be immobilized within the cell, but instead is preferably limited to a particular region within the cell. Examples of other localized tissue components found include localized neurotic tangles, neural plaques (plaques), any drugs, adjuvants, bacteria, viruses, or prions.
[0024]
By confirming the gene product or other component to be discovered and determining its position, the automated apparatus according to the present invention provides a cell karyotype, cell type, tissue structure, tissue in which the component to be discovered occurs. It can be used to find and confirm substructure and tissue type. The component to be discovered can be a drug or compound present in the specimen. In this case, the drug or compound may be used as a label (marker) for other components in the image. Thus, the device can be used to find components that are immobilized within cells, components that are not immobilized within cells but are localized to a portion of cells, and components that occur outside of cells. .
[0025]
In one prior art approach, a researcher first gives a tissue a known label that attaches to a component of a particular cell type within a particular tissue, thereby providing a tissue and / or cellular component having a recognizable molecular structure. Find the location of. The researcher then provides a second label that marks the molecular structure sought. If the two labels are expressed together, it is possible to identify the cell in which the desired molecular structure appears. If the computer device cannot confirm the location or cell type of the cell except by detecting the position of the first label in the image, whether the two labels both appear in the image. Can be determined by a computer device.
[0026]
This prior art is generally used when there is already a known label that can mark a known cell type without marking other cell types, and therefore, this prior art is serious. There are some restrictions. Such specific and selective labels are only known for the very small part of the more than 1500 cell types found in the human body.
[0027]
The devices and methods according to the present invention can be used for tissue analysis without the use of labels to mark known cell types. In the device according to the present invention, a single label attached to a component to be discovered can be applied to one or more tissues from a living body. The devices and methods according to the present invention may be used in an automated fashion for tissue types, tissue structures and / or tissue substructures, cell types and / or, in some cases, subcellular expression of a particular component to be discovered. Check the area.
[0028]
This device is particularly useful for studying gene manifestations across multiple tissues. In this case, the researcher uses a label that selectively attaches to the messenger ribonucleic acid or other gene product for the gene of interest, and attaches the label to many tissue samples from many locations in the body. Give to. Therefore, the apparatus and method according to the present invention analyze the image of each desired tissue sample, confirm the position of each label in the image, and then determine the tissue type, tissue structure, and tissue substructure cells in which the label expresses. It can be used to confirm type and / or subcellular structure.
[0029]
In addition to finding locations where the component of interest is expressed, quantitative methods can be used to determine what amount of component is present at a given location. Such quantitative methods are known in the prior art. For example, the number of molecules of the label attached to the tissue specimen is related to the number of molecules of the component present in the tissue. The number of molecules of the label can be substantially determined by the signal strength of the pixels in the image formed from the label.
[0030]
Certain aspects of the present invention are also discussed in the specification of the US Provisional Patent Application set forth below, all of which are incorporated herein by reference in their entirety. US Patent Application No. 60 / 265,438, entitled "PPF Characteristic Tissue / Cell Pattern Features", filed January 30, 2001. U.S. Patent Application No. 60 / 265,448, entitled "TTFWT Characteristic Tissue / Cell Features", filed January 30, 2001. US Patent Application No. 60 / 265,449, entitled "IDG Characteristic Tissue / Cell Transform Features", filed January 30, 2001. U.S. Patent Application No. 60 / 265,450, entitled "IDG Characteristic Tissue / Cell Point Projection Transform Features", filed January 30, 2001. U.S. Patent Application No. 60 / 265,451, entitled "SVA, Characteristic Signal Variance Features", filed January 30, 2001. U.S. Patent Application No. 60 / 265,452, entitled "RDPH Characteristic Tissue / Cell Features", filed January 30, 2001.
BEST MODE FOR CARRYING OUT THE INVENTION
[0031]
attachment
[0032]
The tissue sample can be fixed cellular tissue or cells separated from cellular tissue such as blood cells, inflammatory cells or primary atypical pneumonia smear cells. The tissue sample can be mounted on a microscope slide by conventional methods to represent the exposed surface for viewing. The tissue can be fresh or soaked in a preservative or preservative (hereinafter "preservative") to preserve the tissue and tissue antigens and to avoid postmortem spoilage. For example, tissue that has been frozen fresh or soaked in a preservative and frozen or embedded in a substance such as paraffin, plastic, epoxy resin or celloidin may be cryostat or gliding. It can be cut into microtome or vibratome slices and mounted on microscope slides.
[0033]
staining
[0034]
Depending on the tissue type to be discovered, the cell type to be discovered, and the method desired to form the image, staining of the sample may or may not be necessary. Some cellular components are well detectable without staining. Methods used to form images without staining include contrasting techniques such as differential interference contrast, Nomarski differential interference contrast, aperture contrast (dark field), phase contrast, and polarization contrast. Additionally, other methods may be used, including techniques that do not rely on reflected light, such as Raman spectroscopy, and techniques that rely on light excitation and emission, such as epifluorescence.
[0035]
In one example, a common histological nuclear staining method such as hematoxylin is used. Eosin, which stains many components in each tissue specimen and cell, can be used. Hematoxylin is a blue to purple dye that dyes a basophil (for example, a substance having an affinity for a base) from blue to purple. Thus, areas around the cell nucleus, for example, areas containing highly concentrated nucleic acids, appear in blue. Conversely, eosin is a red-to-pink dye that stains eosinophilic substances. Thus, the protein stains red or pink. Since glycogen is not stained by hematoxylin or eosin, it is observed as a hollow bumpy space in the sample.
[0036]
Cell nucleus (Foilgen reaction), mast cells (Giemsa, Toluidine blue), carbohydrates (periodic acid Schiff, Alcian blue), connective tissue (trichrome), lipids (Sudan black, oil red O), microorganisms (grams, acid-fast) Special dyes may be used, such as those used to visualize Nissl material (cresyl violet) and myelin (Luxor Fast Blue). The pixel arrangement of these dyes can be found based only on these distinguishable colors.
[0037]
Adding a sign
[0038]
In some embodiments of the invention, a label is added to the sample. The label is typically added before the sample is stained because the dye substance reduces the attachment of the label. Optionally, in some embodiments, the label may be added after staining.
[0039]
A label is a molecule designed to attach to a particular type of location in tissue to represent a detectable location or location in the image. The method according to the invention for determining a tissue component at the location of the sought component detects the presence of some molecules detectable in the image at said location. Occasionally, such as a drug that fluoresces, or a structure that exhibits a discriminable form (or form or shape) that can be identified by pattern recognition with or without staining, The sought component is detected directly at the point of the structure. In other cases, the sought component can be identified by adding a label that attaches to the sought component to facilitate detection. Some labels are not directly detectable, and tags must be attached to the label by a method that adds a radioactive molecule to the label before the label is applied to the sample. Can be. Molecules such as digoxigenin or biotin, or enzymes such as horseradish peroxidase or alkaline phosphatase, are label tags that are usually integrated into the label to facilitate indirect detection of the label.
[0040]
In the prior art, labels that are considered very specific are labels that adhere to known cellular components of known cells. In the present invention, an object is to search for a component in an unknown tissue sample having a tissue type, a tissue structure, a tissue basic structure and / or a cell nucleus in which the component appears. This includes designing labels to find components, applying the labels to tissue samples, including many types of tissues, structures, substructures and cell types, and making sure that any part of the sample is labeled. This is realized by determining whether an object, that is, a sought component is included.
[0041]
The label can be an antibody, drug, ligand, radioactive or fluorescent compound that attaches or binds to the sought component, a compound that has a discriminable color, or a compound that is detectable by other methods. Antibody labels and other labels may be used to bind to and identify antibodies, drugs, ligands or compounds in tissue samples. The antibody or other primary binding label attached to the component sought is coated with another antibody (eg, a secondary antibody) or a secondary antibody or other label capable of detecting the label. The detection may be performed indirectly by attaching to the sought component.
[0042]
The nucleic acid probe can be used as a label. A probe is a nucleic acid that attaches or multiplies to a gene product, such as messenger ribonucleic acid (mRNA), by nucleic acid type binding (base pairing) or stereochemical interaction. The probe may be radioactive, fluorescent, have a distinguishable color, or comprise a labeling molecule such as digoxigenin or biotin. Probes can be directly or indirectly detectable using a sequentially detectable secondary label.
[0043]
Signs or tags with distinguishable colors or fluorescent or other visible indicia can be viewed through a microscope or directly from the image. Other types of labels or labels may provide indicia that can be converted into detectable radiation or images. For example, radioactive molecules are detected by techniques such as adding fluorescent or light emitting other substances by receiving radioactive luminescence or adding substances that change color, such as photographic emulsions or films, by receiving radiative energy. It is possible to do.
[0044]
Image collection
[0045]
Referring to FIG. 1, after sample preparation, the next step in the method is to obtain an image 1 that can be processed by a computer algorithm. The stored image data is converted into a numerical array, which allows for the calculation of parameters and numerical conversion. Several basic operations on the raw data can also be used, including color separation, calculation of grayscale statistics, thresholding and binarization operations, and convolution filters. These methods are commonly used to calculate parameters from an image.
[0046]
In one example of how to obtain image 1, the slides may be a motorized XY stage such as sold by Ludl and Prior and an RGB (red) such as a DVC1310C attached to the stage. , Green, blue) placed under an optical microscope such as the Zeiss Axioplan 2 with a digital camera. This typical camera captures 1300 × 1300 pixels. The camera is connected to a computer via an image capture board, such as an Epix company's pixelLYNX board. The obtained image is stored in the hard disk drive of the computer. The camera is controlled by software such as CView software supplied by DVC and the computer is connected to an RGB monitor device for viewing color images.
[0047]
The microscope is set at a magnification that can identify or discriminate cell features for many cells at once. For typical human tissue, a magnification of 10 or 20 is preferred, but other magnifications are possible. Adjust the height and aperture of the field stop and condenser, set the aperture, adjust the illumination level, focus on the image, and capture the image. These steps are preferably automated by comprehensive software for driving the microscope, motorized stages and cameras.
[0048]
Image 1 is stored in a TIFF format or other suitable format that stores the color signals of image 1 (typically red, green, blue) in a 24-bit file format (8 bits per color).
[0049]
Typically, a resolution of about 1 micron of tissue per pixel is sufficient for tissue and tissue structure recognition. This is equivalent to using a camera having a plurality of 10 micron pixels with a microscope having a 10 × objective. A typical field of view at 10 × is 630 microns × 480 microns. Considering a standard cell with a diameter of 20 microns in the tissue, this results in about 32 cells x about 24 cells. For tissue recognition, the image must represent tissue having a minimum dimension in the range of at least about 120 microns. For tissue structure recognition, some very small structures can be recognized from images representing the tissue with a minimum dimension of at least about 60 microns. For cell nucleus recognition, the image may be as large as a typical cell nucleus of about 20 microns and the pixel size may be as small as about 0.17 microns. In order to capture images with a DVC1310C camera using a Zeiss Axioplan 2 microscope with a 10 × objective lens as described above, each image represents 0.87 mm × 0.69 mm, and each pixel represents 0. Represents 66 microns x 0.66 microns. For recognition of cell nuclei, the objective lens can be changed by a factor of 20, and the resolution can be 0.11 micron per pixel of tissue.
[0050]
Image processing device
[0051]
As shown in FIG. 1, in one embodiment of the image processing apparatus and method, three main components are included. That is, (1) an object division module 51 having a function of extracting object data relating to a tissue / cell sample structure from a background signal, and (2) two (or three) spatial ranges in the data. A parameter calculation (or "feature extraction") module 52 that calculates unique structural pattern features and calculates pixel intensity changes in the data across this spatial range; and (3) an associative voting matrix architecture. And a structural pattern recognition module 53 for assessing the recognition probability (degree of reliability), typically using a plurality of neural networks. Each component will be described in turn. In alternative embodiments, the functionality of component (1) and component (2) may be combined into one module, or any other specialized equipment structure may be used for component (3). . The invention can be implemented in software, implemented in a computer readable medium, or implemented in network signals, such as running on a general purpose computer or a network of general purpose computers. As is known in the art, neural network components execute on dedicated circuitry different from one or more general purpose computers.
[0052]
Signal splitting
[0053]
In one embodiment, a signal segmentation procedure method is used to extract and enhance the color-coded (stained) signals and background structures used for feature analysis based on formal content. In this method, a color image of an object is separated into three RGB multiple spectral bands, and a covariance matrix is calculated. This matrix is paired to determine the eigenvectors representing a set of irrelevant planes arranged by reducing the level of change as a function of the "color-crowded" (structurally related) signal strength. Keratinized. Further, the steps of the division procedure vary depending on each parameter extraction method.
[0054]
Parameter extraction
[0055]
In some aspects of the parameter extraction method according to the present invention, it is necessary to find meaningful pattern information across two or three spatial extents at very small changes in pixel intensity values. For this reason, pixel data must be captured with good intensity gradations and processed. In one embodiment, 256 possible value (8 significant bits) steps or grades are used for accuracy. A value of 128 (7 valid bits) will work, but not enough. On the other hand, a value of 64 (significant bits of 6) produces significant tones, and a value of 32 (significant bits of 5) is useful for extracting parameters that are meaningful when using the method for this aspect of the invention. exceed the limit.
[0056]
The pixel intensity data values are used in a parameter extraction algorithm that operates over two or three ranges by using vector operations, which is different from a single range scan across the data. To obtain pattern data across two ranges, at least six pixels in each range are needed to avoid confusion with noise. Thus, each element of the parameter is extracted from at least a two-dimensional grid of pixels having a minimum range or size of 6 pixels. The smallest such object is an octagonal 24 pixel.
[0057]
Embodiments according to the present invention incorporate a parameter extraction module that calculates a unique structural pattern within each segmented signal / object. Tissue / cell structural patterns are discriminable and type-specific. Such a pattern results in excellent type recognition discrimination. For tissue recognition and tissue structure recognition, in one embodiment, six different parameters are calculated across a window that includes some or all of the image (sometimes a segmented image). In some embodiments, for cell karyotype recognition, parameters can be calculated independently for each region / object of interest, and an integrated diffusion gradient transform (IDG), described below, called IDG Only one parameter calculation algorithm is used, which is an integrated diffusion gradient transform.
[0058]
In one embodiment for tissue and structure recognition, no object segmentation is used so that all pixels are used in the algorithm. For nucleus type recognition, the pixels representing the nucleus are split from the remaining data so that the computationally intensive step does not stop at data that has no useful information. As shown in FIG. 2, for cell nucleus recognition, the segmentation procedure separates the imaging structures 2 to 9 defined as local regions to which object recognition is applied. These object regions are imaging structures with a high probability of containing cell nuclei. They are subject to form content based on parameter calculations that examine the content of the two-dimensional spatial and intensity distribution of the object area to calculate the discriminative properties of the nuclear material.
[0059]
For recognition of the cell nucleus, the first image 1 is obtained as a color RGB image, and then an analysis of the main components of the three color planes is performed, with the R, G and B enhanced for contrast and detail. By extracting the mixed image of the color plane, it is converted to an 8-bit grayscale data array having 256 possible intensity values for each pixel. The mixed 8-bit image is then divided between the imaged area of interest and the total average background content such that the cell nuclei having the stained density are split into objects of interest and the remaining data is discarded. Subject to a single discontinuity enhancement procedure designed to enhance contrast. When there is a large intensity jump across a few pixels, the intermediate intensity pixels are reduced to low intensity, thereby forming a sharp edge around each cluster or mass representing one or more cell nuclei.
[0060]
The division of the objects 2 to 9 is a localized N × N box-shaped polarizing filter of the same size as the individual cell nuclei, from point to point across the entire augmented image, pixel to pixel. Is achieved by applying Flocking with the formation of grouped objects having effective intensity amplitudes exceeding the statistical limit of the polarizing filter and having dimensions greater than or equal to the dimensions of the individual nuclei Alternatively, these agglomerated pixels are individually identified, mapped, and defined as a target object region. As shown in FIG. 3, the cluster of cell nuclei is observed as a unique target area 7 which is a mapping process (mapping) that determines which pixels are subjected to the feature extraction procedure. This observation is obtained by actual measurements made on the 8-bit image with the dominant component enhanced at the same points indicated by the segmented object region mapping process.
[0061]
For each cell nucleus object region, a centerline 10 is defined to substantially divide the object region along a longitudinal midline of the region. A series of six sample analysis windows 11 to 16 having dimensions substantially the same as the dimensions of the individual nuclei are arranged such that the center of the windows overlaps the midline and is aligned along the midline. The individual discriminable intensity pattern measurements are made by being calculated across two spatial extents within each window. These measurements are normalized to be substantially invariant and compared between different object area measurements obtained from different images. Obtaining a sample analysis window from the center of each group of pixels representing cell nuclei greatly enhances the chance of including one or more cell nuclei. The nucleolus is an example of a cell nucleus component that exhibits a discriminable pattern that is a valid discriminant for a cell karyotype.
[0062]
The parameter calculation used for each of the sample windows 11 to 16 for cell nucleus recognition is referred to as "integrated diffusion gradient" (IDG) of the spatial intensity distribution, as described below. This is a set of measurements that automatically separates type-specific pattern features due to relative amplitude, spatial distribution, imaging type, and formal change to a set of characteristic type derivatives. In one embodiment, 21 individual IDG measurements are calculated for each of the 6 sample windows 11 to 16, for a total of 126 IDG calculations per window.
[0063]
In one embodiment for the recognition of cell nuclei, once the IDG parameters have been calculated for each window, the eigenvector or characteristic vector of each object region 7 is calculated using the 126 measurements from each sample window and two additional Created by integrating parameters. The first additional parameter is a measure of the intensity surface occupancy of the object region across the two spatial extents, thereby calculating a "three-dimensional surface fractal" measurement. A second additional parameter is a measure of the relative working dimension of the region compared to the entire imaging field. In combination or mixing, this set of measurements becomes a unique eigenvector for each object region. It contains 128 measurements in a patterned format. All measurements are independent of conventional cross-section nuclear boundary characterization and do not integrate or require the definition or definition of nuclear boundaries. Ideally, all measurements are made within the boundaries of a single nucleus or cluster of pixels representing the nucleus (hereinafter "cluster").
[0064]
For one embodiment for the recognition of tissue type and tissue structure type, as shown in FIG. 4, the method according to the invention provides for a window in each image 1 which is generally the same size as the whole image. The procedure for calculating the parameters of various specific types in Step 6 is performed. Such parameters calculated from the image are often referred to as "extracted""features". There are many different parameter (or feature) extraction (or calculation) methods that produce useful results for this specialized device. In one embodiment, all of the parameter calculations traverse two or three spatial extents using intensity values having at least six significant bits of accuracy for each pixel, including a measure of the change in pixel intensity. Calculate the measurements of the unique pattern. In one embodiment, six parameters are calculated, as described below. All six parameters contain information specific to the underlying type of physical tissue and cell structure about the statistically distinguishable dispersion characteristics of the underlying type of physical tissue and cell structure.
[0065]
1. IDG integrated diffusion gradient
[0066]
The IDG conversion procedure can be used to calculate a basic "signal type response profile" of the structural pattern in the tissue / cell image. This procedure automatically separates the type-specific signal structure by relative amplitude, spatial distribution, signal type, and signal type distribution into a set of specific modes called "special type differentiation". These formal derivatives are modeled as a set of signal formal response functions that exhibit good type recognition discrimination when there is no coupling to a formal response profile (eg, like a linear minimum square).
[0067]
In overview, as shown in FIG. 5, each window 23 (in one embodiment, the windows 23 are small windows 11 to 16 for cell nucleus recognition and the entire image 1 for tissue or structure recognition. ) Are calculated by examining the two-dimensional spatial intensity distribution at various intensity levels 17 to 19 and calculating their local intensity-form differential variance. The placement of each level is a function of the intensity amplitude in the window. FIG. 5 shows three intensity peaks 20 to 22 extending through a first level 17 and a second level 18. Only two of them extend through the third level 19. For tissue recognition and structure recognition, in one embodiment, calculations are performed at all intensity levels (256) for all of the images. For the cell nucleus recognition in this example, the calculation is performed at only three levels, as shown in FIG. 5, to save the calculation time. This is because there are many objects 2 to 9 for each image and there are 6 sample windows 11 to 16 for each object.
[0068]
In particular, in one embodiment, the IDG parameters are extracted from the image data as follows.
[0069]
(1) The pattern image data is fitted with a self-optimizing nth-order polynomial fit, for example, a chi-square fit is calculated over n in the range of 2 to 5, and the order of the best fit is selected. This fit is used to define the flux flow "diffusion" surface for the measurement of the characteristic formal differential function. Depending on the increasing change across the pattern, this diffuse surface is distorted (fit order greater than 2). This is because, in this example, the formal derivative measurement is always perpendicular to the diffuse surface.
[0070]
(2) The diffuser is placed over the enhanced signal pattern and lowered to one unit level in time (dH). At each new location, the rate of change of the amount of signal structure through the plane is integrated and normalized by the number density (d (Si-1-Si) / d (Ni-I-Ni) = dNp). . As a result, the function automatically separates the type-specific signal structure by relative amplitude, signal strength distribution, signal type, and signal type variance into a function called the characteristic type derivative (dNp / dH).
[0071]
(3) The formal differentiation is processed by a low-pass filter to minimize signal noise effects that are evidenced as random high frequency instantaneous spikes superimposed by the main function.
[0072]
(4) Each of the peaks and valleys in the formal differential function represents the occurrence of various signal components, and the transition gradation between structures is a characteristic of signal formal dispersion.
[0073]
(5) In one embodiment, to obtain unique recognition parameters, the unique formal derivatives are decomposed into a linear superposition of these signal-specific response profiles. This optimizes the (1) response profile amplitude, (2) full width at half maximum (FWHH) profile, and (3) their relative placement by fitting the formal differential function as a linear minimum square. Is achieved.
[0074]
(6) Since the signal strength is typically referenced to the background (or noise height) level, the response function that meets the criterion is the background feature as an added feature component (or for the purpose of signal segmentation). Can be used to determine the position of a line. This can be achieved by examining the relative change in response profile measurements over the entire dNp / dH function to ascertain the onset of the signal reference line when the diffusing surface is lowered. From this analysis, the combined signal response and the signal baseline threshold (THD) are calculated.
[0075]
For tissue and structure recognition, the IDG transform extracts 256 formal derivatives fitted with 8 unique response functions. The location of each match is specified by one value, and the amplitude or magnitude is specified by a second value, all 16 values. With a curve of 256 points and two baseline parameters that are minimal for the area under the curve, this produces an input vector of 18 input values for the neural network.
[0076]
2. PPF 2D pattern projection fractal
[0077]
The PPF can be calculated by projecting the tissue / cell split signal onto a two-dimensional binary point pattern distribution. This distribution is subject to an analysis procedure that maps the cluster distribution of the projections over a wide range of sampling sample intervals across the segmented image. Sample measurement is based on the calculation of a fractal probability density function.
[0078]
The PPF focuses on the basic statistical distribution characteristics of a characteristic surface of a shape in a tissue sample. It uses a variety of tissue / cell structure patterns with unique fractal forms to achieve spatial uniformity (invariance under deviation) toning (invariance at moderate tonal changes), and self- It is based on techniques that use similarity (same basic form everywhere), for example, the naturally occurring properties of tissue patterns that represent the properties of the basic fractal form. The mixing of tissue cell types and the manner in which the tissue cell types are distributed into the tissue types results in unique differences in the imaged tissue structure.
[0079]
In one embodiment, the measurement of the PPF parameters is performed as a form of fractal probability density function calculation, using a novel procedure for point pattern projection and generation of different sized samples. In addition, the signal splitting involves analysis of the two-dimensional distribution pattern of the imaged intensity profile, and the optimal contrast image is calculated using principal component analysis, fitted with an n-th order polynomial plane, and the remaining positive projections Partitioning occurs when binarized to generate.
[0080]
(1) The divided pattern data is not biased in signal acquisition (strength). This can be achieved by repeatedly replacing each pixel value in the pattern image with a minimum locality value defined in an octagonal area between pixel widths between about 5 and 15. This results in a pattern that does not change for uniformity or stepwise variance. However, regions of high variance that are smaller than the rest of the region of interest (ROI) are reduced to a minimum level of local background.
[0081]
(2) The pattern image is fitted with a self-optimizing nth order polynomial fit, for example, a chi-square quality fit is calculated over n in the range of 2 to 5, and the best fit order is selected. This fit is used to calculate the remaining positive part of the patterned image and is binarized to generate a point pattern distribution.
[0082]
(3) The measurement of the fractal probability density function is based on the radial density distribution law, d = Cr (D ^-2 ) Is achieved. Where d is the density of tissue / cell pattern points at a given location, C is a constant, r is the distance from the center of the cluster, and D is the Hausdorff fractal dimension. The actual calculation of the fractal dimension is achieved using a box counting procedure. Here, the grid is superimposed on the tissue point pattern image, and the number of grid boxes including an arbitrary part of the fractal pattern is measured. The size of the box grid is increased and the procedure is iteratively repeated until the pattern sample size limits the number of measurements. When the number of boxes in the first and last grids is G1 and G2, and the counting results are C1 and C2, the Hausdorff dimension is given by the formula D = log (number of self-similar occupied parts) / log (magnification factor ) Or, in this case, determined by the formula D = log (C2 / C1) / log (sqrt (G2 / G1))
[0083]
(4) The extraction of the PPF feature set is to calculate the Hausdorff dimensions for a plurality of overlapping target regions (ROIs) in which the additional sampling that changes stepwise with the ROI gradation size is performed in the entire image region. Can be achieved by: Depending on the tissue type, the region of interest (ROI) can be selected to be 128 pixels × 128 pixels or 256 pixels × 256 pixels. In this example, the result is 240 individual fractal measurements on the tissue / cell point distribution pattern at sample cell magnifications varying from 0.156 to 1.0.
[0084]
The PPF algorithm extracts 240 different fractal measurements that have been progressively repositioned and scaled to generate an input vector of 240 input values to the neural network.
[0085]
3. SVA signal dispersion amplitude
[0086]
The SVA procedure relates a tissue / cell color image to three RGB multispectral bands that form the basis for principal component transformation. The covariance matrix is calculated to determine the eigenvectors and a set of irrelevant planes arranged by reducing the level of variance as a function of the "color packed" signal strength, and the diagonalization Is done. This procedure for a two-dimensional tissue / cell pattern represents a rotational transformation that maps the tissue / cell structure pattern into a region of signal dispersion. Thus, the resulting 3 × 3 remapping diagonalization matrix and the magnitude of the relative eigenvectors corresponding to the matrix are unique statistical parameters representing tissue cell signals, cell nuclei, and background signatures. Form the basis of a distributed parameter set.
[0087]
This procedure represents a rotational transformation that maps the tissue / cell structure pattern into a signal dispersion area. Thus, the principal component images (E1, E2, E3) are made irrelevant and trimmed by reducing the level of signal variance. For example, E1 has the largest variance and E3 has the smallest variance. The result is the elimination of the relationships that exist between the axes of the original RGB spectral data by simultaneous compression of the pattern variance into fewer dimensions.
[0088]
For a tissue / cell pattern, the principal component transformation represents the rotation of the original RGB coordinate axes that correspond to the directions of the maximum and minimum variance in the signal (pattern-specific) cluster. For the mean subtraction, the remapping is based on a variance distribution with a distribution about the mean that is multimodal due to the different signal patterns (eg, cells, cell nuclei, background) in the histology. Change to center.
[0089]
The principal component transform does not specifically use any information about the classification signature, but the canonical transform maximizes the separability of the defined signal structure. This separability is directly related to signal recognition, as the characteristics of the staining are class-specific to classification within a specific tissue type.
[0090]
The parameter set is the resulting 3 × 3 remapping diagonalization matrix and the relative eigenvector magnitudes corresponding to the matrix. The SVA algorithm extracts the nine parameters obtained from the RGB color 3x3 diagonalization matrix and generates an input vector of nine input values to the neural network.
[0091]
4. PPT point projection transformation
[0092]
The PPT descriptor extraction procedure converts the tissue / cell structure pattern from the specific basis of linear patterning of tissue / cell structure signals to polar form (Hough transform, similar to xcos θ + ysin θ = r). based on. This linearized projection procedure reduces the dynamic range of the tissue / cell signal split while preserving the structural pattern distribution. The resulting PPT calculation produces a remapping function performed by the "relative spatial organization" requirement to preserve a faithful representation of the image content of the original tissue / cell structure. With further signal splitting, the parameter extraction is based on the analysis of the two-dimensional distribution line pattern of the imaging intensity profile, the optimal contrast image is calculated using principal component analysis, fitted with an nth order polynomial surface and the remaining positive Segmentation occurs when subject to a two-dimensional linearization procedure that has been binarized to produce a projection and forms a line depiction corresponding to the entire tissue image.
[0093]
In one embodiment, the first two steps of the PPT parameter calculation algorithm are the same as the steps for PPF parameters described above. In the method, the following continues.
[0094]
(3) The binarized eigen-pattern is subject to a selective morphological erosion operator that changes the area of the pixel to a singular point along the midline defined as a projected linearization form within the method. This is achieved by applying a modified form of the standard erosion nucleus (kernel) to the remaining images during the iterative process. Here, the erosion operator is modified to include rules that take into account the occupancy of neighboring pairs, for example, when a central erosion point is not connected to a neighboring point that does not form a continuous distribution, the point is not removed. This process turns the projection into a linearized pattern that contains valid topological metric information based on the number of endpoints, the number of nodes where tributaries intersect, and the number of lumens in the unique pattern area.
[0095]
(4) The method uses a modified Huff transform that employs a masking algorithm that combines the selection of Huff-accumulated cells with a specific range of slope and intercept, using a modified Hough transform to transform the linearized pattern from Cartesian space. The actual PPT features are calculated by mapping in polar form.
[0096]
The PPT algorithm extracts 1752 parameters from the Hough transform of the line description of the two-dimensional tissue intensity image and generates an input vector of 1752 input values to the neural network.
[0097]
5. TTFWT textured fractal wavelet transformation
[0098]
The mixing of cell types with the distribution of cell types results in an imaged tissue structure type. Within the tissue / cell structure pattern, the unique geometric form represents a fractal primitive and forms the basis for a set of parent waves that can be used for multidimensional wavelet decomposition. The TTFWT parameter extraction procedure uses an IDG characteristic form derivative shape as a parent wave class, and uses individual wavelet transform (hereinafter, referred to as “DWT”) based on mapping of a self-similar region of a tissue / cell signal pattern image. / Extract fractal representation of cell structure pattern. Parameter extraction is based on multidimensional wavelet decomposition or resampling and integration of radial spacing to generate a characteristic waveform that includes elements related to fractal wavelet coefficient density. In one embodiment, the procedure includes the following steps.
[0099]
(1) The image pattern is 2 ^N For example, a 512 × 512, 1024 × 1024 image selected from the center of the original image is resized and sampled to fit at intervals.
[0100]
(2) The characteristic parent wave (fractal form) relates to the amplitude, spatial distribution, signal type and signal shape variance across a large set of tissue / cell images by statistical techniques under the IDG procedure described above. Determined by examining the structure specific to the signal type.
[0101]
(3) The resampled image is then subjected to a two-dimensional wavelet transform using a uniquely defined fractal-type parent wave.
[0102]
(4) Then, in order to generate a unique feature, the two-dimensional wavelet transform space is sampled and integrated at intervals of wavelet coefficients (intervals of gradation and conversion) and renormalized with respect to a unit area. . These represent the relative element energy densities of the transformation.
[0103]
The TTFWT algorithm generates an input vector of 128 input values to the neural network.
[0104]
6. RDPH radial distribution pattern harmonic function
[0105]
The RDHP parameter extraction procedure is designed to enhance the measurement of the local fractal probability density function (FPDF) in a tissue / cell pattern at a rotation or gradient invariant sample spacing. This procedure relies on the property of local self-similarity in tissue / cell images. The image components can be viewed re-graded with intensity transformation mapping that produces a self-referenced distribution of tissue / cell structure data. Implementation involves the measurement of a series of fractal dimensions measured across two spatial ranges (based on a range dependent on signal strength variance) at a central radial 360 degree scan interval. The resulting radial fractal probability density curve is normalized and subjected to a polar Fourier transform to generate a set of phase invariant parameters.
[0106]
In addition, the signal splitting is based on the analysis of a two-dimensional distribution non-biased pattern of the imaged intensity profile, and the optimal contrast image is generated using a local background to generate a signal acquisition (intensity) non-biased image. The partitioning is performed when calculated using analyzing the major components in a high variance region that is turned to a minimum level of.
[0107]
In one implementation, the first steps of the RDPH parameter calculation algorithm are the same as the steps for PPF parameters described above. In the method, the following continues.
[0108]
(2) The enhanced pattern is then unbiased in signal acquisition (strength). This is achieved by repeatedly replacing the core pixel values in the enhanced pattern image with the minimum local values defined in the octagonal region of interest (ROI). This results in an unchanged pattern of uniformity or gradual variance. However, regions of high variance smaller than the rest of the ROI are turned to the minimum level of local background.
[0109]
(3) In this example, for radial scanning sampling, a set of 360 profiles is formed from the central analysis technique within the unbiased image. For a binary tissue / cell structure pattern with pixel values simplified to black or white, this represents a measure of occupancy at a unit radial spacing constrained by image size constraints. For a continuous gray scale pattern, the profile shows the region where the signal strength is integrated.
[0110]
(4) The fractal dimension of each of the angle-dependent profiles is calculated.
[0111]
(5) In radial format, fractal measurements are normalized to unit size to remove tone dependence. This function is computed by the polar Fourier transform (PFT) to generate a set of polar harmonic functions with each component above the zeroth order representing increasing degrees of deviation from the circle. These represent the RDPH parameter set.
[0112]
The RDPH algorithm extracts 128 parameters from a polar Fourier transform of a 360 two-dimensional distribution that depends on fractal dimension measurements, and generates an input vector of 128 input values to the neural network.
[0113]
Tissue / structure / nucleus recognition
[0114]
The apparatus and method according to one embodiment of the present invention are configured to meet three main design specifications. They include (1) the ability to handle high-throughput automatic classification of tissue and cell structures, (2) the ability to form an assessment of the correlation of classified tissue / cell structure properties, and (3) training. It is a function that expands the experienced experience adaptively and brings about self-expanding evolutionary growth.
[0115]
Achieving these design criteria can be achieved through the use of an association decision matrix that manipulates the output of multiple neural networks. FIG. 6 shows one neural network. As noted above, some parameter calculation methods produce a set of 128 values that are inputs to provide to 128 input nodes 31 of the neural network. Other parameter calculations require other numbers of input nodes. For each neural network, the second layer has half the number of neurons. For example, the network shown in FIG. 6 has 64 neurons 32 in the second layer and also has unique output neurons 33. Each of these neural networks may have a sub-network, as described below.
[0116]
Each network can be trained to classify images into one of many known types. In this case, each network is trained for all types.
[0117]
Alternatively, in another embodiment, each network is trained on only one pattern and returns a level of relationship recognition ranging from 0 when not at all similar to 1 when completely similar. Designed. In this case, the network is trained on only two types of images showing the sought substance and other substances similar to the substance except in the images that should not be analyzed. The output of each network is a probability value represented by a value from 0 to 1, and the substance of the image is the item on which the network was trained. For output to humans, the probabilities may be replaced by a representation in percent as shown in FIG. The outputs of many neural networks are then collected to produce one of the most probable decisions.
[0118]
Thus, each neural network compares the input vector with a "template" resulting from training the network on a single pattern with many images of the pattern. Thus, an individual network is used for each pattern to be recognized. When a sample is to be classified into one of 50 tissue types, 50 networks are used. The network can be implemented by software on a general purpose computer, and each of the 50 networks can be read by a single computer in succession for calculations. Alternatively, the network may be run concurrently or otherwise on 50 computers in parallel.
[0119]
In one structure of a neural network, device analysis from the collection of feature extractions is a method of distributing data degradation (or data degradation) within a tissue processing procedure and within a data collection environment that affects the ability to isolate and measure unique patterns. Can be used to identify or identify various causes of These causes of data degradation can be ascertained by human experience and intuition. Because these causes are usually not independent, they typically cannot be linearly decoupled, removed, or individually corrected.
[0120]
Aspects of the data degradation confirmation mode include: (1) staining types, staining application methods, multiple staining interference / darkening and tissue processing structures such as physical tissue quality control issues, (2) spherical and barrel distortions, Aspects of data collection related to microscopic imaging aberrations, such as RGB color control, pixel dynamic range and resolution, digital quantization, and aliasing effects; (3) patterns based on instrument noise effects and statistical sampling density Measurement variance, and (4) the effects of unwanted changes in the level of staining applied. In one embodiment, they are grouped into seven categories.
[0121]
To compensate for these distributed modes of data degradation and enhance cognitive performance, in one embodiment, for each neural network, there are various device dispersion aspects (modes): seven individual modes and A set of eight different sub-networks responsible for one mixed mode is used. Each subnetwork processes the same input pattern, but is trained on data that describes the key effects inherent in the various distributed modes and their relative coupling to other morphological data degradation aspects. This processing structure is used in tissue preparation, data collection, and recognition based on patternable forms of darkening from other artifacts, interference or noise, by directly incorporating the recognition of the eigenranges of the artifacts in the image. One approach is to provide the ability to reduce and minimize the level of loss to the correlation decision matrix.
[0122]
In one embodiment, a person can select a known image that exhibits a desired data degradation effect and train the sub-network with images that indicate the specific cause of the data degradation. The eighth sub-network can be trained on all or some images. For each image, the sub-network can be educated, whether or not the image indicates the type of tissue, structure or cell nucleus for which the network is to be trained.
[0123]
Cell nucleus recognition
[0124]
For cell nucleus recognition, in some embodiments, only IDG parameters are used for each cell nucleus or swarm, and only one neural network is used for comparison to each recognition "template" ( Although the network may include a sub-network for each data degradation mode.) For example, for cancer oncogenesis, only one neural network is needed, but this neural network can have 8 sub-networks for data degradation mode.
[0125]
For example, for cell nucleus recognition, the IDG parameters yield a set of 128 values for each of the eight subnetworks, with eight outputs 33 from the subnetwork. These eight outputs are provided as inputs 36 to the association decision matrix shown in FIG. Each input may be adjusted with a weighting factor 37. The device according to the invention uses one weight. Other weights can be selected as desired. The weighting number 38 having a relation level in the range of 0 to 1 is added in this embodiment to yield the last number 39 between 0 and 8. The sum of the morphological relationship levels is called determining the association matrix. A determination of 4.0 or higher is considered to be a positive recognition of the cell karyotype being examined.
[0126]
Recognition of cell nuclei can typically determine not only whether a cell nucleus is observed as abnormal, but also all cell types. A list of recognizable tissues, tissue structures, and substructures, as well as a list of normal cell types that can be identified by an indication of the cell type nucleus, is provided in Table 2 below.
[0127]
Abnormal cell types suitable for use with the present invention include, for example, the following four categories:
[0128]
(1) neoplastic and reproductive diseases
[0129]
The altered nuclear properties of the neoplastic cells and the altered growth arrangement of the cells confirm both benign and malignant proliferation, discriminate the proliferation from surrounding normal or reactive tissue, and distinguish between benign and malignant diseases And enables a method for identifying or identifying invasive and pre-invasive components of a malignant disease.
[0130]
Examples of (but not limited to) benign reproductive diseases are shown below. Wounds, connective tissue forming tissue reactions, fibromuscular line growth (such as those of the breast and prostate). Adenomas of the breast, respiratory organs, gastrointestinal tract, salivary glands, liver, gallbladder, endocrine glands. Benign growth of soft tissues such as fibromas, neuromas, neurofibromas, meningiomas, gliomas and leiomyomas. Benign tumors of the epithelium and appendages of the skin, benign melanocyte nevus. Oncocytoma of the kidney and benign tumor of the ovarian surface epithelium.
[0131]
Examples of malignancies suitable for use with the methods, devices and the like according to the invention are listed in Table 1 below, for key points and relocation points in the invasive or pre-invasive phase.
[0132]
[Table 1-1]

[0133]
[Table 1-2]

[0134]
[Table 1-3]

[0135]
[Table 1-4]

[0136]
(2) Infectious, inflammatory and autoimmune diseases
[0137]
The method according to the present invention can be used to identify diseases involving the immune system, including infectious, inflammatory and autoimmune diseases. In these diseases, inflammatory cells are activated and penetrate a defined number of tissues, including the properties detected by the methods of the present invention, and cells within endogenous cell types that appear in tissues. There is a change in tissue structure properties that is the result of damage or repair of the tissue. Inflammatory cells include neutrophils, mast cells, plasma cells, lymphoblasts, eosinophils, histiocytes and macrophages.
[0138]
Examples of inflammatory diseases are granulomatous diseases such as sarcoidosis and clonal colitis, bacterial (bacterial) diseases, viral diseases, fungal diseases, or tuberculosis, Helicobacter pyloric ulcers, meningitis. And other biologically transmitted diseases such as pneumonia. Examples of allergic diseases include asthma, allergic rhinitis (hay fever) or childhood steatorrhea, autoimmune diseases such as rheumatoid arthritis, psoriasis, type I diabetes and ulcerative colitis, multiple sclerosis, transplant rejection Hypersensitivity reactions, such as reactions, and other immune system or inflammatory conditions (endocarditis or myocardium) that produce a unique pattern that includes infiltrating immune cells or the presence of altered forms of existing histological types that are characteristic of the disease. Inflammation, glomerulonephritis, pancreatitis, bronchitis, encephalitis, thyroiditis, prostatitis, gingivitis, cholecystitis, cervicitis, thyroiditis or hepatitis). Atherosclerosis, including the presence of inflammatory cells and the presence of intracellular characteristic structural changes in the lining or lining of arteries, can also be recognized by this method.
[0139]
(3) degenerative diseases and anoxic or chemical injuries
[0140]
The method according to the invention is useful for detecting diseases involving the loss or damage of particular cell types and the appearance of deteriorated cell types. Examples of neurodegenerative diseases include Alzheimer's disease, Parkinson's disease, and amyotrophic lateral sclerosis, which include nerve loss and altered properties in the damaged nerve. Examples of diseases that include damage to cell types due to ischemic damage (loss of blood supply) include stroke, myocardial infarction (heart attack), and thrombotic or embolic damage to organs. Examples of diseases involving loss or modification of particular cell types include osteoarthritis of the joint. Examples of chronic forms of injury include hypertension, cirrhosis and heart failure. Examples of chemical or toxic insults that produce the property of cell death include acute tubular necrosis of the kidney. Examples of aging that occurs in organs include aging that occurs on the skin or hair.
[0141]
(4) Metabolic and genetic diseases
[0142]
Certain genetic diseases also result in characteristic changes in cell numbers that can be recognized in this way. Examples of such diseases include cystic fibrosis, retinitis, neurofibromatosis, and pyometra such as Gaucher's disease or T-Sachs disease. Examples of diseases that result in altered properties of bone marrow or peripheral blood cells include anemia or thrombocytopenia.
[0143]
Recognition of organization and structure
[0144]
In some embodiments, a desired set of images of a known tissue / structure type is subject to the parameter extraction, and for one of many types, rather than as a classifier, for the tissue or structure to be recognized. Individual associated type templates are formed using an artificial neural network for use as a basis for structural patterns into a single template. These criteria indicate the “degree of similarity” between the criteria and the organization or structure of the test, and may also evaluate the recognition probability (reliability) at the same time. Each network then contributes to an association assessment table that creates the "association matrix" shown in FIG. In the embodiment shown in FIG. 8, there are provided sub-networks 61 to 63 having specific templates for each parameter for each recognized organization or structure. Therefore, as shown in FIG. 8, for the recognition of the organization type 1, n sub-networks 61 for each parameter are provided. Similarly, n sub-networks 62 are provided for organization type 2 and n sub-networks 63 are provided for organization type m. As mentioned above, each of these sub-networks may include additional sub-networks, for example, sub-networks for each mode of data degradation in the training set.
[0145]
By this method, the device according to the present invention has sufficient certainty that it is as many useful tissue types and structures as can be recognized by a pathologist using a microscope, including Table 2 below. Can be recognized. In the operation of the device, there is no functional difference between the structure and the substructure. They can all be recognized in the same way. A substructure is simply a form that can be found within a larger form of construction. However, this relative stratum is shown in Table 2 below, which also lists normal cell types, as it is used by pathologists and allows the following table to be smaller or more concise.
[0146]
[Table 2-1]

[0147]
[Table 2-2]

[0148]
[Table 2-3]

[0149]
[Table 2-4]

[0150]
[Table 2-5]

[0151]
[Table 2-6]

[0152]
[Table 2-7]

[0153]
[Table 2-8]

[0154]
[Table 2-9]

[0155]
[Table 2-10]

[0156]
[Table 2-11]

[0157]
[Table 2-12]

[0158]
[Table 2-13]

[0159]
[Table 2-14]

[0160]
[Table 2-15]

[0161]
The brain is the most complex tissue in the human body. They are countless brain structures and other structures, cell types, tissues and others that are imaged in brain scans and can be recognized by this device, not listed above.
[0162]
Some diseases can be identified by the accumulation or accumulation of substances in tissues used as hallmarks of the disease. These substance accumulations often form abnormal structures in the tissue. Such accumulations can be located within cells (eg, Lewy bodies of dopaminergic neurons in the substantia nigra of Parkinson's disease) or found extracellularly (eg, neuritic plaques in Alzheimer's disease). be able to. They can be, for example, accumulations of glycoproteins, proteins, lipids, crystallins, glycogen and / or nucleic acids. Some can be identified in the image without additional labels, others require selective labels to be attached to the others.
[0163]
Examples of protein accumulations (including glycoprotein accumulations) useful for the diagnosis of certain diseases include neuritic plaques and tangles of Alzheimer's disease, plaques of multiple sclerosis, and prion protein of spongiform brain injury Collagen, scleroderma, hyaline deposits and Mallory bodies of hyaline disease, deposits of Kimmel Still Wilson's disease, Lewy bodies and Lewy body diseases of Parkinson's disease, alpha scar formation of glial cells of multiple strains atrophy Includes inclusion bodies, atherosclerotic plaques, type II diabetes collagen, caseous degeneration granuloma of tuberculosis, and amyloid beta precursor protein of inclusion body myositis. Examples of lipid deposits (including fat deposits) include nutrient liver disease deposits, atherosclerotic plaques, liver fat changes, atherosclerotic foam macrophages, Includes xanthomas, other lipid storage disorders, and atrial sclerosis fatty streaks. Examples of crystallin deposits include uric acid and calcium oxalate crystals in kidney stones, uric acid crystals in gout, calcium crystals in atherosclerotic plaques, calcifications in nephrolithiasis, calcifications in valvular heart disease, and papillary carcinoma The sarcoma body is included. Examples of nucleic acid accumulations or inclusion bodies include herpes (herpes) viral DNA, cytomegalovirus viral DNA, human papillomavirus viral DNA, HIV (immunodeficiency virus) viral DNA, and viral hepatitis Councilman. Includes soft body tumours, as well as infectious soft warts.
[0164]
Device self-teaching based on recognition of high certainty
[0165]
The evaluation of the accumulated grading value of the association template evaluation for existing trained organization / structural type experience defines the classification / recognition decision. For this and / or other reasons, the method according to the present invention may include dynamic device adaptability and self-integration development. When the reference evaluation of the test tissue / cell structure falls within the defined boundary limits (within an acceptable probability bandwidth), the apparatus according to the present invention provides a parameter criterion to include small changes in the current sample test. The training of each of the recognition envelopes of the template can be automatically improved. The device dynamically and automatically increases the trained experience of the device. When the baseline assessment is outside of the previous experience, its deviance characteristics emerge from the association to each of the trained forms (self-teaching) and under a similar decentralized effective statistical reappearance. , New criteria can be automatically generated and dynamically added to the association matrix.
[0166]
Components of location and quantity determination including discriminating molecules
[0167]
Using known methods, it is possible to identify pixels that exhibit the color emitted by the sign or the tag on the sign or that are at other discriminable wavelengths, and that the intensity of the color is labeled. It can be related to the amount of the component. Similarly, some tissue components can be distinguished directly from the image by the use of labels. The degree of association or relevance of the primary signal emitted by the component or label or label is determined and is locally limited to structure, cell type, etc. There are several suitable methods.
[0168]
In one method, first identify a pixel or a plurality of adjacent pixels that indicate a distinguishable marker or signature that indicates the presence of the sought component and determine whether the pixel is in or near the cell nucleus. A check is made to determine if the pixel is within or near the cell nucleus to confirm the cell karyotype. When the component appears in the cell nucleus or when the component appears within a small radius such that the component always exists in the cell, the above-described method determines the cell type and also determines the cell nucleus at the location where the component appears. Can be determined to be normal or abnormal. The device according to the invention can also confirm the tissue type. A tissue type has a limited number of structures within the tissue type, and each structure includes a limited number of cell types. The structure is identified when the identified cell type occurs in only one structural type within the tissue type.
[0169]
In some cases, it may be desirable to first discover a structure (which may be a substructure of a structure larger than the structure) and determine whether the sought component is included in the structure. In this method, a number of sample windows, many of which coincide or overlap with each other, typically sufficient to obtain at least one candidate for the structural type of the tissue. Many sample windows having a size can be obtained from the image. Each example is compared with the template for the structural type using the neural network as described above. The sample window, identified as showing structure, is then reduced in size at each end, in turn, until the reduction in size reduces the certainty of recognition.
[0170]
In some embodiments, when the structure from which the components occur is a structure having a known substructure, many smaller windows that coincide or overlap in part are sampled from the reduced window to provide a substructure for the substructure. Can be compared to templates. When the substructure is found, the small windows are reduced again at each end in turn, until there is no certainty of recognition.
[0171]
When the structure or substructure has boundaries that can be determined by changes in pixel intensity, the boundaries of the structure or substructure within the window or smaller window can be identified as a loop of pixels and indicate components. Each pixel can be examined to determine if the pixel is on, within, or outside the loop. The component intensities for all pixels on or within the loop can be summed to determine the amount of presence of the sought component.
[0172]
In some cases, the methods described above can be reversed to start with each set of one or more neighboring pixels indicating the presence of a component that exceeds a threshold. The window surrounding each set of pixels is then acquired and examined for the appearance of structures known to occur in the tissue. Then, when no structure is found, the window is enlarged and the process described above is repeated until a structure is found. Thus, the boundaries of the structure can be ascertained, and a determination is made as to whether the boundaries include the set of pixels representing the components.
[0173]
With respect to the following claims, the above-described embodiments describe specific ways of practicing the invention, but the invention should not be construed as limited by any of the specific examples described above, It should be interpreted only by the claims.
[Brief description of the drawings]
[0174]
FIG. 1 is a diagram showing an entire system.
FIG. 2 is a diagram showing division of an object.
FIG. 3 is a view showing a sample analysis window obtained from an object.
FIG. 4 is a view showing a parameter calculation (feature extraction) method of 6;
FIG. 5 is a diagram showing an IDG parameter extraction method.
FIG. 6 shows an exemplary neural network of sub-networks used for recognition.
FIG. 7 is a diagram showing a decision matrix for cell nucleus recognition.
FIG. 8 is a diagram showing a decision matrix for tissue or structure recognition.
[Explanation of symbols]
[0175]
1 image
2, 3, 4, 5, 6, 7, 8, 9 imaging structure (object area)
10 center line
11, 12, 13, 14, 15, 16 Sample analysis window
17, 18, 19 Strength level
20, 21, 22 intensity peak
23 windows
31 Input node
32 neurons
33 output neuron
36 inputs
37 Weighting factor
38 Number of Weights
39 Last number
51 Object Split Module
52 Parameter Calculation Module
53 Structural pattern recognition module
61, 62, 63 subnetwork

Claims (188)

A computer method that uses an image of an unknown tissue including cells of an organism to classify the unknown tissue into a certain type,
(A) a pixel data image of an unknown tissue, wherein the pixel data image represents tissue having a minimum dimension of at least about 120 microns, wherein each pixel of the pixel data image is represented by at least six significant bits; Receiving a pixel data image of an unknown tissue having intensity value data;
(B) selecting at least one analysis window of pixel data from within said image, each pixel from a two-dimensional grid of pixels in a window having at least six pixels of a minimum dimension for providing calculated parameters; Calculating, from the pixel data for the analysis window, at least one parameter constituting a measurement of a two-dimensional pattern traversing at least two spatial extents of the image intensity value data having at least six significant bits for To do,
(C) comparing the calculated parameter with at least two different corresponding parameters pre-calculated from images of at least two different types of tissue known to be at least a first one; Providing a type and a second type; and (e) determining whether the unknown tissue is relatively similar to any of the first type and the second type. Computer technique. A computer readable data carrier comprising a computer program running on a computer for causing a computer to perform the method of claim 1. The method of claim 1, wherein the first type of tissue is a single type of tissue and the second type of tissue is a plurality of other types of tissue. further,
(F) comparing the calculated parameter with a corresponding parameter previously calculated from an image of a tissue known to be a third type of tissue;
(G) whether the calculated parameter is relatively similar to a parameter previously calculated from an image of a tissue known to be of the third type of tissue different from the other compared parameters; And (h) comparing the calculated parameter to a parameter previously calculated from an image of a tissue known to be of the third type of tissue different from the other compared parameters. The method of claim 1, comprising determining that the unknown tissue is more likely to be the third type of tissue when similar to. The method of claim 1, wherein the comparing operation is performed using at least one neural network. A first neural network trained with an image known to have pixel data having a first eigenmode data reduction and a known neural network having pixel data having a second eigenmode data reduction Using at least two neural networks, including a second neural network trained with images, wherein the calculated parameters are provided to both the first neural network and the second neural network. The method according to claim 5. At least two parameters are calculated in said calculating operation, and at least two neural networks are used in said comparing operation;
(A) supplying the first parameter to a first neural network trained using first parameters calculated from the image of the first type of tissue and the image of the second type of tissue; And
(B) supplying the second parameter to a second neural network trained using a second parameter calculated from the image of the first type of tissue and the image of the second type of tissue; The method of claim 1, wherein: The method of claim 1, wherein the first type includes a type of tissue and the second type includes a different type of tissue. The first type is a first type for a tissue, the second type is a second type for a tissue, and the method is a type wherein the neural network is a trained type, and The method of claim 1, comprising at least one additional type, wherein the unknown tissue is determined to be more likely to be a type of tissue most similar to the calculated parameter. . The second type further includes at least one additional type to provide a third type, wherein the comparing further comprises comparing the calculated parameter with the third type. Determining that the at least one tissue represented in the image is relatively similar to any one of the first type, the second type, and the third type The method of claim 1, comprising determining whether or not. The method of claim 1, wherein the image is obtained from an exposed surface of a portion of the tissue. The method according to claim 11, wherein the exposed surface is colored by nuclear staining before the image is acquired. The method of claim 1, wherein the image data includes a third spatial extent, and wherein calculating the parameter comprises calculating a parameter that traverses all three spatial extents. The method of claim 1, wherein the image is obtained at a location from a biological tissue. The method according to claim 1, wherein the tissue is an animal tissue. The method of claim 15, wherein the first type includes a type of human tissue and the second type includes a type of human tissue that does not include the type of tissue. 17. The method of claim 16, wherein the first type is adrenal tissue. 17. The method of claim 16, wherein the first type is arterial tissue. 17. The method of claim 16, wherein the first type is bladder tissue. 17. The method of claim 16, wherein the first type is bone tissue. 17. The method of claim 16, wherein the first type is bone marrow tissue. 17. The method of claim 16, wherein the first type is brain tissue. 17. The method of claim 16, wherein the first type is breast tissue. 17. The method of claim 16, wherein the first type is bronchial tissue. 17. The method of claim 16, wherein the first type is colon tissue. 17. The method of claim 16, wherein the first type is duodenal tissue. 17. The method of claim 16, wherein the first type is ear tissue. 17. The method of claim 16, wherein the first type is epididymal tissue. 17. The method of claim 16, wherein the first type is esophageal tissue. 17. The method of claim 16, wherein the first type is ocular tissue. 17. The method of claim 16, wherein the first type is eyelid tissue. 17. The method of claim 16, wherein the first type is fallopian tube tissue. 17. The method of claim 16, wherein the first type is fibrocartilage tissue. 17. The method of claim 16, wherein the first type is gallbladder tissue. 17. The method of claim 16, wherein the first type is ganglion tissue. 17. The method of claim 16, wherein the first type is heart tissue. 17. The method of claim 16, wherein the first type is an inflammatory tissue. 17. The method of claim 16, wherein the first type is kidney tissue. 17. The method of claim 16, wherein the first type is laryngeal tissue. 17. The method of claim 16, wherein the first type is lip tissue. 17. The method of claim 16, wherein the first type is liver tissue. 17. The method of claim 16, wherein the first type is lung tissue. 17. The method of claim 16, wherein the first type is lymph node tissue. 17. The method of claim 16, wherein the first type is lymphoid tissue. 17. The method of claim 16, wherein the first type is nasal tissue. 17. The method of claim 16, wherein the first type is neural tissue. 17. The method of claim 16, wherein the first type is ovarian tissue. 17. The method of claim 16, wherein the first type is pancreatic tissue. 17. The method of claim 16, wherein the first type is parathyroid tissue. 17. The method of claim 16, wherein the first type is parotid tissue. 17. The method of claim 16, wherein the first type is penile tissue. 17. The method of claim 16, wherein the first type is peritoneal tissue. 17. The method of claim 16, wherein the first type is pineal tissue. 17. The method of claim 16, wherein the first type is pituitary tissue. 17. The method of claim 16, wherein the first type is placental tissue. 17. The method of claim 16, wherein the first type is pleural tissue. 17. The method of claim 16, wherein the first type is prostate tissue. 17. The method of claim 16, wherein the first type is salivary gland tissue. 17. The method of claim 16, wherein the first type is seminal vesicle tissue. 17. The method of claim 16, wherein the first type is skeletal muscle tissue. 17. The method of claim 16, wherein the first type is skin tissue. 17. The method of claim 16, wherein the first type is smooth muscle tissue. 17. The method of claim 16, wherein the first type is soft tissue. 17. The method of claim 16, wherein the first type is spinal cord tissue. 17. The method of claim 16, wherein said first type is spleen tissue. 17. The method of claim 16, wherein the first type is gastric tissue. 17. The method of claim 16, wherein the first type is synovial tissue. 17. The method of claim 16, wherein the first type is testicular tissue. 17. The method of claim 16, wherein the first type is thymic tissue. 17. The method of claim 16, wherein the first type is thyroid tissue. 17. The method of claim 16, wherein the first type is tongue tissue. 17. The method of claim 16, wherein the first type is tonsil tissue. 17. The method of claim 16, wherein the first type is tooth tissue. 17. The method of claim 16, wherein the first type is tracheal tissue. 17. The method of claim 16, wherein the first type is ureteral tissue. 17. The method of claim 16, wherein the first type is urethral tissue. 17. The method of claim 16, wherein the first type is uterine tissue. 17. The method of claim 16, wherein the first type is vaginal tissue. 17. The method of claim 16, wherein the first type is venous tissue. 17. The method of claim 16, wherein the first type is vascular tissue. A computer method using an image of a tissue containing cells of an organism to determine whether a first tissue structure is present,
(A) receiving a tissue pixel data image, the tissue pixel data image having each pixel of the pixel data image having image intensity value data represented by at least six significant bits;
(B) selecting at least one analysis window of the pixel data of the image, the analysis window representing tissue having a minimum dimension of at least about 60 microns;
(C) from a two-dimensional grid of pixels within a window having a minimum range of at least 6 pixels for the provision of the calculated parameters, the image intensity value having at least 6 significant bits for each pixel. Calculating from the pixel data for the analysis window at least one parameter constituting a measurement of a two-dimensional pattern across at least two spatial extents of the data;
(D) comparing the calculated parameters with at least two different corresponding parameters pre-calculated from images of tissue known to include images of at least two different types of tissue structures, whereby at least Providing a first type and a second type; and (e) determining whether the image includes a tissue structure relatively similar to any of the first type and the second type. Computer techniques, including: A computer readable data carrier comprising a computer program running on a computer for causing a computer to perform the method of claim 1. The method of claim 1, wherein the first type of tissue structure is a single type of tissue structure, and the second type of tissue is a plurality of other structures of the single type of tissue. . further,
(G) comparing the calculated parameter with a corresponding parameter previously calculated from an image of a tissue known to include a third type of tissue structure;
(H) determining whether the calculated parameter is relatively similar to a parameter previously calculated from a tissue known to include the third type of tissue structure different from the other compared parameters. Determining; and (i) said calculated parameter is relatively similar to a parameter previously calculated from a tissue known to comprise said third type of tissue structure different from the other compared parameters 82. The method of claim 81, wherein when determining, determining that there is a high likelihood that the tissue includes the second tissue structure. The method of claim 81, wherein the comparing operation is performed using at least one neural network. A first neural network trained with an image known to have pixel data having a first eigenmode data reduction, and a second neural network trained with image data having a second eigenmode data reduction And at least two neural networks, including a second neural network trained with the image of the above, wherein the calculated parameters are provided to both the first neural network and the second neural network. 86. The method of claim 85. Two or more parameters are calculated in the calculating operation, and two or more neural networks are used in the comparing operation;
(A) supplying the first parameter to a first neural network trained using a first parameter calculated from an image of a tissue including the tissue structure and an image of a tissue not including the tissue structure; ,
(B) supplying the second parameter to a second neural network trained using a second parameter calculated from an image of the tissue including the tissue structure and an image of the tissue not including the tissue structure; 85. The method of claim 84. The method includes at least one additional type in which the neural network has been trained, and the tissue structure is any type of structure of the first type, the second type, and the additional type. 83. The method of claim 81, wherein a determination is made that the likelihood is high. The first type includes a first type for a tissue, the second type includes a second type for a tissue, and the method includes the neural network is a trained type, the method comprising: Determining that the at least one tissue structure of the image includes at least one additional type, including additional types, and that it is highly certain that the tissue structure is of a type most similar to the calculated parameter; 82. The method of claim 81, wherein: 83. The method of claim 81, wherein the image is obtained from an exposed surface of a portion of the tissue. 90. The method according to claim 90, wherein the exposed surface is colored by nuclear staining before the image is acquired. 83. The method of claim 81, wherein the tissue is human tissue and the tissue structure has a unique pattern indicative of disease. 93. The method of claim 92, wherein said disease is a disease exhibited by a proteinaceous accumulation. 93. The method of claim 92, wherein said disease is a disease indicated by lipid accumulation. 93. The method of claim 92, wherein the disease is a disease indicated by a crystallin deposit. 93. The method of claim 92, wherein said disease is a disease indicated by a nucleic acid deposit. 93. The method according to claim 92, wherein said disease is a disease indicated by glycogen accumulation. 82. The method according to claim 81, wherein the tissue is a human body tissue obtained from Table 2, and the tissue structure is a structure obtained from Table 2 or a basic structure. 83. The method of claim 81, wherein the image is obtained at a location from a biological tissue. The method of claim 81, further comprising identifying a set of neighboring pixels representing the tissue structure. (A) before the image is obtained, a label is added to the tissue;
(B) Then, the pixel where the marker appears in the image is confirmed by computer analysis,
(C) After the pixel representing the tissue structure is confirmed, the position of the pixel representing the marker and the position of the pixel representing the tissue structure are compared, and the correlation between the two positions is determined. Item 100. The method according to Item 100. 102. The method of claim 101, wherein the label attaches a label to a gene product. 102. The method of claim 101, wherein the label attaches a label to a drug. The method according to claim 101, wherein the label attaches a label to the antibody. The method according to claim 101, wherein the label attaches a label to the ligand. 102. The method of claim 101, wherein the size of the landmark for at least one of the tissue structures having the landmark is measured by computer analysis of the image. The method of claim 81, wherein the image data includes a third spatial extent, and wherein calculating the parameter is calculating a parameter that traverses all three spatial extents. A computer method of processing an image of a biological tissue of a certain tissue type to determine whether the tissue structure contains a component,
(A) receiving a pixel data image of a tissue and selecting pixel data of an analysis window from the pixel data image;
(B) calculating at least one parameter from the pixel data for the analysis window to provide a calculated parameter;
(C) the calculated parameters and corresponding parameters calculated in advance from an image of a tissue type known to include the tissue structure and an image of a tissue type known not to include the tissue structure. Comparing with the calculated corresponding parameters,
(D) the calculated parameter is relatively more pre-calculated from a tissue known to include the tissue structure than a parameter pre-calculated from a tissue known not to include the tissue structure. When similar, determining that the analysis window is likely to include the tissue structure;
(E) the calculated parameter is relatively more pre-calculated from a tissue known not to include the tissue structure than a parameter pre-calculated from a tissue known to include the tissue structure. When similar, determining that the analysis window is more likely not to include the tissue structure;
(F) confirming pixels within the boundaries of the structure when the analysis window is highly certain that it contains the tissue structure; and (g) analyzing the pixel color intensity to indicate the presence of the component. Computing whether a pixel within the boundary represents. Further, before calculating the parameters for the analysis window, it is determined by analyzing pixel color intensity whether or not pixels within the analysis window represent characteristics indicative of the presence of the component. 109. The method of claim 108, comprising: when does not represent the characteristic, continuing with another analysis window of the image. 110. The method of claim 108, further comprising: identifying, within the analysis window, a substructure within the structure, and determining whether a pixel within the substructure represents a property indicative of the presence of the component. The described technique. 109. The method of claim 108, further comprising identifying a clustered pixel representing at least one cell nucleus closest to a pixel indicating the presence of the component, and identifying, by image recognition, a cell type represented by the clustered pixel. Technique. 109. The method of claim 108, wherein the component is made visible in the image by the addition of a label. The method of claim 111, wherein the sign is made visible in the image by the addition of a sign. 109. The method of claim 108, wherein said component is a gene product. 111. The method of claim 108, wherein the component is a drug. 109. The method of claim 108, wherein said component is an antibody. 109. The method of claim 108, wherein said component is a ligand. 109. The method of claim 108, wherein the magnitude of the component is measured by computer analysis of the image. A computer technique for processing an image of a biological tissue of a tissue type to determine whether the component is located in the tissue structure,
(A) receiving a pixel data image of the tissue, and analyzing a pixel color intensity to identify a group of one or more adjacent pixels representing a characteristic indicative of the presence of the component;
(B) selecting an analysis window from the pixels of the image around the group of pixels;
(C) calculating at least one parameter from pixel data within the analysis window to provide a calculated parameter;
(D) the calculated parameters and the corresponding parameters calculated in advance from the image of the tissue of the tissue type known to include the tissue structure and the image of the tissue type known not to include the tissue structure. Comparing with corresponding parameters calculated in advance from
(E) the calculated parameter is relatively more pre-calculated from a tissue known to include the tissue structure than a parameter pre-calculated from a tissue known not to include the tissue structure. When similar, determining that the analysis window is likely to include the component in the tissue structure;
(F) the calculated parameter is relatively more pre-calculated from a tissue known not to include the tissue structure than a parameter pre-calculated from a tissue known to include the tissue structure. When similar, a computer method, comprising determining that the analysis window is more likely not to include the component in the tissue structure. further,
(G) identifying the pixels within the boundaries of the structure when the analysis window is highly certain that it contains the components in the tissue structure; and (h) analyzing the pixel color intensity to determine 120. The method of claim 119, comprising determining whether a pixel within the boundary represents a property indicating presence. Further comprising: identifying a clustered pixel representing a cell nucleus closest to the group of pixels indicating the presence of the component; and performing image recognition of the clustered pixel to identify a cell type of the clustered pixel. Item 119. The method according to Item 119. 120. The method of claim 119, wherein said component is made identifiable by the addition of a label. The method of claim 122, wherein the sign is made identifiable by the addition of a sign. 120. The method of claim 119, wherein said component is a gene product. 120. The method of claim 119, wherein said component is a drug. 120. The method of claim 119, wherein said component is an antibody. 120. The method of claim 119, wherein said component is a ligand. 120. The method of claim 119, wherein the magnitude of the component is measured by computer analysis of the image. A computer method using an image of at least one cell from a living organism to determine a classification of a cell nucleus,
(A) a pixel data image of at least one cell nucleus, wherein the pixel data image represents at least one image chunk of adjacent pixels, the image chunk having a minimum dimension equal to the dimension of the cell nucleus; Receiving,
(B) an adjacent pixel having at least 6 pixels of the smallest dimension having image intensity value data represented by at least 6 significant bits, wherein said pixel cluster comprises at least 24 separate adjacent pixels of nuclear material; Selecting at least one analysis window of pixel data from within the image;
(C) from a two-dimensional grid of pixels within an analysis window having at least six pixels of the smallest dimension for the provision of the calculated parameters, said image intensity having at least six significant bits for each pixel. Calculating from the pixel data of the analysis window at least one parameter constituting a measurement of the pattern across at least two spatial ranges of the value data;
(D) comparing the calculated parameter with at least two different corresponding parameters pre-calculated from an image of a cell nucleus known to be at least two different types of cell nuclei, whereby at least the first Providing a type and a second type; and (e) determining whether at least one cell nucleus represented in the image mass is relatively similar to any of the first type and the second type. Computer techniques, including making decisions. A computer readable data carrier comprising a computer program running on a computer for causing a computer to perform the method of claim 1. The method of claim 1, wherein the first type includes a single type of cell nucleus and the second type includes a plurality of cell nuclei that do not include the single type. The second type further includes at least one additional type for providing a third type, wherein the comparing further comprises comparing the calculated parameter with the third type. Determining that the at least one cell nucleus represented in the image is relatively similar to any one of the first type, the second type, and the third type. 130. The method of claim 129, comprising determining whether or not. 130. The method of claim 129, wherein the comparing operation is performed using at least one neural network. A first neural network trained with an image known to have pixel data having a first eigenmode data reduction and a known neural network having pixel data having a second eigenmode data reduction Using at least two neural networks, including a second neural network trained with images, wherein the calculated parameters are provided to both the first neural network and the second neural network. 135. The method according to claim 133. 130. The method of claim 129, wherein the first type comprises a first type for a cell nucleus, and wherein the second type comprises a second type for a cell nucleus that is different than the first type for a cell nucleus. Technique. The first type includes a first type for a cell nucleus, the second type includes a second type for a cell nucleus, and the method includes applying the neural network to a trained type, the method comprising: A determination is made that there is a high likelihood that the at least one cell nucleus of the cluster of pixels is of a type that most closely resembles the calculated parameters, including at least one additional type including a typical type. 144. The method of claim 133. 130. The method according to claim 129, wherein the image is obtained from a part of an exposed surface of a tissue of a plurality of cells having a fixed relationship with each other. 138. The method of claim 137, wherein the exposed surface is colored by nuclear staining before the image is obtained. The at least one cell nucleus of the first type is in a normal cell of a human somatic cell type, and the second type of cell nucleus is in a cell of a human somatic cell type appearing in an image showing an abnormality of the cell. 129. The method according to item 129. 140. The method of claim 139, wherein the abnormality is indicative of a reproductive disease. 140. The method of claim 139, wherein the abnormality is indicative of tumor formation. 140. The method of claim 139, wherein the abnormality is indicative of an infectious disease. 140. The method of claim 139, wherein the abnormality is inflammatory. 140. The method of claim 139, wherein the abnormality indicates a degenerative disease. 140. The method of claim 139, wherein the abnormality is indicative of an autoimmune disease. 140. The method of claim 139, wherein the abnormality is indicative of a chemical injury. 140. The method of claim 139, wherein the abnormality is indicative of anoxic injury. 140. The method of claim 139, wherein the abnormality is indicative of a metabolic disease. 140. The method of claim 139, wherein the abnormality is indicative of a genetic disease. 140. The method of claim 139, wherein the abnormality is indicative of a disease listed in Table 1. 130. The method of claim 129, wherein the at least one cell nucleus of the first type is a human somatic cell listed in Table 2. 130. The method of claim 129, wherein the image is obtained from at least one separated cell. 148. The method of claim 147, wherein the separated cells are at least one of blood cells, PAP smear cells, and inflammatory cells. 130. The method of claim 129, wherein the at least one cell nucleus of the first type is a cell nucleus fixed with respect to surrounding tissue and the at least one cell nucleus of the second type is a nucleus of an inflammatory cell. . 157. The method of claim 154, further comprising counting the number of inflammatory cells and reporting a measurement based on the number of inflammatory cells. 130. The method of claim 129, wherein the first type of cell nucleus is a cell nucleus of a first type of inflammatory cell, and the second type of cell nucleus is a cell nucleus of a second type of inflammatory cell. 157. The method of claim 156, further comprising counting the number of inflammatory cells of each type and reporting a measurement based on the number of inflammatory cells of each type. The at least one nucleus of the first type is a nucleus of an inflammatory cell of a first type; the at least one nucleus of the second type is a nucleus of an inflammatory cell of a second type; 142. The method of claim 142, wherein the at least one cell nucleus of the general type is a cell nucleus of at least one additional type of inflammatory cell. 138. The method of claim 137, wherein said first type of cell nucleus is a cell nucleus of a first cell type, and said second type of cell nucleus is a cell nucleus of a second cell type. 150. The method of claim 149, wherein the first cell type substantially comprises cells having a fixed relationship to one another, and wherein the second cell type comprises at least one inflammatory cell. 130. The method according to claim 129, wherein the image is obtained in situ from a plurality of cell tissues having a fixed relationship with each other in a living body. (A) before the image is obtained, a label is added to at least one cell from a living body;
(B) Then, the position where the marker appears in the image is confirmed by the computer analysis,
(C) after the image mass with a certain type of cell nucleus is shown, the position of the label and the pre-calculated position of the label for the type are compared to determine the correlation between the two. 130. The method of claim 129, wherein the method is performed. 163. The method of claim 162, wherein said label is a gene product. 163. The method of claim 162, wherein the label is a drug. 163. The method according to claim 162, wherein the label is an antibody. 163. The method according to claim 162, wherein the label is a ligand. 163. The method of claim 162, wherein for at least one cell nucleus represented in an image mass representing a label, the size of the label is measured by computer analysis of the image. 130. The method of claim 129, wherein said cells are cells of an animal tissue. 169. The method of claim 168, wherein said cells are cells of a human body tissue. 130. The method of claim 129, wherein the image data includes a third spatial extent, and calculating the parameter is to calculate at least one parameter across all three spatial extents. A computer method for processing an image of a tissue of a living body to indicate a position of a component and confirm a cell type including the component,
(A) receiving a pixel data image of a tissue including a plurality of cells having a fixed relationship with each other and representing at least two pixel clusters representing at least one cell nucleus;
(B) confirming one or more adjacent pixels of a group representing a characteristic indicating the presence of the component by analyzing a pixel color intensity of the pixel data image;
(C) pixel data for the pixels in the nearest pixel to identify the nearest pixel cluster in the pixel data image by image recognition and to provide calculated parameters; Calculating at least one parameter from
(D) providing the calculated parameters and at least one corresponding parameter and a second cell type previously calculated from a cell nucleus known to be a cell type nucleus for providing a first cell type; Comparing with at least one corresponding parameter previously calculated from a cell nucleus known not to be of the cell type to
(E) comparing the calculated parameters with at least two corresponding parameters pre-calculated from images of cell nuclei known to be at least two different types of cell nuclei, whereby at least the first type And providing a second type; and (f) determining whether at least one cell nucleus represented in the image mass is relatively similar to any of the first type and the second type. Computer techniques, including making decisions. 172. The method of claim 171, wherein the component is made visible in the image by the addition of a label. 172. The method of claim 172, wherein the sign is made visible in the image by the addition of a sign. 172. The method of claim 171, wherein said component is a gene product. 172. The method of claim 171, wherein the component is a drug. 172. The method of claim 171, wherein said component is an antibody. 172. The method of claim 171, wherein said component is a ligand. 172. The method of claim 171, wherein the magnitude of the component is measured by computer analysis of pixel intensities of pixels in the group. A computer method of processing an image of a biological tissue to determine whether the cell type contains a component,
(A) receiving a pixel data image of a tissue including a plurality of cells in a fixed relationship with each other, the pixel data image representing all of at least one block pixel representing at least one cell nucleus;
(B) defining a boundary of a region including pixels within a distance of the block of pixels;
(C) determining, by analyzing the pixel color intensity, whether a pixel within the bounds exhibits a characteristic indicative of the presence of the component;
(D) calculating at least one parameter for a pixel within the pixel chunk to provide a calculated parameter when the region includes a pixel representing the presence;
(E) comparing the calculated parameter with at least two different corresponding parameters pre-calculated from an image of the cell nucleus known to be at least two different types of cell nuclei, whereby at least the first Providing a type and a second type; and
(F) A computer method, comprising determining whether at least one cell nucleus represented in the image block is relatively similar to the first type or the second type. 179. The method of claim 179, wherein the distance is zero such that the region is coextensive with the pixel mass. 180. The method of claim 179, wherein the distance is greater than zero such that the region is greater than the pixel chunk. 180. The method of claim 179, wherein the component is made visible in the image by the addition of a label. 183. The method of claim 182, wherein the sign is made visible in the image by the addition of a sign. 172. The method of claim 171, wherein said component is a gene product. 172. The method of claim 171, wherein the component is a drug. 172. The method of claim 171, wherein said component is an antibody. 172. The method of claim 171, wherein said component is a ligand. 172. The method of claim 171, wherein the magnitude of the component is measured by computer analysis of the image.

Images