TWM653792U - AI digital pathology image recognition system - Google Patents
AI digital pathology image recognition system Download PDFInfo
- Publication number
- TWM653792U TWM653792U TW112214435U TW112214435U TWM653792U TW M653792 U TWM653792 U TW M653792U TW 112214435 U TW112214435 U TW 112214435U TW 112214435 U TW112214435 U TW 112214435U TW M653792 U TWM653792 U TW M653792U
- Authority
- TW
- Taiwan
- Prior art keywords
- image recognition
- ihc
- module
- image
- fish
- Prior art date
Links
- 230000007170 pathology Effects 0.000 title claims abstract description 32
- 238000004891 communication Methods 0.000 claims abstract description 42
- 238000002509 fluorescent in situ hybridization Methods 0.000 claims abstract description 28
- 238000003745 diagnosis Methods 0.000 claims abstract description 24
- 206010028980 Neoplasm Diseases 0.000 claims description 19
- 201000011510 cancer Diseases 0.000 claims description 18
- 238000010186 staining Methods 0.000 claims description 16
- 238000013136 deep learning model Methods 0.000 claims description 8
- 229940079593 drug Drugs 0.000 claims description 4
- 239000003814 drug Substances 0.000 claims description 4
- 238000003709 image segmentation Methods 0.000 claims description 4
- 238000003364 immunohistochemistry Methods 0.000 description 91
- 210000004027 cell Anatomy 0.000 description 23
- 238000004458 analytical method Methods 0.000 description 15
- 108700020302 erbB-2 Genes Proteins 0.000 description 14
- 102100030086 Receptor tyrosine-protein kinase erbB-2 Human genes 0.000 description 12
- 210000001519 tissue Anatomy 0.000 description 12
- 101150054472 HER2 gene Proteins 0.000 description 11
- 101001012157 Homo sapiens Receptor tyrosine-protein kinase erbB-2 Proteins 0.000 description 11
- 238000000034 method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 8
- 230000001575 pathological effect Effects 0.000 description 8
- 238000004393 prognosis Methods 0.000 description 5
- 206010006187 Breast cancer Diseases 0.000 description 4
- 208000026310 Breast neoplasm Diseases 0.000 description 4
- 210000000349 chromosome Anatomy 0.000 description 4
- 201000010099 disease Diseases 0.000 description 4
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 4
- 238000003384 imaging method Methods 0.000 description 4
- 238000011532 immunohistochemical staining Methods 0.000 description 4
- 238000003860 storage Methods 0.000 description 4
- 239000000427 antigen Substances 0.000 description 3
- 102000036639 antigens Human genes 0.000 description 3
- 108091007433 antigens Proteins 0.000 description 3
- 210000004718 centriole Anatomy 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000014509 gene expression Effects 0.000 description 3
- 230000000984 immunochemical effect Effects 0.000 description 3
- 238000011160 research Methods 0.000 description 3
- 230000004544 DNA amplification Effects 0.000 description 2
- 101001121408 Homo sapiens L-amino-acid oxidase Proteins 0.000 description 2
- 102100026388 L-amino-acid oxidase Human genes 0.000 description 2
- 230000005856 abnormality Effects 0.000 description 2
- 238000009739 binding Methods 0.000 description 2
- 210000000170 cell membrane Anatomy 0.000 description 2
- 238000002651 drug therapy Methods 0.000 description 2
- 239000012634 fragment Substances 0.000 description 2
- 230000002068 genetic effect Effects 0.000 description 2
- 238000007689 inspection Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 230000001737 promoting effect Effects 0.000 description 2
- 108090000623 proteins and genes Proteins 0.000 description 2
- 102000004169 proteins and genes Human genes 0.000 description 2
- 238000004904 shortening Methods 0.000 description 2
- 238000012360 testing method Methods 0.000 description 2
- 102000009465 Growth Factor Receptors Human genes 0.000 description 1
- 108010009202 Growth Factor Receptors Proteins 0.000 description 1
- 241000282412 Homo Species 0.000 description 1
- 206010027476 Metastases Diseases 0.000 description 1
- 230000027455 binding Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 229940044683 chemotherapy drug Drugs 0.000 description 1
- 230000018044 dehydration Effects 0.000 description 1
- 238000006297 dehydration reaction Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 230000008472 epithelial growth Effects 0.000 description 1
- 238000011156 evaluation Methods 0.000 description 1
- 239000007850 fluorescent dye Substances 0.000 description 1
- 238000003208 gene overexpression Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- PCHJSUWPFVWCPO-UHFFFAOYSA-N gold Chemical compound [Au] PCHJSUWPFVWCPO-UHFFFAOYSA-N 0.000 description 1
- 238000007901 in situ hybridization Methods 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 230000009401 metastasis Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000012188 paraffin wax Substances 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001850 reproductive effect Effects 0.000 description 1
- 230000009870 specific binding Effects 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 238000001356 surgical procedure Methods 0.000 description 1
- 208000024891 symptom Diseases 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 210000004881 tumor cell Anatomy 0.000 description 1
Abstract
本新型提供一種AI數位病理影像辨識系統，上述AI數位病理影像辨識系統包括伺服器及中控裝置。上述中控裝置包括多個模組：通訊模組、IHC影像辨識模組、判斷模組、FISH影像辨識模組及病理資料庫。通訊模組用以傳輸資訊。IHC影像辨識模組對IHC影像進行分類辨識，產生IHC影像辨識結果。判斷模組根據IHC影像辨識結果產生診斷與治療方針建議。病理資料庫用以儲存由實體玻片拍攝所得之FISH影像和IHC影像，以及上述IHC和FISH影像之辨識結果 。FISH影像辨識模組，當IHC影像的分級為2+時，接收FISH影像並進行辨識，產生FISH影像辨識結果。The present invention provides an AI digital pathology image recognition system, which includes a server and a central control device. The central control device includes multiple modules: a communication module, an IHC image recognition module, a judgment module, a FISH image recognition module and a pathology database. The communication module is used to transmit information. The IHC image recognition module classifies and recognizes IHC images to generate IHC image recognition results. The judgment module generates diagnosis and treatment policy recommendations based on the IHC image recognition results. The pathology database is used to store FISH images and IHC images obtained by photographing physical slides, as well as the recognition results of the above-mentioned IHC and FISH images. The FISH image recognition module receives and identifies the FISH image when the IHC image is graded as 2+, and generates a FISH image recognition result.
Description
一種AI數位病理影像辨識系統，特別適用於分析辨識IHC影像和FISH影像的AI數位病理影像辨識系統。 An AI digital pathology image recognition system, particularly suitable for analyzing and identifying IHC images and FISH images.
在醫學診斷和生物醫學研究中，IHC(免疫組織化學染色)和FISH(螢光原位雜合)技術用於檢驗和分析組織和細胞中的特定蛋白質或基因異常。而現今，IHC影像和FISH影像的分析主要依賴玻片或由玻片拍攝之影像，執行人工辨識和評估。病理科醫師與技術員往往需要在顯微鏡，螢光顯微鏡或者影像上進行長時間的觀察計數，拍攝與手動分析以識別和分析特定的病徵、蛋白質表達或基因異常。 In medical diagnosis and biomedical research, IHC (immunohistochemistry) and FISH (fluorescence in situ hybridization) techniques are used to detect and analyze specific proteins or genetic abnormalities in tissues and cells. Today, the analysis of IHC and FISH images mainly relies on slides or images taken from slides for manual identification and evaluation. Pathologists and technicians often need to observe, count, photograph, and manually analyze under microscopes, fluorescent microscopes, or images for a long time to identify and analyze specific symptoms, protein expressions, or genetic abnormalities.
然而，以人工分析IHC和FISH玻片與其影像，容易受到觀測者間差異與人為主觀因素，訓練程度之影響，降低結果之復現性與一致性。此外，對玻片與其影像進行人工分析需要大量的時間和精力，若需大量處理，需要龐大受過良好訓練之人力。另FISH玻片亦有壽命問題，其螢光訊號無法長時間保存，更加大機構判讀的時間壓力，且受限於人工計數能力的極限，難以評估腫瘤內部多樣性等特殊情況。 However, manual analysis of IHC and FISH slides and their images is easily affected by differences between observers, subjective factors, and training levels, which reduces the reproducibility and consistency of the results. In addition, manual analysis of slides and their images requires a lot of time and energy. If a large amount of processing is required, a large number of well-trained manpower is required. FISH slides also have a lifespan problem. Their fluorescent signals cannot be preserved for a long time, which further increases the time pressure of institutional interpretation. In addition, due to the limitations of manual counting capabilities, it is difficult to evaluate special situations such as the diversity within the tumor.
因此，如何開發出一種AI數位病理影像辨識系統可解決上述問題，即成為所屬技術領域中有待解決的問題。 Therefore, how to develop an AI digital pathology image recognition system that can solve the above problems has become a problem to be solved in the relevant technical field.
為解決上述問題，根據一實施例，本新型提供一種AI數位病理影像辨識系統，上述系統包括伺服器及中控裝置。 To solve the above problems, according to one embodiment, the present invention provides an AI digital pathological image recognition system, which includes a server and a central control device.
上述伺服器，具有一影像資料庫，上述影像資料庫儲存有一病患之組織檢體經固定，脫水，石蠟包埋，切片與免疫化學染色(Immunohistochemistry,IHC)後製成之IHC玻片，通過顯微鏡或掃描裝置成像拍攝之數位病理影像。上述中控裝置，其包括由電性連接之多個硬體電路所組成的多個硬體模組，上述多個硬體模組包括通訊模組、IHC影像辨識模組、判斷模組及病理資料庫。 The server has an image database, which stores digital pathological images of a patient's tissue specimen that has been fixed, dehydrated, paraffin-embedded, sliced and immunochemically stained (IHC) slides, and imaged and photographed by a microscope or a scanning device. The central control device includes multiple hardware modules composed of multiple electrically connected hardware circuits, and the multiple hardware modules include a communication module, an IHC image recognition module, a judgment module and a pathological database.
上述通訊模組，通訊連接上述伺服器及至少一醫生裝置。上述IHC影像辨識模組，通訊連接上述通訊模組，在透過上述通訊模組接收上述IHC影像後，上述IHC影像辨識模組使用分類深度學習模型，利用影像分類與影像切割對上述IHC影像進行分類與辨識上述IHC影像之癌細胞的染色強度和完全程度，依照癌細胞的染色強度和完全程度進行一分類後，再依照癌細胞的染色強度和完全程度給予上述IHC影像一分級，計算上述IHC影像的癌細胞的上述分級之比例，產生一IHC影像辨識結果，並將上述IHC影像辨識結果透過上述通訊模組傳輸至上述至少一醫生裝置。上述判斷模組，通訊連接上述通訊模組及上述IHC影像辨識模組，當接收到上述IHC影像辨識結果時，上述判斷模組對上述IHC影像辨識結果進行判斷，產生一診斷與治療方針建議，並將上述建議透過上述通訊模組傳輸至上述至少一醫生裝置。上述病理資料庫，通訊連接上述IHC影像辨識模組，用以儲存上述IHC影像辨識結果。 The communication module is communicatively connected to the server and at least one medical device. The IHC image recognition module is communicatively connected to the communication module. After receiving the IHC image through the communication module, the IHC image recognition module uses a classification deep learning model to classify and recognize the staining intensity and completeness of the cancer cells in the IHC image by using image classification and image segmentation. After a classification is performed according to the staining intensity and completeness of the cancer cells, the IHC image is given a grade according to the staining intensity and completeness of the cancer cells, and the ratio of the grade of the cancer cells in the IHC image is calculated to generate an IHC image recognition result, and the IHC image recognition result is transmitted to the at least one medical device through the communication module. The judgment module is communicatively connected to the communication module and the IHC image recognition module. When receiving the IHC image recognition result, the judgment module judges the IHC image recognition result, generates a diagnosis and treatment policy recommendation, and transmits the recommendation to the at least one doctor device through the communication module. The pathology database is communicatively connected to the IHC image recognition module to store the IHC image recognition result.
根據另一實施例，上述分類為高或低。 According to another embodiment, the above classification is high or low.
根據另一實施例，上述分級為0、1+、2+及3+。 According to another embodiment, the above grades are 0, 1+, 2+ and 3+.
根據另一實施例，上述IHC影像辨識模組還包括一通知模組，通訊連接上述通訊模組，上述通訊模組還通訊連接一檢驗人員裝置，當上述IHC影像的上述分級為2+時，上述通知模組透過上述通訊模組傳輸一通知訊息至上述檢驗人員裝置，在檢驗人員對上述病患之上述已經過固定脫水處理與石蠟包埋之組織進行切片，玻片製備與螢光原位雜合(Fluorescent in situ hybridization,FISH)後，通過顯微鏡或掃描裝置成像拍攝所得之上述組織的一FISH影像透過上述檢驗人員裝置儲存在上述伺服器的上述影像資料庫。 According to another embodiment, the IHC image recognition module further includes a notification module, which is communicatively connected to the communication module, and the communication module is also communicatively connected to an inspector device. When the grade of the IHC image is 2+, the notification module transmits a notification message to the inspector device through the communication module. After the inspector slices the fixed, dehydrated and paraffin-embedded tissue of the patient, prepares the slide and performs fluorescent in situ hybridization (FISH), a FISH image of the tissue obtained by imaging and shooting with a microscope or a scanning device is stored in the image database of the server through the inspector device.
根據另一實施例，上述中控裝置還包括一FISH影像辨識模組，通訊連接上述通訊模組、判斷模組及病理資料庫，在透過上述通訊模組接收FISH影像後，FISH影像辨識模組使用分類深度學習模型，利用影像切割對上述FISH影像進行辨識上述FISH影像之紅綠光點，並計算上述FISH影像之紅綠光點之比例，產生一FISH影像辨識結果，並將上述FISH影像辨識結果透過上述通訊模組傳輸至上述至少一醫生裝置。 According to another embodiment, the central control device further includes a FISH image recognition module, which is communicatively connected to the communication module, the judgment module and the pathology database. After receiving the FISH image through the communication module, the FISH image recognition module uses a classification deep learning model and image segmentation to identify the red and green light spots of the FISH image, and calculates the ratio of the red and green light spots of the FISH image to generate a FISH image recognition result, and transmits the FISH image recognition result to the at least one doctor device through the communication module.
根據另一實施例，上述FISH影像辨識結果傳輸至上述判斷模組，上述判斷模組將FISH影像辨識結果與上述IHC影像辨識結果結合進行判斷，產生上述診斷與治療方針建議。 According to another embodiment, the above FISH image recognition results are transmitted to the above judgment module, and the above judgment module combines the FISH image recognition results with the above IHC image recognition results to make a judgment and generate the above diagnosis and treatment policy recommendations.
根據另一實施例，上述診斷與治療方針建議為建議使用標靶藥物或不建議使用標靶藥物。 According to another embodiment, the above-mentioned diagnosis and treatment guidelines recommend the use of targeted drugs or do not recommend the use of targeted drugs.
本新型可主張的功效包括：(1)通過自動化分析病理學影像，協助醫生評估疾病的嚴重程度和預後，並提供診斷與治療方針建議。(2)自動分析IHC和FISH影像，減少人為錯誤，提高診斷和分析的可靠性，可重複性。(3)節約 進行人工分析所需之龐大人力。(4)加大單位工作時間可分析之病人影像數目，縮短臨床醫師治療決策須等待的時間，並推進相關領域之研究。(5)有效管理和存儲大量的影像，包含記錄原本無法永久儲存之FISH玻片和其分析結果，可隨時調閱，復閱與進行外部意見諮詢。 The claimed efficacy of this new technology includes: (1) Assisting doctors in assessing the severity and prognosis of the disease and providing diagnostic and treatment recommendations through automated analysis of pathological images. (2) Automatic analysis of IHC and FISH images to reduce human errors and improve the reliability and repeatability of diagnosis and analysis. (3) Saving the huge manpower required for manual analysis. (4) Increasing the number of patient images that can be analyzed per unit of working time, shortening the waiting time for clinical physicians to make treatment decisions, and promoting research in related fields. (5) Effectively managing and storing a large number of images, including recording FISH slides and their analysis results that were originally unable to be permanently stored, which can be accessed, reviewed, and consulted with external opinions at any time.
100:AI數位病理影像辨識系統 100: AI digital pathology image recognition system
110:伺服器 110: Server
112:影像資料庫 112: Image database
120:中控裝置 120: Central control device
130:通訊模組 130: Communication module
140:IHC影像辨識模組 140:IHC image recognition module
142:通知模組 142: Notification module
150:判斷模組 150: Judgment module
160:FISH影像辨識模組 160:FISH image recognition module
170:病理資料庫 170: Pathology database
200:檢驗人員裝置 200: Inspection personnel device
220:醫生裝置 220: Doctor's Device
300~314:步驟 300~314: Steps
為讓本新型之上述技術和其他目的、特徵、優點與實施例能更簡單明瞭，所附附圖之說明如下：圖1係繪示根據本新型一實施例之一種AI數位病理影像辨識系統之系統架構示意圖。 In order to make the above-mentioned technology and other purposes, features, advantages and embodiments of the present invention simpler and clearer, the attached drawings are described as follows: Figure 1 is a schematic diagram of the system architecture of an AI digital pathological image recognition system according to an embodiment of the present invention.
圖2係繪示根據本新型一實施例之一種AI數位病理影像辨識系統之流程示意圖。 Figure 2 is a schematic diagram showing the process of an AI digital pathological image recognition system according to an embodiment of the present invention.
圖3A係繪示根據本新型一實施例之分級為0的IHC影像之示意圖。 FIG. 3A is a schematic diagram showing an IHC image with a grade of 0 according to an embodiment of the present invention.
圖3B係繪示根據本新型一實施例之分級為1+的IHC影像之示意圖。 FIG. 3B is a schematic diagram showing an IHC image graded as 1+ according to an embodiment of the present invention.
圖3C係繪示根據本新型一實施例之分級為2+的IHC影像之示意圖。 FIG. 3C is a schematic diagram showing an IHC image graded as 2+ according to an embodiment of the present invention.
圖3D係繪示根據本新型一實施例之分級為3+的IHC影像之示意圖。 Figure 3D is a schematic diagram showing an IHC image graded as 3+ according to an embodiment of the present invention.
為更具體說明本新型之各實施例，以下輔以附圖進行說明。應當理解的是，元件被稱為「連接」或「設置」於另一元件時，可以表示元件是直 接位於另一元件上，或者可以也存在中間元件，透過中間元件連接元件與另一元件。相反地，當元件被稱為「直接在另一元件上」或「直接連接到另一元件」時，可以理解的是，此時明確定義了不存在中間元件。 In order to more specifically illustrate the various embodiments of the present invention, the following is a description with the aid of attached drawings. It should be understood that when an element is referred to as being "connected" or "disposed" on another element, it can mean that the element is directly located on the other element, or there may also be an intermediate element through which the element and the other element are connected. Conversely, when an element is referred to as being "directly on another element" or "directly connected to another element", it can be understood that it is clearly defined that there is no intermediate element.
圖1所繪為根據本新型之一實施例之一種AI數位病理影像辨識系統100之系統架構示意圖。在圖1中，AI數位病理影像辨識系統100包括伺服器110及中控裝置120。上述伺服器110，具有一影像資料庫112，上述影像資料庫112儲存有經脫水，石蠟包埋，切片與免疫化學染色IHC)後製成之IHC玻片，通過顯微鏡或掃描裝置成像拍攝之數位病理影像。上述中控裝置120，其包括由電性連接之多個硬體電路所組成的多個硬體模組，上述多個硬體模組包括通訊模組130、IHC影像辨識模組140、判斷模組150及病理資料庫170。 FIG1 is a schematic diagram of the system architecture of an AI digital pathology image recognition system 100 according to an embodiment of the present invention. In FIG1 , the AI digital pathology image recognition system 100 includes a server 110 and a central control device 120. The server 110 has an image database 112, which stores IHC slides made after dehydration, paraffin embedding, sectioning and immunochemical staining (IHC), and digital pathology images imaged and photographed by a microscope or a scanning device. The central control device 120 includes a plurality of hardware modules composed of a plurality of electrically connected hardware circuits, and the plurality of hardware modules include a communication module 130, an IHC image recognition module 140, a judgment module 150 and a pathology database 170.
HER2是人類上皮因子接受體第2蛋白(Human Epidermal Growth Factor Receptor 2)的縮寫。HER2基因是影響乳癌預後的一個重要因素，大約有25%-30%左右的乳癌患者，受到體內癌細胞HER2基因過量表現的影響，他們的癌細胞不僅繁殖能力強，對部份化學治療藥物也容易有抗藥性，造成病患即使接受手術治療，癌細胞仍然有較高復發及轉移的機率，無法長期存活。 HER2 is the abbreviation of Human Epidermal Growth Factor Receptor 2. The HER2 gene is an important factor affecting the prognosis of breast cancer. About 25%-30% of breast cancer patients are affected by the excessive expression of the HER2 gene in their cancer cells. Their cancer cells not only have strong reproductive ability, but are also easily resistant to some chemotherapy drugs. Even if the patients receive surgical treatment, the cancer cells still have a higher chance of recurrence and metastasis, and they cannot survive for a long time.
目前HER2的檢驗方法以免疫組織化學染色法(IHC)及螢光原位雜合(FISH)為主。免疫組織化學染色法是指在抗體上結合可呈色的化學物質，利用抗原和抗體間專一性的結合反應，檢測細胞或組織中是否有目標抗原的存在；此法的優點可測知抗原的表現量。由病理醫師根據細胞膜上之HER2第二型人類上皮細胞生長因子接受器染色強弱及其佔乳癌細胞比例，結果以0、1+、2+、3+呈現，HER2基因位於人類第17對染色體上，約四分之一的乳癌腫瘤細胞會有很多HER2基因的重複片段，這些過度的HER2基因就會製造出更 多的細胞膜上HER2接受器，螢光原位雜交法的檢驗原理就是利用不同螢光去偵測HER2基因以及第17對染色體中心粒及HER2基因的重複片段，算出HER2基因相對第17對染色體中心粒螢光染色之比率，以拷貝倍數(copy number)來呈現。目前以免疫化學染色法3+或螢光原位雜交法的檢驗陽性(拷貝倍數2.0或2.2以上)才能被確定為HER2陽性。 Currently, the main methods for testing HER2 are immunohistochemical staining (IHC) and fluorescent in situ hybridization (FISH). Immunohistochemical staining refers to the use of specific binding reactions between antigens and antibodies to detect the presence of target antigens in cells or tissues by binding colored chemicals to antibodies. The advantage of this method is that it can measure the amount of antigen expression. The pathologist will present the results as 0, 1+, 2+, or 3+ based on the staining intensity of the HER2 type II human epithelial growth factor receptor on the cell membrane and its proportion in breast cancer cells. The HER2 gene is located on the 17th chromosome of humans. About a quarter of breast cancer tumor cells have many repeated fragments of the HER2 gene. These excessive HER2 genes will produce more HER2 receptors on the cell membrane. The principle of the fluorescent in situ hybridization method is to use different fluorescence to detect the HER2 gene and the centriole of the 17th chromosome and the repeated fragments of the HER2 gene, calculate the ratio of the HER2 gene to the fluorescent staining of the centriole of the 17th chromosome, and present it as the copy number. Currently, only a positive test (copy number 2.0 or above) using immunochemical staining 3+ or fluorescent in situ hybridization can be determined as HER2 positive.
檢驗人員將一病患之透過免疫組織化學染色法所得之一組織切片，透過顯微鏡取像軟體或掃描裝置擷取IHC影像，由醫師選取細胞型態清楚、染色均勻且平整的玻片，拍攝具代表性的特定視野，將IHC影像透過檢驗人員裝置200傳輸至伺服器110內的影像資料庫112儲存。 The inspector takes a tissue slice obtained from a patient through immunohistochemical staining, and captures the IHC image through a microscope imaging software or a scanning device. The doctor selects a glass slide with clear cell morphology, uniform staining, and flatness, and takes a representative specific field of view. The IHC image is transmitted to the image database 112 in the server 110 through the inspector device 200 for storage.
當要進行IHC影像的影像辨識分析時，AI數位病理影像辨識系統100透過通訊模組130從伺服器110的影像資料庫112中傳輸至IHC影像辨識模組140。 When image recognition analysis of IHC images is to be performed, the AI digital pathology image recognition system 100 transmits the image database 112 of the server 110 to the IHC image recognition module 140 via the communication module 130.
IHC影像辨識模組140通訊連接通訊模組130。在透過通訊模組130接收IHC影像後，IHC影像辨識模組140使用分類深度學習模型，先利用影像分類依其癌細胞的染色強度和完全程度對上述IHC影像進行分類，將IHC影像分為高和低兩類，再利用影像切割依其癌細胞的染色強度和完全程度對上述IHC影像進行辨識，將IHC影像分級。上述分級為0、1+、2+及3+。其中0和1+的分類為低；2+及3+的分類為高。各分級IHC影像請參閱圖3A~3D。 The IHC image recognition module 140 is connected to the communication module 130. After receiving the IHC image through the communication module 130, the IHC image recognition module 140 uses the classification deep learning model to first classify the IHC image according to the staining intensity and completeness of the cancer cells using image classification, and divide the IHC image into two categories: high and low. Then, the IHC image is identified according to the staining intensity and completeness of the cancer cells using image cutting, and the IHC image is graded. The above grades are 0, 1+, 2+ and 3+. Among them, 0 and 1+ are classified as low; 2+ and 3+ are classified as high. Please refer to Figures 3A~3D for each grade of IHC images.
分級的具體標準如下表1所示。 The specific standards for grading are shown in Table 1 below.
HER2免疫化學染色法的分級為2+的IHC影像，由於各家醫院使用的HER-2/NEU單株抗體不同，或是組織處理上有所差異，陽性率也會有差距。因此，對於染色結果意義不明確的分級為2+的IHC影像，必須採用螢光原位雜交法進一步測定它們是否有HER2基因增幅，螢光原位雜交法是目前用來判定有無HER-2/NEU基因過度表現的黃金標準。根據目前HER2的檢測方式。免疫組織化學染色結果分級為3+的IHC影像，或者是螢光原位雜交結果顯示有HER2基因增幅，定義為HER2陽性。分級為0或1+的IHC影像定義為HER2陰性。 For IHC images graded as 2+ by HER2 immunochemical staining, the positive rate varies due to the different HER-2/NEU monoclonal antibodies used by different hospitals or differences in tissue processing. Therefore, for IHC images graded as 2+ with unclear staining results, fluorescent in situ hybridization must be used to further determine whether they have HER2 gene amplification. Fluorescent in situ hybridization is currently the gold standard for determining whether there is HER-2/NEU gene overexpression. According to the current HER2 detection method. IHC images with immunohistochemical staining results graded as 3+, or fluorescent in situ hybridization results showing HER2 gene amplification, are defined as HER2 positive. IHC images graded as 0 or 1+ are defined as HER2 negative.
IHC影像辨識模組140將分級為0、1+、2+或3+的IHC影像計算癌細胞的分級之比例(例如：分級為0之細胞數/總細胞數)，產生一IHC影像辨識結果，並將IHC影像辨識結果傳輸至判斷模組150及病理資料庫170。 The IHC image recognition module 140 calculates the ratio of cancer cell grades (e.g., the number of cells graded 0/total number of cells) for IHC images graded 0, 1+, 2+, or 3+, generates an IHC image recognition result, and transmits the IHC image recognition result to the judgment module 150 and the pathology database 170.
判斷模組150通訊連接通訊模組130及IHC影像辨識模組140。當接收到IHC影像辨識結果時，判斷模組150對上述IHC影像辨識結果進行判斷，產生一診斷與治療方針建議。上述診斷與治療方針建議如下表2。 The judgment module 150 is connected to the communication module 130 and the IHC image recognition module 140. When receiving the IHC image recognition result, the judgment module 150 judges the above IHC image recognition result and generates a diagnosis and treatment policy recommendation. The above diagnosis and treatment policy recommendation is shown in Table 2 below.
IHC影像辨識模組140與判斷模組150透過通訊模組130傳輸IHC影像辨識結果與診斷與治療方針建議至醫生裝置220，協助醫生根據IHC影像辨識結果與診斷與治療方針建議評估疾病的嚴重程度和預後，以及提供醫生診斷與治療方針建議。 The IHC image recognition module 140 and the judgment module 150 transmit the IHC image recognition results and the diagnosis and treatment policy recommendations to the doctor device 220 through the communication module 130, to assist the doctor in assessing the severity and prognosis of the disease based on the IHC image recognition results and the diagnosis and treatment policy recommendations, and to provide the doctor with diagnosis and treatment policy recommendations.
上述IHC影像辨識結果還傳輸至病理資料庫170儲存。 The above IHC image recognition results are also transmitted to the pathology database 170 for storage.
繼續說明分級為2+的IHC影像，IHC影像辨識模組140將IHC影像辨識結果傳輸至判斷模組150後，IHC影像辨識模組140中的一通知模組142會傳輸一通知訊息至檢驗人員裝置200，通知檢驗人員對此病患之組織進行螢光原位雜交玻片之製備。 Continuing to explain the IHC image graded as 2+, after the IHC image recognition module 140 transmits the IHC image recognition result to the judgment module 150, a notification module 142 in the IHC image recognition module 140 transmits a notification message to the inspector device 200, informing the inspector to prepare a fluorescent in situ hybridization slide for the patient's tissue.
檢驗人員將一病患之透過螢光原位雜交法處理之組織切片，通過顯微鏡或掃描裝置成像拍攝FISH影像，由醫師選取反應區域正中間附近的視野，視野大小通常為方便計數、細胞邊緣清楚且沒有混到太多非惡性細胞的部分。取像方式為先利用顯微鏡拍攝藍光影像，調整設定後再拍攝紅綠光影像，連結到電腦後透過程式做影像處理，再將兩張影像疊加合成新影像，即輸出藍光、紅綠光以及合成後的紅綠藍光三種FISH影像。將FISH影像透過檢驗人員裝置200傳輸至伺服器110內的影像資料庫112儲存。 The inspector takes a tissue slice of a patient treated by fluorescent in situ hybridization and uses a microscope or scanning device to take FISH images. The doctor selects the field of view near the center of the reaction area. The field of view is usually large enough to facilitate counting, with clear cell edges and not too many non-malignant cells mixed in. The imaging method is to first use a microscope to take a blue light image, then adjust the settings and take a red and green light image. After connecting to a computer, the program is used to process the image, and then the two images are superimposed to form a new image, that is, three FISH images of blue light, red and green light, and synthesized red, green and blue light are output. The FISH image is transmitted to the image database 112 in the server 110 through the inspector device 200 for storage.
當要進行FISH影像的影像辨識分析時，AI數位病理影像辨識系統100透過通訊模組130從伺服器110的影像資料庫112中傳輸至中控裝置120內的FISH影像辨識模組160。 When image recognition analysis of FISH images is to be performed, the AI digital pathology image recognition system 100 transmits the image database 112 of the server 110 to the FISH image recognition module 160 in the central control device 120 through the communication module 130.
FISH影像辨識模組160通訊連接通訊模組130、判斷模組150及病理資料庫170。在透過上述通訊模組130接收FISH影像後，FISH影像辨識模組160使用分類深度學習模型，利用影像切割對上述FISH影像進行辨識FISH影像之紅綠光點。上述FISH影像分為RG圖及RGB圖兩種，其中RG圖顯示紅光點(即HER2基因)，RGB圖顯示綠光點(即第17對染色體的中心粒)。FISH影像辨識模組160再計算紅光點及綠光點之比例，若紅光點大於等於6或紅光點介於4到6之間且紅光點/綠光點大於等於2，就表示有HER2基因過度表現(因正常細胞之紅光點小於4或紅光點介於4到6之間且紅光點/綠光點小於2)，產生一FISH影像辨識結果。 The FISH image recognition module 160 is connected to the communication module 130, the judgment module 150 and the pathology database 170. After receiving the FISH image through the communication module 130, the FISH image recognition module 160 uses the classification deep learning model and image segmentation to identify the red and green light spots of the FISH image. The FISH image is divided into two types: RG image and RGB image, wherein the RG image shows red light spots (i.e., HER2 gene) and the RGB image shows green light spots (i.e., the centrioles of the 17th pair of chromosomes). The FISH image recognition module 160 then calculates the ratio of red dots to green dots. If the red dots are greater than or equal to 6 or between 4 and 6 and the ratio of red dots to green dots is greater than or equal to 2, it indicates that the HER2 gene is overexpressed (because the red dots of normal cells are less than 4 or between 4 and 6 and the ratio of red dots to green dots is less than 2), and a FISH image recognition result is generated.
FISH影像辨識模組160將FISH影像辨識結果傳輸至判斷模組150。判斷模組150結合FISH影像辨識結果與IHC影像辨識結果產生一診斷與治療方針建議。若紅光點大於等於6或紅光點介於4到6之間且紅光點/綠光點大於等於2則其診斷與治療方針建議為建議使用標靶藥物治療，若紅光點小於4或紅光點介於4到6之間且紅光點/綠光點小於2則其診斷與治療方針建議為不建議使用標靶藥物治療。 The FISH image recognition module 160 transmits the FISH image recognition result to the judgment module 150. The judgment module 150 combines the FISH image recognition result with the IHC image recognition result to generate a diagnosis and treatment policy recommendation. If the red light spot is greater than or equal to 6 or the red light spot is between 4 and 6 and the red light spot/green light spot is greater than or equal to 2, the diagnosis and treatment policy recommendation is to recommend the use of targeted drug therapy. If the red light spot is less than 4 or the red light spot is between 4 and 6 and the red light spot/green light spot is less than 2, the diagnosis and treatment policy recommendation is not to recommend the use of targeted drug therapy.
FISH影像辨識模組160與判斷模組150透過通訊模組130傳輸FISH影像辨識結果與診斷與治療方針建議至醫生裝置220，協助醫生根據IHC影像辨識結果、FISH影像辨識結果與診斷與治療方針建議評估疾病的嚴重程度和預後，以及提供醫生診斷與治療方針建議。 The FISH image recognition module 160 and the judgment module 150 transmit the FISH image recognition results and the diagnosis and treatment policy recommendations to the doctor device 220 via the communication module 130, to assist the doctor in assessing the severity and prognosis of the disease based on the IHC image recognition results, the FISH image recognition results and the diagnosis and treatment policy recommendations, and to provide the doctor with diagnosis and treatment policy recommendations.
上述FISH影像辨識結果還傳輸至病理資料庫170儲存。 The above FISH image identification results are also transmitted to the pathology database 170 for storage.
請參閱圖2，圖2係繪示根據本新型一實施例之一種AI數位病理影像辨識系統100之流程示意圖。 Please refer to Figure 2, which is a schematic diagram showing the process of an AI digital pathological image recognition system 100 according to an embodiment of the present invention.
在步驟300中，通過顯微鏡與附加之取像裝置，或掃描裝置拍攝玻片，得到影像，並透過檢驗人員裝置200將病患的組織切片的IHC影像傳輸至伺服器110的影像資料庫112。 In step 300, the slide is photographed by a microscope and an attached imaging device or a scanning device to obtain an image, and the IHC image of the patient's tissue slice is transmitted to the image database 112 of the server 110 through the inspector device 200.
在步驟301中，IHC影像辨識模組140透過通訊模組130接收IHC影像。 In step 301, the IHC image recognition module 140 receives the IHC image through the communication module 130.
在步驟302中，IHC影像辨識模組140使用分類深度學習模型對IHC影像進行分類辨識。 In step 302, the IHC image recognition module 140 uses a classification deep learning model to classify and recognize IHC images.
在步驟303中，IHC影像辨識模組140依照IHC影像之一癌細胞的染色強度和完全程度進行分類，將IHC影像分為高低兩類。 In step 303, the IHC image recognition module 140 classifies the IHC image into two categories: high and low, according to the staining intensity and completeness of the cancer cells in the IHC image.
在步驟304中，IHC影像辨識模組140依照IHC影像之一癌細胞的染色強度和完全程度給予一分級，分為0、1+、2+或3+四級。 In step 304, the IHC image recognition module 140 assigns a grade according to the staining intensity and completeness of a cancer cell in an IHC image, which is divided into four grades: 0, 1+, 2+ or 3+.
在步驟305中，IHC影像辨識模組140判斷分級是否為2+。如是，則繼續進行步驟306a，如否，則繼續進行步驟306b。 In step 305, the IHC image recognition module 140 determines whether the grade is 2+. If so, proceed to step 306a, if not, proceed to step 306b.
繼續進行步驟306a，在步驟306a中，IHC影像辨識模組140計算IHC影像的癌細胞的各分級之比例，並產生IHC影像辨識結果。 Continuing to step 306a, in step 306a, the IHC image recognition module 140 calculates the ratio of each grade of cancer cells in the IHC image and generates an IHC image recognition result.
在步驟307中，IHC影像辨識模組140中的通知模組142傳輸一通知訊息至檢驗人員裝置200，檢驗人員對病患的組織進行螢光原位雜交玻片之製備。 In step 307, the notification module 142 in the IHC image recognition module 140 transmits a notification message to the inspector device 200, and the inspector prepares a fluorescent in situ hybridization slide for the patient's tissue.
在步驟308中，通過顯微鏡與附加之取像裝置，或掃描裝置拍攝玻片，得到影像，檢驗人員裝置200將病患的組織切片的FISH影像傳輸至伺服器110的影像資料庫112。 In step 308, the slide is photographed by a microscope and an attached imaging device, or a scanning device to obtain an image, and the inspector device 200 transmits the FISH image of the patient's tissue slice to the image database 112 of the server 110.
在步驟309中，FISH影像辨識模組160透過通訊模組130接收FISH影像。 In step 309, the FISH image recognition module 160 receives the FISH image through the communication module 130.
在步驟310中，FISH影像辨識模組160使用分類深度學習模型對FISH影像的紅綠光點進行辨識。 In step 310, the FISH image recognition module 160 uses a classification deep learning model to identify the red and green light spots of the FISH image.
在步驟311中，FISH影像辨識模組160計算FISH影像的紅綠光點的比例，並產生FISH影像辨識結果。 In step 311, the FISH image recognition module 160 calculates the ratio of red and green light spots in the FISH image and generates a FISH image recognition result.
在步驟312中，判斷模組150根據IHC影像辨識模組140傳輸的IHC影像辨識結果結合FISH影像辨識模組160傳輸的FISH影像辨識結果產生診斷與治療方針建議。 In step 312, the judgment module 150 generates diagnosis and treatment policy recommendations based on the IHC image recognition results transmitted by the IHC image recognition module 140 and the FISH image recognition results transmitted by the FISH image recognition module 160.
在步驟313中，IHC影像辨識模組140、FISH影像辨識模組160及判斷模組150透過通訊模組130傳輸IHC影像辨識結果、FISH影像結果及診斷與治療方針建議至醫生裝置220。 In step 313, the IHC image recognition module 140, the FISH image recognition module 160 and the judgment module 150 transmit the IHC image recognition results, the FISH image results and the diagnosis and treatment policy recommendations to the doctor device 220 via the communication module 130.
如否，則繼續進行步驟306b，在步驟306b中，IHC影像辨識模組140計算IHC影像的癌細胞的各分級之比例。 If not, proceed to step 306b, in which the IHC image recognition module 140 calculates the ratio of each grade of cancer cells in the IHC image.
在步驟314中，IHC影像辨識模組140產生IHC影像辨識結果，並繼續進行步驟312。 In step 314, the IHC image recognition module 140 generates an IHC image recognition result and proceeds to step 312.
本新型可主張的功效包括：(1)通過自動化分析病理學影像，協助醫生評估疾病的嚴重程度和預後，並提供診斷與治療方針建議。(2)自動分析IHC和FISH影像，減少人為錯誤，提高診斷和分析的可靠性，可重複性。(3)節約進行人工分析所需之龐大人力。(4)加大單位工作時間可分析之病人影像數目，縮短臨床醫師治療決策須等待的時間，並推進相關領域之研究。(5)有效管理和 存儲大量的影像，包含記錄原本無法永久儲存之FISH玻片和其分析結果，可隨時調閱，復閱與進行外部意見諮詢。 The claimed efficacy of this new technology includes: (1) Assisting doctors in assessing the severity and prognosis of the disease and providing diagnostic and treatment recommendations through automated analysis of pathological images. (2) Automatic analysis of IHC and FISH images to reduce human errors and improve the reliability and repeatability of diagnosis and analysis. (3) Saving the huge manpower required for manual analysis. (4) Increasing the number of patient images that can be analyzed per unit of working time, shortening the waiting time for clinical physicians to make treatment decisions, and promoting research in related fields. (5) Effectively managing and storing a large number of images, including recording FISH slides and their analysis results that were originally unable to be permanently stored, which can be accessed, reviewed, and consulted with external opinions at any time.
本說明書中的各個實施例均採用遞進的方式描述，各個實施例之間相同相似的部分互相參見即可，每個實施例重點說明的都是與其他實施例的不同之處。尤其，對於系統實施例而言，由於其基本相似於方法實施例，所以描述的比較簡單，相關之處參見方法實施例的部分說明即可。 Each embodiment in this specification is described in a progressive manner. The same or similar parts between the embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For the relevant parts, refer to the partial description of the method embodiment.
上述之實施案例僅為舉例性之具體說明，而非為限制本新型之範圍，凡任何對其進行之等效修改或變更者，皆未脫離本新型之精神與範疇，均應包含於本案申請專利範圍中。 The above implementation cases are only specific examples and are not intended to limit the scope of this new model. Any equivalent modifications or changes made to them do not deviate from the spirit and scope of this new model and should be included in the scope of the patent application of this case.
100:AI數位病理影像辨識系統 100: AI digital pathology image recognition system
110:伺服器 110: Server
112:影像資料庫 112: Image database
120:中控裝置 120: Central control device
130:通訊模組 130: Communication module
140:IHC影像辨識模組 140:IHC image recognition module
142:通知模組 142: Notification module
150:判斷模組 150: Judgment module
160:FISH影像辨識模組 160:FISH image recognition module
170:病理資料庫 170: Pathology database
200:檢驗人員裝置 200: Inspection personnel device
220:醫生裝置 220: Doctor's Device
Claims (7)
Publications (1)
| Publication Number | Publication Date |
|---|---|
| TWM653792U true TWM653792U (en) | 2024-04-01 |
Family
ID=
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6800152B2 (en) | Classification of nuclei in histological images | |
| US7760927B2 (en) | Method and system for digital image based tissue independent simultaneous nucleus cytoplasm and membrane quantitation | |
| US8417015B2 (en) | Methods and system for validating sample images for quantitative immunoassays | |
| EP1576521B1 (en) | Scoring estrogen and progesterone receptors expression based on image analysis | |
| US8787651B2 (en) | Methods for feature analysis on consecutive tissue sections | |
| Masmoudi et al. | Automated quantitative assessment of HER-2/neu immunohistochemical expression in breast cancer | |
| US10565430B2 (en) | Histology recognition to automatically score and quantify cancer grades and individual user digital whole histological imaging device | |
| US9567651B2 (en) | System and method for the detection of abnormalities in a biological sample | |
| US20050136549A1 (en) | Method and system for automatically determining diagnostic saliency of digital images | |
| US20110111435A1 (en) | Detecting Cell Surface Markers | |
| US20050265588A1 (en) | Method and system for digital image based flourescent in situ hybridization (FISH) analysis | |
| US7865000B2 (en) | Medical image processing apparatus for analyzing properties of living body based on image data thereof | |
| JP2000513465A (en) | Automatic microscope-assisted inspection method for living tissue samples and body fluid samples | |
| JP5469070B2 (en) | Method and system using multiple wavelengths for processing biological specimens | |
| López et al. | Digital image analysis in breast cancer: an example of an automated methodology and the effects of image compression | |
| Gavrielides et al. | Observer performance in the use of digital and optical microscopy for the interpretation of tissue-based biomarkers | |
| WO2012041333A1 (en) | Automated imaging, detection and grading of objects in cytological samples | |
| EP3117259A1 (en) | Assessment of staining quality | |
| WO2007024264A2 (en) | Method and system for digital image based tissue independent simultaneous nucleus, cytoplasm and membrane quantitation | |
| Guillaud et al. | Exploratory analysis of quantitative histopathology of cervical intraepithelial neoplasia: Objectivity, reproducibility, malignancy‐associated changes, and human papillomavirus | |
| CN113011257A (en) | Breast cancer immunohistochemical artificial intelligence interpretation method | |
| EP4260324A1 (en) | Systems and methods for generating histology image training datasets for machine learning models | |
| WO2005076216A2 (en) | Method and system for automaticed digital image based flourescent in situ hybridization (fish) analysis | |
| US8693743B1 (en) | Analysis and display of multiple biomarker co-expression in cells and tissues | |
| WO2005001438A1 (en) | System for classifying slides using scatter plot distributions |