TWI781408B - Artificial intelligence based cell detection method by using hyperspectral data analysis technology - Google Patents
Artificial intelligence based cell detection method by using hyperspectral data analysis technology Download PDFInfo
- Publication number
- TWI781408B TWI781408B TW109117840A TW109117840A TWI781408B TW I781408 B TWI781408 B TW I781408B TW 109117840 A TW109117840 A TW 109117840A TW 109117840 A TW109117840 A TW 109117840A TW I781408 B TWI781408 B TW I781408B
- Authority
- TW
- Taiwan
- Prior art keywords
- cell
- hyperspectral
- time point
- cells
- normalized
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
- G06F18/24133—Distances to prototypes
- G06F18/24137—Distances to cluster centroïds
- G06F18/2414—Smoothing the distance, e.g. radial basis function networks [RBFN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30024—Cell structures in vitro; Tissue sections in vitro
Abstract
Description
本發明描述一種利用高光譜資料分析技術之人工智慧的細胞檢測方法及其系統,尤指一種具有偵測細胞品質以及辨識細胞功能之人工智慧的細胞檢測方法及其系統。 The present invention describes an artificial intelligence cell detection method and system using hyperspectral data analysis technology, especially an artificial intelligence cell detection method and system with the ability to detect cell quality and identify cell functions.
隨著科技日新月異,適逢生育年齡的婦女常會因為工作壓力、飲食習慣、文明病、排卵功能異常、賀爾蒙失調或是一些慢性病而導致不孕。在現今,不孕症是高自費的療程,在台灣、中國以及國際市場需求巨大,且其需求每年是高度成長。許多婦女會選擇體外人工受孕(In Vitro Fertilization,IVF)來治療不孕症的問題。體外人工受孕是將卵子與精子取出,在人為操作下進行體外受精,並培養成胚胎,再將胚胎植回母體內。然而,現有的不孕症療程之成功率僅三成。不孕症的療程之重點在於胚胎的選擇。然而,現有選擇胚胎的方法主要還是由胚胎醫師利用胚胎照片或是縮時影片的資料,以主觀方式判斷胚胎的優劣。以目前的技術而論,由於缺乏一種系統化以及自動化的方式判斷胚胎的良劣,故在不孕症療程中,由醫師主觀地選擇植入胚胎,其植入成功率仍低,故也是目前不孕症療程的瓶頸。 With the rapid development of science and technology, women of reproductive age often suffer from infertility due to work pressure, eating habits, civilized diseases, abnormal ovulation function, hormonal imbalance or some chronic diseases. Nowadays, infertility is a high-paying treatment, and there is a huge demand in Taiwan, China and the international market, and its demand is growing rapidly every year. Many women will choose In Vitro Fertilization (IVF) to treat infertility problems. In vitro artificial fertilization is to take out eggs and sperm, carry out in vitro fertilization under artificial operation, and cultivate them into embryos, and then implant the embryos back into the mother. However, the success rate of existing infertility treatments is only 30%. The focus of infertility treatment is embryo selection. However, the existing method for selecting embryos is mainly for embryologists to use the data of embryo photos or time-lapse videos to judge the quality of embryos subjectively. As far as the current technology is concerned, due to the lack of a systematic and automated way to judge whether the embryo is good or bad, in the course of infertility treatment, the doctor subjectively selects the embryo to be implanted, and the implantation success rate is still low. Bottleneck in infertility treatment.
換句話說,在目前的不孕症療程下,醫生僅能以主觀的方式觀察胚胎在分裂時的情況。例如,醫生會以主觀的方式,依據胚胎在發育中的細胞數目、細胞分裂的均勻程度以及分裂時的碎片程度,將胚胎由優到劣區分為多個等級。例如,均勻地分裂為雙數細胞的胚胎較優,而產生不完整的單數細胞分裂以及碎片越多的胚胎,生長潛力較差。然而,如前述提及,由於目前判斷胚胎的優劣主要還是依據醫生的經驗,以主觀的判斷方式來選擇較佳胚胎。因此,目前技術治療不孕症的成功率很難提升,且容易受到不同醫生的主觀見解而影響成功率(例如誤判)。 In other words, with current infertility treatments, doctors can only observe subjectively what happens to embryos as they divide. For example, doctors will subjectively classify embryos into several grades from good to bad according to the number of developing cells, the uniformity of cell division, and the degree of fragmentation during division. For example, embryos that divide evenly into even numbers of cells are superior, while embryos that produce incomplete odd cell divisions and more fragmentation have poorer growth potential. However, as mentioned above, since judging the quality of embryos is mainly based on the doctor's experience, the selection of better embryos is based on subjective judgment. Therefore, it is difficult to improve the success rate of the current technical treatment of infertility, and the success rate is easily affected by the subjective opinions of different doctors (such as misjudgment).
本發明一實施例提出一種利用高光譜資料分析技術之人工智慧的細胞檢測方法。利用高光譜資料分析技術之人工智慧的細胞檢測方法包含取得複數個細胞,在第一時間點以及第二時間點之間,取樣該些細胞的N張影像,依據該些細胞的N張影像,進行影像校正程序,以產生N張正規化細胞高光譜影像,決定每一個細胞在發育期間,內部至少一種化學成分發生改變時對應的觀測區域,在第一時間點以及第二時間點之間,分析N張正規化細胞高光譜影像,以產生觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值,將每一個細胞之觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值及/或N張正規化細胞高光譜影像輸入至類神經網路,以訓練類神經網路,以及利用類神經網路,以人工智慧的程序建立細胞品質檢測模型,以檢測細胞品質及/或辨識細胞。第一時間點在第二時間點之前,且N為大於2的正整數。 An embodiment of the present invention proposes a cell detection method using artificial intelligence of hyperspectral data analysis technology. The artificial intelligence cell detection method using hyperspectral data analysis technology includes obtaining a plurality of cells, sampling N images of these cells between the first time point and the second time point, and based on the N images of these cells, Perform image correction procedures to generate N normalized cell hyperspectral images, and determine the corresponding observation area when at least one internal chemical composition of each cell changes during development, between the first time point and the second time point, Analyze N normalized cell hyperspectral images to generate the key normalized cell hyperspectral image difference characteristic value corresponding to the observation area, and calculate the key normalized cell hyperspectral image difference characteristic value and/or N corresponding to the observation area of each cell A normalized cell hyperspectral image is input to the neural network to train the neural network, and the neural network is used to establish a cell quality detection model with an artificial intelligence program to detect cell quality and/or identify cells. The first time point is before the second time point, and N is a positive integer greater than 2.
本發明另一實施例提出一種利用高光譜資料分析技術之人工智慧的細胞檢測系統。利用高光譜資料分析技術之人工智慧的細胞檢測系統包含載具、透鏡模組、高光譜儀、處理器以及記憶體。載具具有容置槽,用以放置複 數個細胞。透鏡模組面對載具,用以放大該些細胞的細節。高光譜儀面對透鏡模組,用以透過透鏡模組取得該些細胞的影像。處理器耦接於透鏡模組及高光譜儀,用以調整透鏡模組的放大倍率以及處理該些細胞的影像。記憶體耦接於處理器,用以儲存訓練資料以及影像處理的分析資料。載具之容置槽放置該些細胞後,處理器控制高光譜儀,透過透鏡模組在第一時間點以及第二時間點之間,取樣該些細胞的N張影像。處理器依據該些細胞的N張影像,進行影像校正程序,以產生N張正規化細胞高光譜影像,決定每一個細胞在發育期間,內部至少一種化學成分發生改變時對應的觀測區域,並在第一時間點以及第二時間點之間,分析N張正規化細胞高光譜影像,以產生觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值。處理器包含類神經網路,每一個細胞之觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值及/或N張正規化細胞高光譜影像用以訓練類神經網路。處理器利用類神經網路,以人工智慧的程序建立細胞品質檢測模型,以檢測細胞品質及/或辨識細胞。第一時間點在第二時間點之前,且N為大於2的正整數。 Another embodiment of the present invention proposes a cell detection system using artificial intelligence of hyperspectral data analysis technology. The artificial intelligence cell detection system using hyperspectral data analysis technology includes a carrier, a lens module, a hyperspectral instrument, a processor, and a memory. The carrier has a receiving slot for placing complex several cells. The lens module faces the carrier and is used to enlarge the details of the cells. The hyperspectral instrument faces the lens module, and is used for obtaining images of the cells through the lens module. The processor is coupled to the lens module and the hyperspectral instrument for adjusting the magnification of the lens module and processing the images of the cells. The memory is coupled to the processor for storing training data and analysis data of image processing. After placing the cells in the holding tank of the carrier, the processor controls the hyperspectral instrument to sample N images of the cells between the first time point and the second time point through the lens module. The processor performs image correction procedures based on the N images of these cells to generate N normalized cell hyperspectral images, and determines the corresponding observation area when at least one chemical composition in each cell changes during development, and in the Between the first time point and the second time point, N normalized cell hyperspectral images are analyzed to generate key normalized cell hyperspectral image difference characteristic values corresponding to the observation area. The processor includes a neural network, and the key normalized cell hyperspectral image difference characteristic values corresponding to the observation area of each cell and/or N normalized cell hyperspectral images are used to train the neural network. The processor uses a neural network to establish a cell quality detection model with an artificial intelligence program to detect cell quality and/or identify cells. The first time point is before the second time point, and N is a positive integer greater than 2.
100:利用高光譜資料分析技術之人工智慧的細胞檢測系統 100: Cell detection system using artificial intelligence of hyperspectral data analysis technology
10:載具 10: Vehicle
11:透鏡模組 11: Lens module
12:高光譜儀 12:Hyperspectrometer
13:處理器 13: Processor
14:記憶體 14: Memory
S201至S207:步驟 S201 to S207: Steps
S301至S302:步驟 S301 to S302: Steps
D1:關鍵正規化細胞高光譜影像差異特徵數值 D1: Key normalized cell hyperspectral image difference feature value
D2:N張正規化細胞高光譜影像 D2: N normalized cell hyperspectral images
D3:波長正規化細胞應向邊緣特徵數據 D3: Wavelength normalized cell should be towards the edge feature data
D4:細胞品質輸出資料 D4: Cell quality output data
第1圖係為本發明之利用高光譜資料分析技術之人工智慧的細胞檢測系統之實施例的方塊圖。 Fig. 1 is a block diagram of an embodiment of an artificial intelligence cell detection system utilizing hyperspectral data analysis technology of the present invention.
第2圖係為第1圖之利用高光譜資料分析技術之人工智慧的細胞檢測系統執行細胞檢測方法的流程圖。 FIG. 2 is a flowchart of the cell detection method executed by the artificial intelligence cell detection system using hyperspectral data analysis technology in FIG. 1 .
第3圖係為第1圖之利用高光譜資料分析技術之人工智慧的細胞檢測系統中,加入額外步驟以增強細胞檢測的精確度的示意圖。 Figure 3 is a schematic diagram of adding additional steps to enhance the accuracy of cell detection in the artificial intelligence cell detection system using hyperspectral data analysis technology in Figure 1 .
第4圖係為第1圖之利用高光譜資料分析技術之人工智慧的細胞檢測系統中,具 有類神經網路的處理器之輸入資料以及輸出資料的示意圖。 Figure 4 is the artificial intelligence cell detection system using hyperspectral data analysis technology in Figure 1, with Schematic diagram of the input data and output data of a neural network-like processor.
第1圖係為本發明之利用高光譜資料分析技術之人工智慧的細胞檢測系統100之實施例的方塊圖。為了簡化描述,利用高光譜資料分析技術之人工智慧的細胞檢測系統100後文稱為「細胞檢測系統100」。細胞檢測系統100包含載具10、透鏡模組11、高光譜儀12、處理器13以及記憶體14。載具10具有容置槽,用以放置複數個細胞。舉例而言,載具10可為培養皿,其內部的容置槽可包含一些培養液。複數個細胞可在培養液中發育。複數個細胞可為複數個生殖細胞、複數個胚胎或是任何欲觀察且可分裂的複數個細胞。透鏡模組11面對載具10,用以放大該些細胞的細節。透鏡模組11可為任何具有光學或是數位變焦能力的透鏡模組,例如顯微鏡模組。高光譜儀12面對透鏡模組11,用以透過透鏡模組11取得該些細胞的影像。因此,在細胞檢測系統100中,高光譜儀12所擷取之該些細胞的影像可為(A)高光譜儀影像中之任意一個波長對應的細胞影像,(B)高光譜影像各波長之細胞影像所合成的灰階細胞影像。任何合理的影像格式都屬於本發明所揭露的範疇。處理器13耦接於透鏡模組11及高光譜儀12,用以調整透鏡模組11的放大倍率以及處理該些細胞的影像。處理器13可為中央處理器、微處理器、或是任何的可程式化處理單元。處理器13具有類神經網路,例如深度神經網路(Deep Neural Networks,DNN),可以執行機器學習以及深度學習的功能。因此,處理器13的類神經網路可以被訓練,可視為人工智慧的處理核心。記憶體14耦接於處理器13,用以儲存訓練資料以及影像處理的分析資料。
FIG. 1 is a block diagram of an embodiment of an artificial intelligence
在細胞檢測系統100中,在載具10之容置槽放置該些細胞後,處理器13可控制高光譜儀12,透過透鏡模組11在第一時間點以及第二時間點之間,取
樣該些細胞的N張影像。接著,處理器13可以決定每一個細胞在發育期間,內部至少一種化學成分發生改變時對應的觀測區域,並在第一時間點以及第二時間點之間,分析N張正規化細胞高光譜影像,以產生觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值。如前述提及,處理器13包含可訓練的類神經網路。因此,每一個細胞之觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值及/或N張正規化細胞高光譜影像可用以訓練類神經網路。在類神經網路訓練完成後,處理器13可以利用類神經網路,以人工智慧的程序建立細胞品質檢測模型,以檢測細胞品質及/或辨識細胞。在細胞檢測系統100中,第一時間點在第二時間點之前,且N為大於2的正整數。換句話說,細胞檢測系統100可以用兩個不同時間點之間的時間序列的細胞影像資訊來訓練類神經網路。在類神經網路訓練完成後,細胞檢測系統100即具備人工智慧的細胞檢測功能,具有自動化地檢測細胞品質及/或辨識細胞的能力。細胞檢測系統100如何訓練類神經網路以執行人工智慧的細胞檢測功能的細節,將描述於後文。
In the
第2圖係為利用高光譜資料分析技術之人工智慧的細胞檢測系統100執行細胞檢測方法的流程圖。細胞檢測方法可包含步驟S201至步驟S207。任何合理的步驟變或是技術更動都屬於本發明所揭露的範疇。步驟S201至步驟S207的內容描述於下:步驟S201:取得複數個細胞;步驟S202:在第一時間點以及第二時間點之間,取樣該些細胞的N張影像;步驟S203:依據該些細胞的N張影像,進行影像校正程序,以產生N張正規化細胞高光譜影像;步驟S204:決定每一個細胞在發育期間,內部至少一種化學成分發生改變時對應的觀測區域;
步驟S205:在第一時間點以及第二時間點之間,分析N張正規化細胞高光譜影像,以產生觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值;步驟S206:將每一個細胞之觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值及/或N張正規化細胞高光譜影像輸入至類神經網路,以訓練類神經網路;步驟S207:利用類神經網路,以人工智慧的程序建立細胞品質檢測模型,以檢測細胞品質及/或辨識細胞。
FIG. 2 is a flow chart of the cell detection method executed by the
為了描述簡化,後文的「細胞」僅以「胚胎」為實施例進行說明,然而本發明並不限於此,細胞的定義可為生殖細胞、神經細胞、組織細胞、動植物細胞或是任何需要研究及觀察的細胞。在步驟S201中,研究人員或是醫療人員可先取得多個胚胎。在步驟S202中,處理器13可以控制高光譜儀12,在第一時間點以及第二時間點之間,取樣該些細胞的N張影像。舉例而言,高光譜儀12可在一段時間內(第一時間點以及第二時間點之間),以錄影的方式(如30fps或是60fps)取得多張幀(Frames)構成的影像集合。或是,高光譜儀12可在一段時間內(第一時間點以及第二時間點之間),週期性地對多個胚胎拍照,以取得該些胚胎的N張影像。並且,第一時間點以及第二時間點可為該些胚胎於培養液中發育且分裂之觀察週期中的任兩時間點。舉例而言,第一時間點以及第二時間點可以分別選為第一天以及第五天,以觀察該些胚胎發育以及分裂的狀況。換句話說,高光譜儀12可以在第0個小時到第120小時之內對該些胚胎連續地拍照,以取得N張影像。在步驟S203中,處理器13可以依據該些細胞的N張影像,進行影像校正程序,以產生N張正規化細胞高光譜影像。N張影像可為N張的高光譜影像。處理器13可以依據N張高光譜影像及/或環境光參數,取得亮場(Bright Field)資訊及暗場(Dark Field)資訊。接著,處理器13可以依據亮場資訊及暗場資訊,
產生光穿透率百分比數值,並依據光亮場資訊及/或暗場資訊與穿透率百分比數值,校正N張高光譜影像,以產生N張正規化細胞高光譜影像。舉例而言,由於高光譜儀12每次取樣(拍攝)該些胚胎時的環境光不同,因此高光譜影像需要正規化(Normalized)以進行一致性之影像處理。例如,在時間點T1時,高光譜儀12附近的光能量為100單位(如nits)。高光譜儀12所產生的正規化細胞高光譜影像中,亮場的光能量為200單位。因此,針對亮場而言,亮場的光能量(200單位)可能有100單位是環境光所貢獻的,故光線穿透率百分比數值可視為50%。例如,在時間點T2時,高光譜儀12附近的光能量為200單位(如nits)。高光譜儀12所產生的正規化細胞高光譜影像中,亮場的光能量為400單位。因此,針對亮場而言,亮場的光能量(400單位)可能有200單位是環境光貢獻的,故光線穿透率百分比數值可視為50%。因此,高光譜儀12在時間點T1以及在時間點T2所產生的兩張高光譜影像具有相同的光線穿透率百分比數值(亮場)。類似地,若是高光譜儀12在時間點T1以及在時間點T2所產生的兩張高光譜影像具有不同的光線穿透率百分比數值(暗場),則處理器13需要將暗場的影像進行校正,如增加暗場亮度以使兩張影像不會發生高反差的情況。換句話說,高光譜儀在取樣過程中、環境光、光強值、亮場以及暗場資訊均有差異,故細胞檢測系統100要執行正規化的程序,以使正規化後的影像資料具有一致性。並且,經過正規化的高光譜資料,可帶有穿透率的資訊(可用百分比表示)。若是穿透率經過正規化的處理,則影像亮度即具有一致性。在細胞檢測系統100中,任何讓N張高光譜影像之正規化的方法或是技術都屬於本發明所揭露的範疇。
In order to simplify the description, the following "cells" are only described with "embryo" as an example, but the present invention is not limited thereto. The definition of cells can be germ cells, nerve cells, tissue cells, animal and plant cells, or any research needs and observed cells. In step S201, researchers or medical personnel can obtain multiple embryos first. In step S202, the processor 13 may control the hyperspectral instrument 12 to sample N images of the cells between the first time point and the second time point. For example, the hyperspectral instrument 12 can acquire an image set composed of multiple frames (Frames) in a video recording mode (such as 30 fps or 60 fps) within a period of time (between the first time point and the second time point). Alternatively, the hyperspectral instrument 12 can periodically take pictures of multiple embryos within a period of time (between the first time point and the second time point), so as to obtain N images of the embryos. Moreover, the first time point and the second time point can be any two time points in the observation period during which the embryos develop and divide in the culture medium. For example, the first time point and the second time point can be respectively selected as the first day and the fifth day to observe the development and division of the embryos. In other words, the hyperspectral instrument 12 can continuously take pictures of the embryos from the 0th hour to the 120th hour to obtain N images. In step S203, the processor 13 may perform an image correction procedure according to the N images of the cells to generate N normalized cell hyperspectral images. The N images may be N hyperspectral images. The processor 13 can obtain Bright Field information and Dark Field information according to the N hyperspectral images and/or ambient light parameters. Next, the processor 13 may, according to the bright field information and the dark field information,
Generate light transmittance percentage values, and correct N hyperspectral images according to bright field information and/or dark field information and transmittance percentage values, so as to generate N normalized cell hyperspectral images. For example, since the ambient light of the embryos is different each time the hyperspectral instrument 12 samples (photographs), the hyperspectral images need to be normalized for consistent image processing. For example, at the time point T1, the light energy near the hyperspectrometer 12 is 100 units (eg, nits). In the normalized cell hyperspectral image generated by the hyperspectrometer 12 , the light energy of the bright field is 200 units. Therefore, for the bright field, 100 units of the light energy (200 units) of the bright field may be contributed by ambient light, so the light transmittance percentage value can be regarded as 50%. For example, at the time point T2, the light energy near the hyperspectrometer 12 is 200 units (eg, nits). In the normalized cell hyperspectral image generated by the hyperspectrometer 12 , the light energy of the bright field is 400 units. Therefore, for the bright field, 200 units of the light energy (400 units) of the bright field may be contributed by ambient light, so the light transmittance percentage value can be regarded as 50%. Therefore, the two hyperspectral images generated by the hyperspectral instrument 12 at the time point T1 and the time point T2 have the same light transmittance percentage value (bright field). Similarly, if the two hyperspectral images produced by the hyperspectral instrument 12 at the time point T1 and the time point T2 have different light transmittance percentage values (dark field), the processor 13 needs to correct the dark field image , such as increasing the brightness of the dark field so that the high contrast between the two images does not occur. In other words, during the sampling process of the hyperspectrometer, there are differences in ambient light, light intensity value, bright field and dark field information, so the
並且,應當理解的是,高光譜儀12取得該些細胞的N張高光譜影像,包含高光譜儀12在第一時間點以及第二時間點之間,於至少一個特定波長下,取得該些細胞的N張高光譜影像。一般而言,在可見光的物體影像中,可以看到物體的輪廓、顏色、偏光特性。然而,當不同物質具有相同的可見光特性(顏色、 形狀、偏光特性等等)時,肉眼將難以察覺。然而,物質的特性會顯示在其光譜特性上,因此,高光譜儀12可以針對高光譜訊號做定量的分析,以進一步解析物質的差異性。高光譜儀12所支援的光譜波段數目可為數百個,且光譜解析度可為奈米等級。因此,高光譜儀12取得該些細胞的N張高光譜影像,可以包含不同波長下的高光譜影像。光譜波段數目越多,高光譜影像的數量將越大(N越大),影像分析能力也會越好。在步驟S204中,處理器13決定每一個細胞在發育期間,內部至少一種化學成分發生改變時對應的觀測區域。如前述提及,高光譜影像可以分析物質的差異性,例如胚胎的新陳代謝物質(特徵細胞)或是各種化學成分。例如,在特徵細胞中,有均勻代謝的判斷可根據葡萄糖(Glucose)、放射性同位素(Radioisotope)、自發螢光(Auto-fluorescence)、3H2O以及14CO2的特性而決定。例如,在健康胚胎中,有較高的核糖核酸(RNA)、蛋白含量(Protein Content)以及糖解依賴(Glycolytic Dependence)。換句話說,當高光譜儀12產生之高光譜影像中,某個胚胎之核糖核酸、蛋白含量以及糖解依賴的比例符合健康胚胎的比例,則某個胚胎在此時間點可被視為健康胚胎。如前述提及,胚胎在發育期間會產生數次分裂,每次分裂後,胚胎內某個部份的化學成分比例就會發生改變。因此,在步驟S204中,處理器13之定義的觀測區域可為該些胚胎中,每一個胚胎的囊胚(Blastocyst)區域或該些胚胎之至少一種化學成分發生改變時,差異性最大的區域。 Moreover, it should be understood that the hyperspectral instrument 12 acquires N hyperspectral images of the cells, including that the hyperspectral instrument 12 acquires the hyperspectral images of the cells at at least one specific wavelength between the first time point and the second time point. N hyperspectral images. Generally speaking, in the object image of visible light, the outline, color, and polarization characteristics of the object can be seen. However, when different substances have the same visible light properties (color, shape, polarization properties, etc.), it will be difficult for the naked eye to detect them. However, the characteristics of the substance will be displayed in its spectral characteristics. Therefore, the hyperspectral instrument 12 can perform quantitative analysis on the hyperspectral signal to further analyze the difference of the substance. The number of spectral bands supported by the hyperspectral instrument 12 may be hundreds, and the spectral resolution may be at the nanometer level. Therefore, the hyperspectral instrument 12 obtains N hyperspectral images of the cells, which may include hyperspectral images at different wavelengths. The larger the number of spectral bands, the larger the number of hyperspectral images (the larger N is), and the better the image analysis capability will be. In step S204, the processor 13 determines the corresponding observation area when at least one internal chemical composition of each cell changes during development. As mentioned above, hyperspectral images can analyze the differences of substances, such as the metabolic substances (characteristic cells) of embryos or various chemical components. For example, in characteristic cells, the judgment of uniform metabolism can be determined according to the characteristics of Glucose, Radioisotope, Auto-fluorescence, 3 H 2 O and 14 CO 2 . For example, in healthy embryos, there are higher ribonucleic acid (RNA), protein content (Protein Content) and glycolytic dependence (Glycolytic Dependence). In other words, when in the hyperspectral image generated by the hyperspectral instrument 12, the ratio of ribonucleic acid, protein content, and glycolysis dependence of a certain embryo matches the proportion of a healthy embryo, then a certain embryo can be regarded as a healthy embryo at this time point . As mentioned above, the embryo divides several times during development, and after each division, the chemical composition ratio of a certain part of the embryo will change. Therefore, in step S204, the observation area defined by the processor 13 may be among the embryos, the blastocyst (Blastocyst) area of each embryo or the area with the greatest difference when at least one chemical composition of the embryos changes. .
接著,在步驟S205中,處理器13可以在第一時間點以及第二時間點之間,分析N張正規化細胞高光譜影像,以產生觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值。在細胞檢測系統100中,關鍵正規化細胞高光譜影像差異特徵數值可包含各波長下正規化光譜之間的比例關係、正規化光譜之波峰及/或波谷數值、該波峰及/或波谷對應的波長數據、以及該些波長數據間之相關性。廣義地說,N張正規化細胞高光譜影像中的任何可量化的特徵,可以被處理
器13進行量化處理,而變為二位元的數位數值。並且,由於處理器13包含類神經網路,因此,在步驟S206中,每一個細胞之觀測區域對應的關鍵正規化細胞高光譜影像差異特徵數值及/或N張正規化細胞高光譜影像可以輸入至類神經網路,以訓練類神經網路。在類神經網路被訓練完成後,依據步驟S207,處理器13可以用人工智慧的程序,建立細胞品質檢測模型,以檢測細胞品質及/或辨識細胞。在細胞檢測系統100中,類神經網路具有建立深度學習模型之訓練數據蒐集的能力。例如,類神經網路可以蒐集胚胎由第一天至第五天內,各階段原始的影像資料、最後階段(或是發育期間的任何時間點)的高光譜技術擷取的高光譜資料、將高光譜資料分析後的影像特徵資料、以及時間序列資料(N張影像)以執行深度學習的訓練。並且,類神經網路還可以依據胚胎著床成功或是著床失敗的結果(著床率)修正細胞品質檢測模型。任何用以訓練類神經網路以進行人工智慧之細胞檢測功能的方式都屬於本發明所揭露的範疇。換句話說,細胞檢測系統100利用人工智慧檢測胚胎品質及/或辨識胚胎可包含兩個階段。第一階段為訓練階段。第二階段為人工智慧之偵測階段。在類神經網路訓練完成後,處理器13可以利用已經訓練完成的類神經網路之細胞品質檢測模型,判斷胚胎品質以及辨識胚胎。因此,本發明的細胞檢測系統100可以避免醫療人員或是研究人員以主觀的方式來判斷胚胎的優劣,因此針對不孕症的療程,其受孕成功率能夠大幅度地提升。
Next, in step S205, the processor 13 can analyze N normalized cell hyperspectral images between the first time point and the second time point to generate key normalized cell hyperspectral image difference characteristic values corresponding to the observation area . In the
第3圖係為利用高光譜資料分析技術之人工智慧的細胞檢測系統100中,加入額外步驟以增強細胞檢測的精確度的示意圖。為了進一步加強判斷胚胎品質以及辨識胚胎的精確度,細胞檢測系統100還可以引入型態學的偵測技術來強化類神經網路的細胞檢測能力,如下所示。
FIG. 3 is a schematic diagram of adding additional steps to enhance the accuracy of cell detection in the
步驟S201:取得複數個細胞;步驟S301:在第一時間點以及第二時間點之間,依據N張正規化細胞高光 譜影像,擷取至少一個波長的正規化細胞影像;步驟S302:利用型態學的邊緣偵測技術,依據至少一個波長的正規化細胞影像,取得每一個時間點及每一個波長下之波長正規化細胞影像邊緣特徵數據,並將波長正規化細胞影像之邊緣特徵數據輸入至類神經網路,以訓練類神經網路。 Step S201: Obtain a plurality of cells; Step S301: Between the first time point and the second time point, according to N pieces of normalized cell highlights Spectrum image, extracting normalized cell image of at least one wavelength; Step S302: Using morphological edge detection technology, according to the normalized cell image of at least one wavelength, obtain the normalized wavelength of each time point and each wavelength The edge feature data of the cell image is normalized, and the edge feature data of the wavelength normalized cell image is input to the neural network to train the neural network.
類似地,為了描述簡化,後文的「細胞」僅以「胚胎」為實施例進行說明,然而本發明並不限於此,細胞的定義可為生殖細胞、神經細胞、組織細胞、動植物細胞或是任何需要研究及觀察的細胞。為了進一步加強判斷胚胎品質以及辨識胚胎的精確度,細胞檢測系統100在前述步驟S201取得複數個細胞(胚胎)後,依據步驟S301,處理器13可以在第一時間點以及第二時間點之間,依據N張正規化細胞高光譜影像,擷取至少一個波長的正規化細胞影像。如前述提及,高光譜儀12所支援的光譜波段數目可為數百個,因此處理器13在校正高光譜儀12產生之影像的亮場/暗場參數後(正規化),可以取得每一個波長下的正規化細胞影像。接著,依據步驟S302,處理器13可以利用型態學的邊緣偵測技術,依據至少一個波長的正規化細胞影像,取得每一個時間點及每一個波長下之波長正規化細胞影像之邊緣特徵數據,並將波長正規化細胞影像之邊緣特徵數據輸入至類神經網路,以訓練類神經網路。在此,邊緣特徵數據可為胚胎整體或是某一個特定部分(例如囊胚)的輪廓,其數據格式可用多個座標的方式表示。例如在二維平面上,單一胚胎的輪廓可為封閉型的線段,可用(X1,Y1)至(XM,YM)表示,M為正整數且M越大解析度越高。對比於第2圖所述之步驟S201至步驟S207的細胞偵測方法,細胞檢測系統100可以引入額外的步驟S301至步驟S302,以獲取更多的資訊(如波長正規化細胞影像之邊緣特徵數據)來訓練類神經網路。因此,類神經網路的訓練會更加優化,從而增加了人工智慧偵測細胞品質的精確度。
Similarly, in order to simplify the description, the following "cell" is only described with "embryo" as an example, but the present invention is not limited thereto, and the definition of a cell can be a germ cell, a nerve cell, a tissue cell, an animal or plant cell, or Any cell that needs to be studied and observed. In order to further enhance the accuracy of judging the quality of embryos and identifying embryos, after the
第4圖係為利用高光譜資料分析技術之人工智慧的細胞檢測系統100中,具有類神經網路的處理器13之輸入資料以及輸出資料的示意圖。如前述提及,高光譜儀12可以在兩個不同的時間點對多個胚胎拍照,以產生N張影像。N張影像可以經過影像校正程序,而轉換為N張正規化細胞高光譜影像D2。並且,藉由分析時間序列下的N張正規化細胞高光譜影像,可以決定觀測區域,並進一步產生關鍵正規化細胞高光譜影像差異特徵數值D1。關鍵正規化細胞高光譜影像差異特徵數值D1以及正規化細胞高光譜影像D2可用於訓練處理器13內的類神經網路。當類神經網路訓練完成後,處理器13即可利用人工智慧的程序判斷胚胎品質及/或辨識胚胎。處理器13可輸出細胞品質輸出資料D4。應當理解的是,N張影像中之每一張影像可為二維影像或三維影像。若N張影像中之每一張影像為二維影像時,輸入至類神經網路的資料格式可為K個維度。舉例而言,在時間點T、高光譜儀之特定波長λ下,二維座標(x,y)的畫素S1的光訊號可以表示為S1(λ,T,x,y)。畫素S1的光訊號S1(λ,T,x,y)為四個維度的訊號格式(K=4)。同理,在時間點T、高光譜儀之特定波長λ下,三維座標(x,y,z)的畫素S2的光訊號可以表示為S2(λ,T,x,y,z)。畫素S2的光訊號S2(λ,T,x,y,z)為五個維度的訊號格式(K=5)。K為大於2的正整數。因此可預期地,當細胞檢測系統100的資料格式為較高的維度時,運算複雜度將變高。當細胞檢測系統100的資料格式為較低的維度時,運算複雜度將變低。
FIG. 4 is a schematic diagram of the input data and output data of the processor 13 having a neural network in the artificial intelligence
並且,如前述提及,處理器13的類神經網路,可以利用波長正規化細胞影像之邊緣特徵數據D3進行訓練後,類神經網路可以利用人工智慧的程序優化細胞品質檢測模型。因此,如第4圖所示,處理器內的類神經網路可以接收關鍵正規化細胞高光譜影像差異特徵數值D1、正規化細胞高光譜影像D2、波長正規化細胞影像之邊緣特徵數據D3。並且,當處理器內的類神經網路訓練完成後,處理器即可利用人工智慧的程序對不孕症婦女的取卵/胚胎進行遴選,以輸 出細胞品質輸出資料D4。細胞品質輸出資料D4可為任何形式的資料格式,如輸出胚胎優劣的分級資料、輸出至少一個胚胎的優劣排序百分率、或是輸出至少一個胚胎的細胞品質。細胞品質可為細胞之詳細化學成分數值含量多寡、遺傳基因優劣,也可以為細胞於特定時間內之發育狀態的優劣,或是細胞是否發生病變而定。若為生殖細胞也可為懷孕與否、新生兒健康以及性別而定。 Moreover, as mentioned above, the neural network of the processor 13 can use the edge feature data D3 of the wavelength normalized cell image for training, and then the neural network can optimize the cell quality detection model by using artificial intelligence programs. Therefore, as shown in FIG. 4, the neural network in the processor can receive the key normalized cell hyperspectral image difference feature value D1, normalized cell hyperspectral image D2, and wavelength normalized cell image edge feature data D3. Moreover, after the neural network-like training in the processor is completed, the processor can use artificial intelligence programs to select eggs/embryos from infertile women for input Output cell quality output data D4. The cell quality output data D4 can be in any data format, such as outputting grading data of good or bad embryos, outputting the good or bad sorting percentage of at least one embryo, or outputting the cell quality of at least one embryo. Cell quality can be determined by the amount of the detailed chemical composition of the cell, the quality of the genetic gene, the quality of the developmental state of the cell within a certain period of time, or whether the cell is diseased or not. If it is a germ cell, it can also be dependent on pregnancy or not, the health of the newborn, and sex.
綜上所述,本發明描述一種利用高光譜資料分析技術之人工智慧的細胞檢測方法及其系統。細胞檢測系統的應用族群可為不孕症的婦女。醫療人員可先依據大量細胞數據,建立人工智慧之細胞品質檢測模型後,即可對不孕症的婦女進行療程。並且,人工智慧的類神經網路可以接收各波長下之高光譜資料及型態學相關的各種參數,例如關鍵正規化細胞高光譜影像差異特徵數值、正規化細胞高光譜影像、波長正規化細胞影像之邊緣特徵數據。因此,類神經網路的使用可以避免醫療人員或是研究人員以主觀的方式來判斷胚胎的優劣。不孕症的婦女可以先進行多個胚胎培養,細胞檢測系統利用人工智慧之細胞品質間檢測模型決定最佳胚胎後,再以人工的方式植入母體子宮,以增加受孕的成功率。 In summary, the present invention describes a cell detection method and system using artificial intelligence of hyperspectral data analysis technology. The application group of the cell detection system can be infertile women. Medical personnel can first establish an artificial intelligence cell quality detection model based on a large amount of cell data, and then treat infertile women with a course of treatment. Moreover, artificial intelligence-like neural networks can receive hyperspectral data at various wavelengths and various parameters related to morphology, such as key normalized cell hyperspectral image difference characteristic values, normalized cell hyperspectral images, wavelength normalized cell Image edge feature data. Therefore, the use of neural networks can prevent medical staff or researchers from judging the quality of embryos in a subjective way. Infertile women can carry out multiple embryo cultures first. The cell detection system uses the cell quality detection model of artificial intelligence to determine the best embryo, and then artificially implants into the mother's uterus to increase the success rate of conception.
以上所述僅為本發明之較佳實施例,凡依本發明申請專利範圍所做之均等變化與修飾,皆應屬本發明之涵蓋範圍。 The above descriptions are only preferred embodiments of the present invention, and all equivalent changes and modifications made according to the scope of the patent application of the present invention shall fall within the scope of the present invention.
100:利用高光譜資料分析技術之人工智慧的細胞檢測系統 100: Cell detection system using artificial intelligence of hyperspectral data analysis technology
10:載具 10: Vehicle
11:透鏡模組 11: Lens module
12:高光譜儀 12:Hyperspectrometer
13:處理器 13: Processor
14:記憶體 14: Memory
Claims (8)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201962941612P | 2019-11-27 | 2019-11-27 | |
US62/941,612 | 2019-11-27 |
Publications (2)
Publication Number | Publication Date |
---|---|
TW202121241A TW202121241A (en) | 2021-06-01 |
TWI781408B true TWI781408B (en) | 2022-10-21 |
Family
ID=75996128
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109117822A TWI754945B (en) | 2019-11-27 | 2020-05-28 | Artificial intelligence based cell detection method by using optical kinetics and system thereof |
TW109117840A TWI781408B (en) | 2019-11-27 | 2020-05-28 | Artificial intelligence based cell detection method by using hyperspectral data analysis technology |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109117822A TWI754945B (en) | 2019-11-27 | 2020-05-28 | Artificial intelligence based cell detection method by using optical kinetics and system thereof |
Country Status (2)
Country | Link |
---|---|
CN (2) | CN112862742A (en) |
TW (2) | TWI754945B (en) |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1563947A (en) * | 2004-03-18 | 2005-01-12 | 中国科学院上海技术物理研究所 | High microspectrum imaging system |
CN106815566A (en) * | 2016-12-29 | 2017-06-09 | 天津中科智能识别产业技术研究院有限公司 | A kind of face retrieval method based on multitask convolutional neural networks |
US10043054B2 (en) * | 2014-10-17 | 2018-08-07 | Cireca Theranostics, Llc | Methods and systems for classifying biological samples, including optimization of analyses and use of correlation |
CN109883966A (en) * | 2019-02-26 | 2019-06-14 | 江苏大学 | A method of Eriocheir sinensis amount of cure is detected based on multispectral image technology |
TWI664582B (en) * | 2018-11-28 | 2019-07-01 | 靜宜大學 | Method, apparatus and system for cell detection |
CN110268414A (en) * | 2016-12-01 | 2019-09-20 | 伯克利之光生命科技公司 | The automatic detection and repositioning of speck body in microfluidic device |
Family Cites Families (17)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2004088574A1 (en) * | 2003-04-02 | 2004-10-14 | Amersham Biosciences Uk Limited | Method of, and computer software for, classification of cells into subpopulations |
US7711174B2 (en) * | 2004-05-13 | 2010-05-04 | The Charles Stark Draper Laboratory, Inc. | Methods and systems for imaging cells |
CN101331500B (en) * | 2005-10-14 | 2015-04-29 | 尤尼森斯繁殖技术公司 | Determination of a change in a cell population |
US8744775B2 (en) * | 2007-12-28 | 2014-06-03 | Weyerhaeuser Nr Company | Methods for classification of somatic embryos comprising hyperspectral line imaging |
JP2009229274A (en) * | 2008-03-24 | 2009-10-08 | Nikon Corp | Method for analyzing image for cell observation, image processing program and image processor |
CN101903532A (en) * | 2008-03-24 | 2010-12-01 | 株式会社尼康 | Method for analyzing image for cell observation, image processing program, and image processing device |
CN102803952A (en) * | 2009-06-25 | 2012-11-28 | 耶路撒冷希伯来大学伊萨姆研发有限公司 | Hyperspectral identification of egg fertility and gender |
AU2012280743A1 (en) * | 2011-07-02 | 2014-01-16 | Unisense Fertilitech A/S | Adaptive embryo selection criteria optimized through iterative customization and collaboration |
EP2757372B1 (en) * | 2011-09-16 | 2023-08-23 | AVE Science & Technology Co., Ltd. | Device and method for erythrocyte morphology analysis |
EP2890781B1 (en) * | 2012-08-30 | 2020-09-23 | Unisense Fertilitech A/S | Automatic surveillance of in vitro incubating embryos |
US9971966B2 (en) * | 2016-02-26 | 2018-05-15 | Google Llc | Processing cell images using neural networks |
CN106226247A (en) * | 2016-07-15 | 2016-12-14 | 暨南大学 | A kind of cell detection method based on EO-1 hyperion micro-imaging technique |
CN107064019B (en) * | 2017-05-18 | 2019-11-26 | 西安交通大学 | The device and method for acquiring and dividing for dye-free pathological section high spectrum image |
US20200202051A1 (en) * | 2017-06-16 | 2020-06-25 | Ge Healthcare Bio-Sciences Ab | Method for Predicting Outcome of an Modelling of a Process in a Bioreactor |
CN108550133B (en) * | 2018-03-02 | 2021-05-18 | 浙江工业大学 | Cancer cell detection method based on fast R-CNN |
CN110136775A (en) * | 2019-05-08 | 2019-08-16 | 赵壮志 | A kind of cell division and anti-interference detection system and method |
CN110390676A (en) * | 2019-07-26 | 2019-10-29 | 腾讯科技(深圳)有限公司 | The cell detection method of medicine dye image, intelligent microscope system under microscope |
-
2020
- 2020-05-28 TW TW109117822A patent/TWI754945B/en active
- 2020-05-28 TW TW109117840A patent/TWI781408B/en active
- 2020-06-03 CN CN202010493873.6A patent/CN112862742A/en active Pending
- 2020-06-04 CN CN202010498111.5A patent/CN112862743A/en active Pending
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN1563947A (en) * | 2004-03-18 | 2005-01-12 | 中国科学院上海技术物理研究所 | High microspectrum imaging system |
US10043054B2 (en) * | 2014-10-17 | 2018-08-07 | Cireca Theranostics, Llc | Methods and systems for classifying biological samples, including optimization of analyses and use of correlation |
CN110268414A (en) * | 2016-12-01 | 2019-09-20 | 伯克利之光生命科技公司 | The automatic detection and repositioning of speck body in microfluidic device |
CN106815566A (en) * | 2016-12-29 | 2017-06-09 | 天津中科智能识别产业技术研究院有限公司 | A kind of face retrieval method based on multitask convolutional neural networks |
TWI664582B (en) * | 2018-11-28 | 2019-07-01 | 靜宜大學 | Method, apparatus and system for cell detection |
CN109883966A (en) * | 2019-02-26 | 2019-06-14 | 江苏大学 | A method of Eriocheir sinensis amount of cure is detected based on multispectral image technology |
Also Published As
Publication number | Publication date |
---|---|
TWI754945B (en) | 2022-02-11 |
CN112862743A (en) | 2021-05-28 |
TW202120906A (en) | 2021-06-01 |
CN112862742A (en) | 2021-05-28 |
TW202121241A (en) | 2021-06-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
JP7072067B2 (en) | Systems and methods for estimating embryo viability | |
US9619881B2 (en) | Method and system for characterizing cell populations | |
WO2021139258A1 (en) | Image recognition based cell recognition and counting method and apparatus, and computer device | |
CN110763678A (en) | Pathological section interpretation method and system | |
CN109564680A (en) | Information processing equipment, information processing method and program | |
JPWO2018101004A1 (en) | Cell image evaluation apparatus and cell image evaluation control program | |
US20220383986A1 (en) | Complex System for Contextual Spectrum Mask Generation Based on Quantitative Imaging | |
US20040014165A1 (en) | System and automated and remote histological analysis and new drug assessment | |
TWI781408B (en) | Artificial intelligence based cell detection method by using hyperspectral data analysis technology | |
US10748288B2 (en) | Methods and systems for determining quality of an oocyte | |
WO2018193612A1 (en) | Correlation calculation device, correlation calculation method, and correlation calculation program | |
WO2023154851A1 (en) | Integrated framework for human embryo ploidy prediction using artificial intelligence | |
Malmsten et al. | Automated cell stage predictions in early mouse and human embryos using convolutional neural networks | |
Zeman et al. | Deep learning for human embryo classification at the cleavage stage (Day 3) | |
US20210174147A1 (en) | Operating method of image processing apparatus, image processing apparatus, and computer-readable recording medium | |
JP2022547900A (en) | Automated evaluation of quality assurance metrics used in assisted reproductive procedures | |
Betegón-Putze et al. | MyROOT: A novel method and software for the semi-automatic measurement of plant root length | |
AU2019101174A4 (en) | Systems and methods for estimating embryo viability | |
RU2800079C2 (en) | Systems and methods of assessing the viability of embryos | |
Sharma et al. | Deep learning methods to forecasting human embryo development in time-lapse videos | |
RU2810125C1 (en) | Automated assessment of quality assurance indicators for assisted reproduction procedures | |
JP7274936B2 (en) | Cell image compression device, method and program | |
Eswaran et al. | Assessment of Human Blastocyst using Deep Learning Algorithm | |
JP2018116376A (en) | Image processor and image processing method | |
JP2024513659A (en) | Systems and methods for assessing embryo viability using artificial intelligence |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
GD4A | Issue of patent certificate for granted invention patent |