AU2021231904B2 - Determining tissue characteristics using multiplexed immunofluorescence imaging - Google Patents

Determining tissue characteristics using multiplexed immunofluorescence imaging Download PDF

Info

Publication number
AU2021231904B2
AU2021231904B2 AU2021231904A AU2021231904A AU2021231904B2 AU 2021231904 B2 AU2021231904 B2 AU 2021231904B2 AU 2021231904 A AU2021231904 A AU 2021231904A AU 2021231904 A AU2021231904 A AU 2021231904A AU 2021231904 B2 AU2021231904 B2 AU 2021231904B2
Authority
AU
Australia
Prior art keywords
cell
cells
image
mxif
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
AU2021231904A
Other languages
English (en)
Other versions
AU2021231904A1 (en
Inventor
Ravshan Ataullakhanov
Alexander BAGAEV
Ilia GALKIN
Pavel OVCHAROV
Ekaterina POSTOVALOVA
Viktor SVEKOLKIN
Arina VARLAMOVA
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
BostonGene Corp
Original Assignee
BostonGene Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by BostonGene Corp filed Critical BostonGene Corp
Publication of AU2021231904A1 publication Critical patent/AU2021231904A1/en
Priority to AU2025201658A priority Critical patent/AU2025201658A1/en
Application granted granted Critical
Publication of AU2021231904B2 publication Critical patent/AU2021231904B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2323—Non-hierarchical techniques based on graph theory, e.g. minimum spanning trees [MST] or graph cuts
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/243—Classification techniques relating to the number of classes
    • G06F18/2431—Multiple classes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/174—Segmentation; Edge detection involving the use of two or more images
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G06V10/443—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components by matching or filtering
    • G06V10/449—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters
    • G06V10/451—Biologically inspired filters, e.g. difference of Gaussians [DoG] or Gabor filters with interaction between the filter responses, e.g. cortical complex cells
    • G06V10/454—Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G06V10/7635—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks based on graphs, e.g. graph cuts or spectral clustering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/695—Preprocessing, e.g. image segmentation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/60—Type of objects
    • G06V20/69—Microscopic objects, e.g. biological cells or cellular parts
    • G06V20/698—Matching; Classification
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10056—Microscopic image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10064—Fluorescence image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30024—Cell structures in vitro; Tissue sections in vitro

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Evolutionary Computation (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Molecular Biology (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Quality & Reliability (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Discrete Mathematics (AREA)
  • Investigating Or Analysing Biological Materials (AREA)
  • Image Analysis (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Image Processing (AREA)
  • Measuring Or Testing Involving Enzymes Or Micro-Organisms (AREA)
AU2021231904A 2020-03-06 2021-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging Active AU2021231904B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2025201658A AU2025201658A1 (en) 2020-03-06 2025-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202062986010P 2020-03-06 2020-03-06
US62/986,010 2020-03-06
PCT/US2021/021265 WO2021178938A1 (en) 2020-03-06 2021-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging

Related Child Applications (1)

Application Number Title Priority Date Filing Date
AU2025201658A Division AU2025201658A1 (en) 2020-03-06 2025-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging

Publications (2)

Publication Number Publication Date
AU2021231904A1 AU2021231904A1 (en) 2022-09-22
AU2021231904B2 true AU2021231904B2 (en) 2025-04-10

Family

ID=75340246

Family Applications (2)

Application Number Title Priority Date Filing Date
AU2021231904A Active AU2021231904B2 (en) 2020-03-06 2021-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging
AU2025201658A Pending AU2025201658A1 (en) 2020-03-06 2025-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging

Family Applications After (1)

Application Number Title Priority Date Filing Date
AU2025201658A Pending AU2025201658A1 (en) 2020-03-06 2025-03-06 Determining tissue characteristics using multiplexed immunofluorescence imaging

Country Status (7)

Country Link
US (3) US11250568B2 (https=)
EP (1) EP4115324A1 (https=)
JP (2) JP7592737B2 (https=)
AU (2) AU2021231904B2 (https=)
CA (1) CA3172827A1 (https=)
IL (1) IL295881B1 (https=)
WO (1) WO2021178938A1 (https=)

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7592737B2 (ja) 2020-03-06 2024-12-02 ボストンジーン コーポレイション 多重免疫蛍光イメージングを使用する組織特性の決定
JP7071037B2 (ja) * 2020-04-28 2022-05-18 ゼネラル・エレクトリック・カンパニイ 推論装置、医用システム、およびプログラム
CN113628157A (zh) * 2020-05-06 2021-11-09 德克萨斯大学体系董事会 利用病理图像来表征肿瘤微环境的系统和方法
WO2022079919A1 (ja) * 2020-10-16 2022-04-21 富士通株式会社 検知プログラム、検知方法および検知装置
EP4238052A1 (en) * 2020-10-27 2023-09-06 Ventana Medical Systems, Inc. Identifying auto-fluorescent artifacts in a multiplexed immunofluorescent image
WO2022173828A1 (en) * 2021-02-09 2022-08-18 The Board Of Trustees Of The Leland Stanford Junior University Identification of cell types in multiplexed in situ images by combining expression profiling and spatial information
US11636663B2 (en) * 2021-02-19 2023-04-25 Microsoft Technology Licensing, Llc Localizing relevant objects in multi-object images
EP4334838A1 (en) * 2021-05-07 2024-03-13 X-Zell Biotech Pte Ltd Systems and methods for identifying and classifying rare cells
WO2022251556A1 (en) * 2021-05-27 2022-12-01 Genentech, Inc. Tumor immunophenotyping based on spatial distribution analysis
DE102021126965A1 (de) * 2021-10-18 2023-05-04 Mira Vision Holding Gmbh Computer-implementiertes Verfahren zur Erzeugung synthetischer Bilder zum Trainieren künstlicher Intelligenz (AI)
US12056857B2 (en) * 2021-11-05 2024-08-06 Adobe Inc. Digital image inpainting utilizing plane panoptic segmentation and plane grouping
CN114399470B (zh) * 2021-12-21 2024-12-20 北京大学 超高时空分辨率细胞牵引力定量方法、装置、设备及介质
CN114240907B (zh) * 2021-12-21 2025-05-23 上海元芒数字科技有限公司 一种基于电子秤的商品展示图自动生成的算法
CN115170599B (zh) * 2022-05-25 2026-03-03 北京大学 通过图神经网络的链路预测进行血管分割的方法和装置
CN117237980B (zh) * 2022-06-06 2025-09-19 上海思路迪生物医学科技有限公司 多重荧光免疫组化图像识别方法及相关装置
CN115019142B (zh) * 2022-06-14 2024-03-29 辽宁工业大学 基于融合特征的图像标题生成方法、系统、电子设备
JP2024017211A (ja) * 2022-07-27 2024-02-08 セイコーエプソン株式会社 画像処理シーケンスの生成方法、生成装置、および、コンピュータープログラム
EP4616365A1 (en) * 2022-11-10 2025-09-17 Nucleai Ltd. System and method for multiplex imaging cell typing and phenotypic marker quantification
WO2025093815A1 (en) * 2023-10-30 2025-05-08 Helsingin Yliopisto Method for an image analysis pipeline
WO2025096811A1 (en) 2023-10-31 2025-05-08 Bostongene Corporation Machine learning technique for identifying ici responders and non-responders
US20250266163A1 (en) * 2024-02-20 2025-08-21 Siemens Healthineers Ag Vulnerable plaque assessment and outcome prediction in coronary artery disease
WO2025198590A1 (en) * 2024-03-20 2025-09-25 Hitachi America, Ltd. Set prediction for visual inspection

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110091091A1 (en) * 2009-10-16 2011-04-21 General Electric Company Process and system for analyzing the expression of biomarkers in cells
US20160341731A1 (en) * 2015-05-21 2016-11-24 Ge Healthcare Bio-Sciences Corp. Method and system for classification and quantitative analysis of cell types in microscopy images

Family Cites Families (25)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101916359B (zh) * 2005-01-27 2012-06-20 剑桥研究和仪器设备股份有限公司 把样本的不同区域分类到相应类别的方法和设备
US9779213B2 (en) * 2008-07-25 2017-10-03 Fundacao D. Anna Sommer Champalimaud E Dr. Carlos Montez Champalimaud System for evaluating a pathological stage of prostate cancer
US20110091081A1 (en) 2009-10-16 2011-04-21 General Electric Company Method and system for analyzing the expression of biomarkers in cells in situ in their tissue of origin
US20120010528A1 (en) * 2010-04-26 2012-01-12 Aureon Biosciences, Inc. Systems and methods for predicting disease progression in patients treated with radiotherapy
JP5870821B2 (ja) 2012-04-03 2016-03-01 国立研究開発法人理化学研究所 多重蛍光画像の画像解析のための装置、システム、方法、およびプログラム
AU2014217893A1 (en) 2013-02-18 2015-07-30 Theranos Ip Company, Llc Systems and methods for multi-analysis
CN105209636A (zh) * 2013-03-15 2015-12-30 麦塔马克基因股份有限公司 用于癌症预后的组合物和方法
US9064304B2 (en) * 2013-03-18 2015-06-23 General Electric Company Image quality assessment of microscopy images
JP6629762B2 (ja) 2014-05-23 2020-01-15 ベンタナ メディカル システムズ， インコーポレイテッド 画像内の生物学的構造及び／又はパターンの検出のためのシステム及び方法
CN107209934B (zh) 2014-12-03 2021-03-30 文塔纳医疗系统公司 用于定量分析异质生物标志物分布的方法、系统和装置
AU2015373423A1 (en) 2014-12-30 2017-05-18 Ventana Medical Systems, Inc. Systems and methods for co-expression analysis in immunoscore computation
US9607375B2 (en) * 2015-06-03 2017-03-28 Eileen B. Gallagher Biological data annotation and visualization
WO2016201186A1 (en) 2015-06-11 2016-12-15 University Of Pittsburgh-Of The Commonwealth System Of Higher Education Systems and methods for finding regions of interest in hematoxylin and eosin (h&e) stained tissue images and quantifying intratumor cellular spatial heterogeneity in multiplexed/hyperplexed fluorescence tissue images
AU2016341306B2 (en) 2015-10-23 2022-10-27 Novartis Ag Computer processes behind an enhanced version of aqua
EP3664705A4 (en) 2017-08-09 2021-09-29 Allen Institute IMAGE PROCESSING SYSTEMS, DEVICES AND METHODS FOR GENERATING AN IMAGE WITH PREDICTIVE MARKING
EP3721406B1 (en) 2017-12-05 2024-01-24 Ventana Medical Systems, Inc. Method of computing tumor spatial and inter-marker heterogeneity
US11307142B2 (en) 2018-05-03 2022-04-19 Akoya Biosciences, Inc. Multispectral sample imaging
EP3591574A3 (en) 2018-07-06 2020-01-15 Universität Zürich Method and computer program for clustering large multiplexed spatially resolved data of a biological sample
US11756203B2 (en) * 2018-09-21 2023-09-12 The Johns Hopkins University Characterization platform for scalable, spatially-resolved multispectral analysis of tissue
CN110060245A (zh) * 2019-04-15 2019-07-26 青岛大学 一种基于深度神经网络的胃癌淋巴结染色病理图像自动识别方法
US12019674B2 (en) * 2019-05-29 2024-06-25 New York University System and method for tumor characterization
US12094105B2 (en) * 2019-06-10 2024-09-17 Datma, Inc. System and method for automatic labeling of pathology images
KR20220019778A (ko) 2019-06-12 2022-02-17 퀀텀-에스아이 인코포레이티드 머신 학습을 이용한 단백질 식별을 위한 기법들 및 관련된 시스템들 및 방법들
US20200394459A1 (en) * 2019-06-17 2020-12-17 Nvidia Corporation Cell image synthesis using one or more neural networks
JP7592737B2 (ja) 2020-03-06 2024-12-02 ボストンジーン コーポレイション 多重免疫蛍光イメージングを使用する組織特性の決定

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110091091A1 (en) * 2009-10-16 2011-04-21 General Electric Company Process and system for analyzing the expression of biomarkers in cells
US20160341731A1 (en) * 2015-05-21 2016-11-24 Ge Healthcare Bio-Sciences Corp. Method and system for classification and quantitative analysis of cell types in microscopy images

Also Published As

Publication number Publication date
AU2021231904A1 (en) 2022-09-22
US20240161284A1 (en) 2024-05-16
US11250568B2 (en) 2022-02-15
JP2023517309A (ja) 2023-04-25
JP7824383B2 (ja) 2026-03-04
CA3172827A1 (en) 2021-09-10
JP7592737B2 (ja) 2024-12-02
EP4115324A1 (en) 2023-01-11
US20210279866A1 (en) 2021-09-09
IL295881A (en) 2022-10-01
US11915422B2 (en) 2024-02-27
AU2025201658A1 (en) 2025-03-27
US20220122252A1 (en) 2022-04-21
WO2021178938A1 (en) 2021-09-10
IL295881B1 (en) 2026-01-01
JP2025037907A (ja) 2025-03-18

Similar Documents

Publication Publication Date Title
AU2021231904B2 (en) Determining tissue characteristics using multiplexed immunofluorescence imaging
JP7583041B2 (ja) 組織画像分類用のマルチインスタンス学習器
US20240233416A1 (en) Analysis of histopathology samples
US20240054639A1 (en) Quantification of conditions on biomedical images across staining modalities using a multi-task deep learning framework
AU2021349226C1 (en) Critical component detection using deep learning and attention
US9111179B2 (en) High-throughput biomarker segmentation utilizing hierarchical normalized cuts
JP2022527145A (ja) 予測的組織パターン特定のためのマルチプルインスタンスラーナ
CN120259303B (zh) 一种生物微环境图像分析与分类方法
Eldho et al. Lung cancer detection and severity analysis with a 3D deep learning CNN Model using CT-DICOM clinical dataset
Bull et al. Extended correlation functions for spatial analysis of multiplex imaging data
Saranyaraj et al. Early prediction of breast cancer based on the classification of HER‐2 and ER biomarkers using deep neural network
Chen et al. An interpretable Algorithm for uveal melanoma subtyping from whole slide cytology images
WO2025096951A1 (en) Tissue-based classifier of allograft inflammation
Meloni AI-driven morphological clustering for lymphoma stratification
Wright Automated analysis of colorectal cancer
Dovbysh et al. Decision Support System for Diagnosing Early Stages of Prostate Cancer Based on Pathomorphological Features
Gortana et al. HEDeST: An Integrative Approach to Enhance Spatial Transcriptomic Deconvolution with Histology
Zhou IfThresholds Software Package: score ranking standard threshold methods by comparing images produced by thresholding with manually labeled images
Kothari Developing Image Informatics Methods for Histopathological Computer-Aided Decision Support Systems
GALLO Explainable AI in Digital Pathology

Legal Events

Date Code Title Description
FGA Letters patent sealed or granted (standard patent)