WO2011071363A2 - Système et procédé permettant de visualiser et d'apprendre l'anatomie humaine - Google Patents

Système et procédé permettant de visualiser et d'apprendre l'anatomie humaine Download PDF

Info

Publication number
WO2011071363A2
WO2011071363A2 PCT/MY2010/000258 MY2010000258W WO2011071363A2 WO 2011071363 A2 WO2011071363 A2 WO 2011071363A2 MY 2010000258 W MY2010000258 W MY 2010000258W WO 2011071363 A2 WO2011071363 A2 WO 2011071363A2
Authority
WO
WIPO (PCT)
Prior art keywords
interest
semantic
human
concepts
reconstructed
Prior art date
Application number
PCT/MY2010/000258
Other languages
English (en)
Other versions
WO2011071363A3 (fr
Inventor
Weng Onn Kow
Mohammad Reza Beikzadeh
Dickson Lukose
Pramod G.Bagali
Rajan Bharadwaj
Singh Armandeep
Abdul Qayoom
Sachim Angrish
Original Assignee
Mimos Berhad
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mimos Berhad filed Critical Mimos Berhad
Publication of WO2011071363A2 publication Critical patent/WO2011071363A2/fr
Publication of WO2011071363A3 publication Critical patent/WO2011071363A3/fr

Links

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • G09B5/02—Electrically-operated educational appliances with visual presentation of the material to be studied, e.g. using film strip
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
    • G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00—Electrically-operated teaching apparatus or devices working with questions and answers
    • G09B7/02—Electrically-operated teaching apparatus or devices working with questions and answers of the type wherein the student is expected to construct an answer to the question which is presented or wherein the machine gives an answer to the question presented by a student

Definitions

  • the present invention relates generally to a field of medical knowledge, more particularly to a system and method for visualizing and learning of human anatomy.
  • CT computed tomography
  • a system for semantic images browsing and navigation for visualizing and learning of human anatomy comprises an information database containing records of medical knowledge bases; a first display interface which allows user to manipulate an ontology for requested concepts on subject of interest, wherein the concepts and its knowledge bases in the information database having tags which are harmonized and synchronized; a base server connected to the information database and first display interface, a graphical database containing three-dimensional images of human anatomy; a second display interface which allows user to manipulate images of subject of interest, wherein the images having tags which are harmonized and synchronized, and a graphic engine (16) connected to the graphical database and second display interface) , the graphic engine is linked to the base server for fetching the corresponding concept or image upon user requests .
  • Figure 1 shows a diagram of a system for semantic images browsing and navigation for visualizing and learning of human anatomy of the present invention where Figure la showing the user using the semantic browser to interact with the system and Figure lb showing the user using the 3D visualizer;
  • Figure 2 depicts a scalable architecture of the present system
  • Figure 3 shows a plurality of individual MSCT images generated by scanning a human head
  • Figure 4 shows a generated 3D skull using the present system
  • Figure 5 illustrates a screenshot of system with the semantic browser window where on the right shows the skull concept
  • Figure 6 is a diagram showing the semantic technology platform for virtual anatomy architecture.
  • Figure 7 is a flowchart of a method for semantic images browsing and navigation for visualizing and learning of human anatomy of the present invention.
  • the system (10) provides the user with a multimodal approach to access medical knowledge.
  • the system (10) includes a 3D imaging program which takes DICOM data from multi slice CT scans which are two dimensional of the entire human body and reconstructs a three dimensional image body structure. These 3D visuals are built from real scans where the characteristics and dimensions of visualized human body and organs are more accurate than drawings in books or molded models.
  • User (20) may interact with the visuals of the system (10) and manipulate the images by zooming in and out, rotating and even slicing the anatomical part being ' displayed through the use of a graphical semantic browser (11) or a 3D visualizer (12) as shown in Figure la and lb respectively.
  • the image can also be annotated with text and saved to a file for future reference.
  • the system (10) also includes a semantic web of an information database (13) where the necessary medical knowledge bases from various medical topics or subjects are kept therein and a graphical database (14) where the 3D images of the human body and organs are kept therein.
  • the knowledge is linked semantically around a human anatomy to allow user to explore and navigate through the linked knowledge for both learning and reference purposes.
  • the images data is acquired from the MSCT scan of the human body.
  • Figure 3 shows the MSCT images from scanning a human head.
  • the data is generated in a DICOM format which is the international standard for Diagnostic Communication in Medicine.
  • the images will be on a specific plane mostly axial.
  • There are numerous parameters that determine the data acquisition including slice thickness, slice interval, vp, mA, milliseconds of exposure etc.
  • DCM parser will then separate the non co-axial slices from the rest.
  • the images data is then rendered in 3D after preprocessing of this large data, which can be up to approximately 10 to 20 gigabytes of size.
  • the proprietary large data management (LDM) algorithms do an efficient handling of this big data, . which cuts down time to few seconds. This is a real volumetric data, and the user can intuitively interact with it by operations such as zooming, panning and rotating can be done easily on the image rendered.
  • Figure 4 shows the high definition visuals of the 3D skull generated by the system (10) .
  • the semantic browser (11) is a graphical user interface that takes a semantic web in the form of an RDF file and shows portions of it in the form of nodes and links.
  • the semantic browser (11) includes three main views graphical, HTML and type hierarchy. In graphical view, concepts and relations are shown as nodes and links between nodes. It uses a focus node concept, where the currently selected concept and its links are shown in detail while the other nodes and links are minimized into the background. The depiction of concepts and relations as nodes and links between the nodes is not new. However, other features to make it even easier for the user to understand and relate the knowledge they see in the browser are added into the present system. For example, different coloured nodes and icons are used to represent different classes of concepts. Similarly for the links between the nodes. Self-configuring layouts offer a balance between clarity and depth of knowledge displayed in the browser (11) .
  • All navigation can be done using the mouse button with the ability to change the graphical display to the user' s liking such as the ability to zoom in and out, and to drag the nodes and screen around.
  • the user also can limit what knowledge he wants to view by filtering out all relations that he is not interested in. He can hide and show relations and nodes at any time.
  • the concepts shown by the semantic browser (11) need not be just in text form. If the knowledge base contains links to artifacts such as documents, web pages and multimedia files, the browser (11) can launch the relevant application to view these artifacts when they are selected.
  • HTML view concepts are represented as a dynamically generated HTML page with relations being shown as header titles and the target concepts listed beneath the headers as links or text.
  • the knowledge base is shown as a three dimensional tree.
  • the current selected concept is shown along with its child concepts (if any) and a path through its ancestors to the root of the tree.
  • the browser (11) includes a keyword search function that allows the user to search for any concept just by typing in part of the concept name.
  • the browser (11) will list all concepts that match the typed characters.
  • semantic query function (17) that allows the user to type in structured natural language questions such as "What is the common location of Osteoid Osteoma?” and get the answer returned in natural language if it can be found in the knowledge base as shown in Figure 6. The user can then select the anatomy portion mentioned in the answer, and the image of the part is shown along with its semantic we .
  • the knowledge information in the semantic web (13) and the images in the graphical database (14) are tagged with tags and synchronized to each other so that when the user selects a concept in the browser (11), then the relevant anatomy image (if any) is shown in the visualizer (12). Similarly, selecting a portion of the anatomy in the visualizer (12) leads to semantic information about that concept to be shown in the browser window (11) as sho.wn in Figure 5.
  • the tags of the central concept on display in the semantic browser (11) will be transmitted by a base server (15) which is linked to a graphic engine (16) to fetch the corresponding image from the graphical database (14) based on the tags and displays it on the 3D visualizer (12) as shown in Figure la.
  • the graphic engine (16) will send the tags associated to the image to the base server (15) which will retrieve the corresponding concept and render it on the semantic browser (11) .
  • the tibia bone is shown in 3D and similarly, if the user chooses to view the heart in the 3D viewer, the concept of the heart and its related concepts are shown in ( the browser .
  • the system (10) also has unlimited scalability. It can work as a stand alone, as well as for large number of concurrent users on a distributed architecture as shown in Figure 2.
  • the present system (10) is using a method for inter-linking and navigating bi-directionally between the 3D reconstructed digital human anatomical and sub-cellular DICOM volumes.
  • the method includes providing a plurality of 3D reconstructed human gross anatomy and sub-cellular histological as well as histopathological volume for medical teaching and training of a structure of interest against physiological functionality or deficits in human which is to be con-currently displayed on a semantic web using a service oriented architecture.
  • the digital anatomical reconstructed 3D structure comprising a plurality of intensities corresponding to a domain of points on a 3-dimensional grid.
  • the structure of interest for each image is tagged and annotated to form a 3D mesh of points.
  • the motion and orientation between tags or annotations and structure of interest within the 3D grid are harmonized and synchronized.
  • User may constantly point at the same structure of interest at all times irrespective of position in the 3D grid axis or motion without the annotations being hide, lost or distorted from viewing.
  • the annotations always orient itself to the region of the body where it was pointing originally against it intended region.
  • the spatial position of annotations is synchronized to its specific region in 3D volume which makes it to point to the original region.
  • Each mesh is then oriented, organized and aligned wherein a registration transformation between each pair of 3D reconstructed volume is calculated.
  • Various measurement parameters as comparison between abnormal ⁇ structures of interest and normal human anatomical structure within the aligned mesh are then calculated and displayed.
  • the normal and abnormal with pathophysiological findings of 3D reconstructed human body volumes from plurality of images can be retrieved and displayed.
  • a feature vector for each structure of interest in the plurality of the gross and sub-cellular human anatomy and pathophysiological volumes is then calculated.
  • a classifier is then trained using boosting to categorize key meaning in the medical ontology, comment and annotation mapped against the structure of interest in the plurality of 3D reconstructed gross human bodies into a predefined category based on the meaning, association and complexity of each structure of interest, where the classifier is adapted to segmenting a corresponding structure of interest from a plurality of 3D reconstructed human bodies.
  • a plurality of bi-directional links between anatomical or pathological information in association with the structure of interest; in a semantic web of medical ontology; to its corresponding MPR (Multiplanar reformation) is created and the links which using the service oriented architecture are adapted to facilitate navigation, derivation of information and related medical knowledge through the established bidirectional linkage.
  • Pixel segmentation, isolation and enhancement for small features and structure or point of interest in a 3D reconstructed gross and sub-cellular human anatomy and pathophysiology volume that needs clarity and greater definition in multi-dimensional data can be provided to define small anatomical feature demarcation that correspond closely to those selected by the user but does so with less complexity.
  • Hounsfield units in a plurality of 3D reconstructed human gross anatomy and sub-cellular structures allowing comparison of pathopysiological between abnormal and normal structures are calculated and compared.
  • the density and volume calculation can be provided to identify the pathological changes.
  • the query to identify one or more ontology or overall meaning in the query is then parsed.
  • the meaning to the query is then mapped and flagged to a corresponding structure in the anatomical structure against spatial information to associate such medical information to the location within a physiological system, where the 3D reconstructed gross human anatomical and pathopysiological and histological structure images are associated with at least one of the links to the corresponding structure in the image.
  • at least one link to inter-relate, inter-link bi-directionally is performed and the human anatomical structure of interest mapped against the established semantic web medical ontology is displayed.
  • a list of highly customized forensic pathology-specific pre-sets within the tool library is used to quickly narrow down to an area or structure of interest.
  • a pre-set and pre-trained classifier is also used to isolate and identify the structure of interest from the volume within a plurality of 3D reconstructed human bodies. Then bi- directional links between the structure of interest to a corresponding semantic web of medical ontology are created and the links which using the service oriented architecture are adapted to facilitate navigation and derivation of information and related medical knowledge through the 3D reconstructed human gross anatomy and pathophysiological structures to the structure of interest.
  • the cause-and-effect medical knowledge specifically within the possible and ideal pharmaceutical intervention, potential microbiological aetiology causing pathophysiological changes grossly and sub-cellularly as well as traumatic events that affects these changes using the semantic ontology model and associating these set of domain knowledge and expert opinions bi-directionally to a specific anatomical or histological structure of interest are also developed.
  • the system (10) allows saving, storing, viewing and replaying chronological video for segmentation, association between the structure of interest against the medical ontology and knowledge base of the structure of interest and the links to the corresponding structure in a metafile.
  • the snapshots capturing both the medical ontology and structure of interest volume are used to be tagged and annotated for future reference or replaying in sequential manner or as a part of a case file to narrate the chronology for sharing and learning.
  • Figure 7 shows a flowchart of the steps in using the present system (10) .
  • the concepts from the ontology will be loaded (21) onto the semantic browser (11) .
  • the image of the full human anatomy will be loaded (22) onto the 3D visualizer (12) .
  • User may choose to manipulate (23) the ontology from the semantic browser (11) or manipulate (24) the image from the 3D visualizer (12) .
  • the base server (15) will identify (25) the central concept displayed on the semantic browser (11) and send the associated tags to the graphic engine (16) and the corresponding image will be displayed on the 3D visualizer (12) .
  • 3D visualizations can be implemented in the existing curricula of the medical, nursing, first aid and physiotherapy programs. This can be used for theory lectures, practical demonstrations and tutorial sessions. This is also ideal for Self-study as students cannot have access to a dissection cadaver for 24 hours a day.
  • the 3D visualization solution will definitely stimulate the students to understand more and help them to get insights about pathological, microbiological variations and different organs size, space extent and relation to each other.
  • the virtual dissections will give a clearer picture than ordinary dissections and the possibility to turn structures around will be self instructive. Since this is based on authentic, true human scanning data, it will add a new dimension of learning material in anatomy, physiology and probably also pathophysiology .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Medical Informatics (AREA)
  • Chemical & Material Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Medicinal Chemistry (AREA)
  • Algebra (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Mathematical Physics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Medical Treatment And Welfare Office Work (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

Un système permettant de parcourir des images sémantiques et de naviguer entre celles-ci pour visualiser et apprendre l'anatomie humaine comprend une base de données d'informations (13) comprenant des enregistrements de bases de connaissances médicales; une première interface d'affichage (11) qui permet à un utilisateur de manipuler une ontologie pour des concepts demandés sur un sujet d'intérêt, les concepts et leurs bases de connaissances dans la base de données d'informations (13) comprenant des balises qui sont harmonisées et synchronisées; et un serveur de base (15) connecté à la base de données d'informations (13) et à la première interface d'affichage (11). Le système (10) comprend également une base de données graphique (14) contenant des images tridimensionnelles de l'anatomie humaine; une seconde interface d'affichage (12) qui permet à un utilisateur de manipuler les images d'un sujet d'intérêt, les images comportant des balises qui sont harmonisées et synchronisées; et un moteur graphique (16) connecté à la base de données graphique (14) et à la seconde interface d'affichage (12), le moteur graphique (16) étant relié au serveur de base (15) pour extraire le concept correspondant ou l'image correspondante à la demande d'un utilisateur.
PCT/MY2010/000258 2009-12-09 2010-11-10 Système et procédé permettant de visualiser et d'apprendre l'anatomie humaine WO2011071363A2 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
MYPI20095245 2009-12-09
MYPI20095245 MY148824A (en) 2009-12-09 2009-12-09 System and method for visualizing and learning of human anatomy

Publications (2)

Publication Number Publication Date
WO2011071363A2 true WO2011071363A2 (fr) 2011-06-16
WO2011071363A3 WO2011071363A3 (fr) 2011-11-10

Family

ID=44146087

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/MY2010/000258 WO2011071363A2 (fr) 2009-12-09 2010-11-10 Système et procédé permettant de visualiser et d'apprendre l'anatomie humaine

Country Status (2)

Country Link
MY (1) MY148824A (fr)
WO (1) WO2011071363A2 (fr)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013003571A1 (fr) * 2011-06-29 2013-01-03 The Johns Hopkins University Système pour interface tridimensionnelle et base de données
US9691156B2 (en) 2012-02-01 2017-06-27 Koninklijke Philips N.V. Object image labeling apparatus, method and program
CN109166183A (zh) * 2018-07-16 2019-01-08 中南大学 一种解剖标志点识别方法及识别设备
CN113012781A (zh) * 2021-02-07 2021-06-22 重庆三峡医药高等专科学校 一种药理学数字人系统
US11593691B2 (en) 2016-06-30 2023-02-28 Koninklijke Philips N.V. Information retrieval apparatus

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997012356A1 (fr) * 1993-12-30 1997-04-03 Kuch Nina J Technologie multimedia pour l'education nutritionnelle et la planification de regimes
US20040064298A1 (en) * 2002-09-26 2004-04-01 Robert Levine Medical instruction using a virtual patient
US20070065793A1 (en) * 1998-11-13 2007-03-22 Anuthep Benja-Athon Hybrid intelligence in medicine
US20080249802A1 (en) * 2006-01-30 2008-10-09 Sandro Micieli Intelligent medical device
US20090202972A1 (en) * 2008-02-12 2009-08-13 Immersion Corporation Bi-Directional Communication of Simulation Information

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1997012356A1 (fr) * 1993-12-30 1997-04-03 Kuch Nina J Technologie multimedia pour l'education nutritionnelle et la planification de regimes
US20070065793A1 (en) * 1998-11-13 2007-03-22 Anuthep Benja-Athon Hybrid intelligence in medicine
US20040064298A1 (en) * 2002-09-26 2004-04-01 Robert Levine Medical instruction using a virtual patient
US20080249802A1 (en) * 2006-01-30 2008-10-09 Sandro Micieli Intelligent medical device
US20090202972A1 (en) * 2008-02-12 2009-08-13 Immersion Corporation Bi-Directional Communication of Simulation Information

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2013003571A1 (fr) * 2011-06-29 2013-01-03 The Johns Hopkins University Système pour interface tridimensionnelle et base de données
US11294547B2 (en) 2011-06-29 2022-04-05 The Johns Hopkins University Query-based three-dimensional atlas for accessing image-related data
US9691156B2 (en) 2012-02-01 2017-06-27 Koninklijke Philips N.V. Object image labeling apparatus, method and program
US11593691B2 (en) 2016-06-30 2023-02-28 Koninklijke Philips N.V. Information retrieval apparatus
CN109166183A (zh) * 2018-07-16 2019-01-08 中南大学 一种解剖标志点识别方法及识别设备
CN109166183B (zh) * 2018-07-16 2023-04-07 中南大学 一种解剖标志点识别方法及识别设备
CN113012781A (zh) * 2021-02-07 2021-06-22 重庆三峡医药高等专科学校 一种药理学数字人系统

Also Published As

Publication number Publication date
WO2011071363A3 (fr) 2011-11-10
MY148824A (en) 2013-06-14

Similar Documents

Publication Publication Date Title
US8860717B1 (en) Web browser for viewing a three-dimensional object responsive to a search query
US8843852B2 (en) Medical interface, annotation and communication systems
US7453472B2 (en) System and method for visual annotation and knowledge representation
Trelease et al. Transforming clinical imaging and 3D data for virtual reality learning objects: HTML5 and mobile devices implementation
Trelease et al. Transforming clinical imaging data for virtual reality learning objects
Halle et al. The open anatomy browser: a collaborative web-based viewer for interoperable anatomy atlases
US20080140706A1 (en) Image retrieval system
WO2011071363A2 (fr) Système et procédé permettant de visualiser et d'apprendre l'anatomie humaine
Borgbjerg Web‐based imaging viewer for real‐color volumetric reconstruction of human visible project and DICOM datasets
WO2012117103A2 (fr) Système et procédé d'indexation et d'interrogation de données à partir d'un modèle 3d
de Ridder et al. A web-based medical multimedia visualisation interface for personal health records
Marsh The Creation of a global telemedical information society
Marsh Chickens, aprons, markets, and cans: how the National Agricultural Library uses Omeka as its content management system for digital exhibits
Ma et al. Rembrain: Exploring dynamic biospatial networks with mosaic matrices and mirror glyphs
Müller et al. D9. 1: Report on image use behaviour and requirements
Liu et al. A non-contact interactive stereo display system for exploring human anatomy
Liepins et al. A browser based image bank, useful tool or expensive toy?
Dadzie et al. Providing visualisation support for the analysis of anatomy ontology data
Becker Visualizing time series data using web map service time dimension and SVG interactive animation
US20230334763A1 (en) Creating composite drawings using natural language understanding
Hazarika et al. DSpace information retrieval system: a study using DICOM metadata standard
Wan et al. Addressing the ‘Pathology Gap’in Clinical Education and Internship: The Impetus to Develop Digital (3D) Anatomic Pathology Learning Resources
Kenderdine et al. Cultural data sculpting: Omnispatial visualization for cultural datasets
Castanheira Representing and Indexing Archaeological Information
Dong et al. Visualization index for image-enabled medical records

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 10836254

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 10836254

Country of ref document: EP

Kind code of ref document: A2