WO2022011417A1 - Generating animated images - Google Patents

Generating animated images Download PDF

Info

Publication number
WO2022011417A1
WO2022011417A1 PCT/AU2021/050749 AU2021050749W WO2022011417A1 WO 2022011417 A1 WO2022011417 A1 WO 2022011417A1 AU 2021050749 W AU2021050749 W AU 2021050749W WO 2022011417 A1 WO2022011417 A1 WO 2022011417A1
Authority
WO
WIPO (PCT)
Prior art keywords
animated
heart
sequence
dimensional
dimensional models
Prior art date
Application number
PCT/AU2021/050749
Other languages
French (fr)
Inventor
Philip BRITTAIN
Original Assignee
Storm-Road Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2020902467A external-priority patent/AU2020902467A0/en
Application filed by Storm-Road Pty Ltd filed Critical Storm-Road Pty Ltd
Publication of WO2022011417A1 publication Critical patent/WO2022011417A1/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00Animation
    • G06T13/203D [Three Dimensional] animation
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/339Displays specially adapted therefor
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/346Analysis of electrocardiograms
    • A61B5/349Detecting specific parameters of the electrocardiograph cycle
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00Simulators for teaching or training purposes
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00Surgical instruments, devices or methods, e.g. tourniquets
    • A61B2017/00017Electrical control of surgical instruments
    • A61B2017/00022Sensing or detecting at the treatment site
    • A61B2017/00039Electric or electromagnetic phenomena other than conductivity, e.g. capacity, inductivity, Hall effect
    • A61B2017/00044Sensing electrocardiography, i.e. ECG
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0004Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by the type of physiological signal transmitted
    • A61B5/0006ECG or EEG signals
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/004Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
    • A61B5/0044Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part for the heart
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/346Analysis of electrocardiograms
    • A61B5/349Detecting specific parameters of the electrocardiograph cycle
    • A61B5/353Detecting P-waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/346Analysis of electrocardiograms
    • A61B5/349Detecting specific parameters of the electrocardiograph cycle
    • A61B5/355Detecting T-waves
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/346Analysis of electrocardiograms
    • A61B5/349Detecting specific parameters of the electrocardiograph cycle
    • A61B5/358Detecting ST segments
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • A61B5/346Analysis of electrocardiograms
    • A61B5/349Detecting specific parameters of the electrocardiograph cycle
    • A61B5/36Detecting PQ interval, PR interval or QT interval
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/74Details of notification to user or communication with user or patient ; user input means
    • A61B5/7475User input or interface means, e.g. keyboard, pointing device, joystick
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/04Texture mapping
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2210/00Indexing scheme for image generation or computer graphics
    • G06T2210/41Medical

Definitions

  • This disclosure relates to generating an animated three-dimensional image of a beating heart.
  • the heart is an essential organ for the survival of humans and animals. Accordingly, it is important that clinical practitioners have in-depth knowledge and understanding about this vital organ. While the anatomy of the heart is accessible through autopsy, it is difficult to access the dynamics of a beating heart. Therefore, electronic material, such as videos and images, can be provided. However, these forms of presentation are not interactive so that they do not enable exploration of the heart while it is beating.
  • One solution may be to provide an animation that is generated by simulating the heart in three dimensions.
  • the heart is a complex organ including electrical conduction pathways and muscle fibres that contract in a complex pattern. While a three-dimensional model of these elements may be constructed, for many modes of delivery, such a simulation is not practical as it is computationally too complex. For example, in an online delivery, the simulation would be performed by a browser application which would result in a significant time lag between starting the simulation and seeing the produced result.
  • a computer-implemented method for generating an animated three-dimensional image of a beating heart comprises: storing multiple pre-configured animated three-dimensional models of components of the heart; based on a sequence of events related to an electro-cardiogram (ECG), determining a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
  • ECG electro-cardiogram
  • the components may comprise one or more of atria, ventricles and electrical pathways.
  • the pre-configured animated three-dimensional models may comprise animated three-dimensional models of muscle contractions.
  • the pre-configured animated three-dimensional models may comprise animated three-dimensional models of electrical pathways.
  • the pre-configured animated three-dimensional models may comprise animated three-dimensional wireframe models.
  • the animated wireframe models may be based on nonuniform rational B splines.
  • Displaying the selection of the multiple pre-configured animated three- dimensional models of the heart may comprise rendering the animated three- dimensional image from the multiple pre-configured animated three-dimensional models of the heart.
  • the sequence of events may comprise a sequence of landmarks in an ECG waveform.
  • the landmarks may comprise one or more of P wave, QRS complex, T wave, and ST segment.
  • the sequence of events may comprise timing information of the landmarks in the ECG waveform.
  • the timing information may comprise one or more of: a PR interval and a QT interval.
  • the method may further comprise generating a user interface comprising user control element for controlling the animated image.
  • the user control elements may comprise user control elements for rotating the animated three-dimensional image in three dimensions and for playback of the animated image.
  • the method may be performed by a web server.
  • the method may be performed by a browser application of a web client.
  • a computer system for generating an animated three- dimensional image of a beating heart.
  • the computer system comprises: memory to store multiple pre-configured animated three-dimensional models of components of the heart; a processor configured to: based on a sequence of events related to an electro-cardiogram (ECG), determine a timing sequence of a selection of the multiple pre -configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
  • ECG electro-cardiogram
  • Fig. 1 illustrates a human heart.
  • Fig. 2 illustrates an electro cardiogram (ECG).
  • FIG. 3 illustrates a computer-implemented method for generating an animated three- dimensional image of a beating heart.
  • Fig. 4 illustrates a timing sequence related to the ECG in Fig. 2.
  • Fig. 5 illustrates a user interface for displaying an animated three-dimensional image of a heart.
  • Fig. 6 illustrates a computer system for generating an animated three- dimensional image of a beating heart. Description of Embodiments
  • Fig. 1 illustrates a human heart 100 and it is noted that the present disclosure equally applies to hearts of animals, such as domesticated animals including cattle, sheep and horses.
  • the heart 100 includes a series of volumes comprising an aorta 101, left atrium 102, left ventricle 103, right ventricle 104 and right atrium 105.
  • Heart 100 also includes a conduction system comprising an sinoatrial (SA) node (also ‘sinus’ node) 110, intermodal pathways 111, an atrioventricular (AV) node 112, a right bundle branch 113, a left bundle branch 114, and Purkinje fibres 115.
  • SA sinoatrial
  • AV atrioventricular
  • An electrical stimulus is generated by the SA node 110.
  • This is a small mass of specialized tissue located in the right upper chamber (atria) 105 of the heart 100.
  • the SA node 110 generates an electrical stimulus regularly, 60 to 100 times per minute under normal conditions.
  • the atria 102, 105 are then activated.
  • the electrical stimulus travels down through the conduction pathways 111 to AV node 112 and further along branches 113, 144 and causes the heart's ventricles to contract and pump out blood.
  • the two atria 102, 105 are stimulated first and contract for a short period of time before the two ventricles 103, 104.
  • the electrical impulse travels from the SA node 110 to the AV node 112. There, impulses are slowed down for a very short period, then continue down the conduction pathway via the bundles 113, 114 into the ventricles 103, 104.
  • the bundles 113, 114 divide into right and left pathways, called bundle branches, to stimulate the right and left ventricles. 103, 104, respectively.
  • Fig. 2 illustrates an electro cardiogram (ECG) 200 comprising the following sequence of ECG events: a P-wave 201, a PR interval 202, a QRS complex 203, an ST segment 204, a QT interval 205 and a T wave 206.
  • ECG 200 is typically a recording of the electrical activity of the heart.
  • the AV node 112 functions as a critical delay in the conduction system. Without this delay, the atria 102, 105 and ventricles 103, 104 would contract at the same time, and blood would not flow effectively from the atria to the ventricles.
  • the delay in the AV node 112 forms much of the PR segment 202 on the ECG 200, and part of atrial repolarization can be represented by the PR segment 202.
  • the distal portion of the AV node 112 is known as the bundle of His.
  • the bundle of His splits into two branches in the interventricular septum: the left bundle branch 114 and the right bundle branch 113.
  • the left bundle branch 114 activates the left ventricle 103, while the right bundle branch 113 activates the right ventricle 104.
  • the two bundle branches 113, 114 taper out to produce numerous Purkinje fibres 115, which stimulate individual groups of myocardial cells to contract.
  • the spread of electrical activity through the ventricular myocardium produces the QRS complex 203 on the ECG 200.
  • Atrial repolarization occurs and is masked during the QRS complex 203 by ventricular depolarization on the ECG.
  • the last event of the cycle is the repolarization of the ventricles 103, 104. It is the restoring of the resting state.
  • repolarization includes the ST segment 204, and T wave 206.
  • An impulse (action potential) that originates from the SA node 110 at a relative rate of 60-100bpm is a normal sinus rhythm.
  • SA nodal impulses occur at a rate less than 60bpm, the heart rhythm is referred to as sinus bradycardia. If SA nodal impulses occur at a rate exceeding lOObpm, the consequent rapid heart rate is sinus tachycardia. These conditions are not necessarily bad symptoms, however. Trained athletes, for example, usually show heart rates slower than 60bpm when not exercising.
  • the AV junction can take over as the main pacemaker of the heart.
  • the AV junction consists of the AV node, the bundle of His, and the surrounding area; it has a regular rate of 40 to 60bpm. These "junctional" rhythms are characterized by a missing or inverted P wave. If both the SA node and the AV junction fail to initialize the electrical impulse, the ventricles can fire the electrical impulses themselves at a rate of 20 to 40bpm and will have a QRS complex of greater than 120ms. This is necessary for the heart to be in good function.
  • Each event in the ECG 200 represents an activity in the heart 100 and the relative timing between the events in the sequence of ECG 200 defines the relative timing between activities in the heart 100. In that sense, ECG 200 defines a sequence of events including the events shown in Fig. 2.
  • Fig. 3 illustrates a computer-implemented method 300 for generating an animated three-dimensional image of a beating heart as performed by a computer processor.
  • An animated image in this context may comprise a sequence of images that are generated ‘on the fly’ as the user views a computer screen. Since computer screens are two-dimensional, the actual image is presented as a two-dimensional projection. However, reference is made herein to a three-dimensional (animated) image, which means that the two-dimensional representation or projection illustrates a three- dimensional object. So here, the three-dimensional object is the heart and the two- dimensional representation is the computer screen display. Therefore, the term “animated three-dimensional image” is used to highlight the three-dimensional nature of the object. More particularly, the user can manipulate the orientation of the three- dimensional object, which essentially changes the viewing direction and angle (the ‘view’). In response, the processor generates the images that represent that view. Therefore, it would not be practical to generate a single video or Graphics Interchange Format (GIF) image, as it would not be possible to change the view.
  • GIF Graphics Interchange Format
  • the animated three-dimensional image shows the heart while it is beating, which means the animated image illustrates the movement and shape change (contraction) of the different components of the heart as it would look like if an equivalent real-world heart would beat in a human body to pump blood through that body.
  • the computer stores 301 multiple pre-configured animated three-dimensional models of components of the heart. “Pre-configured” in this context means that the animation is configured at an earlier time and not ‘on the fly’ or in ‘real- time’. In other words, the relative movement of points of the model with respect to each other is set and stored before the method 300 is performed. [0041] For example, the computer stores an animated three-dimensional model of each of the components shown in Fig.
  • the animated three-dimensional models of the atria 102, 105 and ventricles 103, 104 may represent a single contraction of each of these components, while the animated three-dimensional models for the SA node 110, pathways 111, AV node 112, bundles 113, 114 and fibres 115 may represent the conduction of an electrical signal through these components during a single heartbeat.
  • the animated three-dimensional models are created using Maya (Alias Wavefront, Toronto, Canada) in the form of nonuniform rational B-spline (NURBS) curves.
  • NURBS curve is defined by its order, a set of weighted control points, and a knot vector, and these parameters can be set so that the curve matches the shape of the heart optimally.
  • These parameters can then be varied over time, such as by using a deformation tool, to create an animated model. This may be performed by hand by a human model designer, by matching fiduciary points to video recordings of a real-world heart, or by simulating the electrical conduction and muscle activation of the heart.
  • the resulting three-dimensional model is essentially an animated polygonial wireframe that can be rotated or zoomed in or out to change the view.
  • the processor can then use texture maps to render a realistic animated three-dimensional image.
  • the processor can retrieve the sequence of events related to electro cardiogram (ECG) 200. Based on that sequence, the processor determines 302 a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart. In other words, the processor determines at what time which animated three-dimensional model ought to be used for rendering the image to the user. For example, for P-wave 201 in Fig. 2, the processor selects an animated three- dimensional model of a contraction of the atria and an animate three-dimensional model of resting ventricles.
  • ECG electro cardiogram
  • Fig. 4 illustrates a timing sequence 400 related to ECG 200.
  • the sequence comprises a first time 401 tp, indicating the beginning of the P-wave and therefore contraction of the atria.
  • a second time 402 IQRS indicates the beginning of the QRS complex and therefore, the contraction of the ventricles.
  • a third time 403 ⁇ t indicates the beginning of the T-wave and therefore, the repolarisation of the ventricles. While only three times are shown in Fig. 4 for simplicity of illustration, there may be more times in the sequence.
  • the sequence may also comprise durations which may transform the timing of the pre -configured animated three-dimensional model. Again, for each time 401, 402, 403, the processor selects one of the pre-configured models for each component of the heart.
  • the processor displays 303 the selection of the multiple pre-configured animated three-dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
  • the processor joins the animated three-dimensional models and ‘plays’ them according to the timing sequence and according to the current view set by the user. This may involve real-time rendering or ray tracing of the components.
  • Displaying the animated models may be over a browser-based web- application.
  • the browser may execute a script, such as Java Script, that downloads the pre-configured animated three-dimensional models of the components and then renders the animated image based on the view selected by the user.
  • a script such as Java Script
  • it is particularly advantageous that the selection of pre-configured animated models enables the real-time generation of the three-dimensional model because a real-time simulation of the heart would be computationally difficult for the client device running the browser application.
  • additional sequences of events can be added for a variety of different heart conditions without major work on the animation model. Instead, only the sequence needs to be defined and the animation will be generated accurately.
  • Fig. 5 illustrates a user interface 500 as generated by the processor performing method 300 from Fig. 3.
  • user interface 500 may be displayed through a browser application running on a client machine, while a server provides the required Java Scripts and pre-configured models.
  • the method 300 in Fig. 3 may be performed by a web server, which then sends the resulting animated three-dimensional image to a web client.
  • the web server sends the coded software instructions representing method 300 to the client, such as in a Java Script object, and the browser application of the web client executes the Java Script to perform method 300.
  • User interface 500 comprises an ECG panel 501 showing the measurements across the 12 standard ECG leads.
  • User interface 500 further comprises an animation panel 502 where the animated three-dimensional image of the heart is displayed.
  • User interface 500 also comprises a control panel 503, which includes a play/pause control 504, a zoom control 505, a decelerate control 506 for reducing the display speed of the animation, an accelerate control 507 for increasing the display speed of the animation, a resent speed control 508 to return the display speed back to the natural heart rate.
  • Control panel 503 further comprises a display selection input 509 where the user can select which panels should be displayed. This way, the user can select, for example, to display only the animation panel 502 without the ECG panel 501.
  • the ECG panel 501 is also animated in the sense that, as the heart in the animation panel 502 beats, the ECG curve appears from left to right. In other words, each contraction may represent one animation and that animation may be repeated about every 1 s (depending on the heart rate).
  • the waveforms in the ECG panel 501 are not there yet and as the animation progresses through the 1 s animation interval, the 12 waveforms gradually appear at the same time from left to right. This means that after 0.5 s, all 12 ECG waveforms are half present.
  • Control panel 503 also includes a full screen control 510 for toggling full screen display on and off and a rotate control 511.
  • the rotate control 511 allows to rotate the animated three-dimensional image of the heart in the animation panel 502 to more clearly visualise the aspects that the user is interested in. For example, in some heart conditions, the operation of the atria is of particular importance and therefore, the user can bring the atria into view by selecting the rotation control 511. It is noted that the layout and details of user interface 500 are provided only for an exemplary illustration and a wide variety of different ways of implementing user interface 500 can be employed.
  • control button 512 for selecting different ECG rhythms, which relate to different heart conditions.
  • Activating control button 512 causes the processor to display a list of heart conditions (not shown).
  • the processor receives user input indicative of one of the heart condition and determines the sequence of events based on the user input.
  • the processor may store a list of heart conditions and for each heart condition, the processor stores the sequence of events, such as tp, tQRs, and ⁇ t, as explained with reference to Fig. 4. In this sense, the processor determines the sequence by selecting the sequence from a pre-configured list of heart conditions.
  • the heart conditions may include (among others):
  • FIG. 5 illustrates a clinical application example scenario 500 comprising a patient 501 under examination and a medical device 502, connected to ECG leads 503.
  • Device 502 records ECG signals measured via leads 503 and displays the ECG signals on ECG display 504.
  • Device 502 may determine the sequence of events from the recorded ECG signals. For example, device 502 may determine the time values tp, tQRS, and tr for the events of the ECG.
  • the sequence of events comprises a sequence of landmarks in an ECG waveform, such as the P-wave, QRS complex, T-wave and ST- period, and device 502 determines this sequence with corresponding timing information.
  • the timing information comprises the PR interval and the QT interval. Based on this determined sequence, device 502 can then display the sequence of accordingly selected pre-configured animated three-dimensional models to generate the animated three-dimensional image of the heart as described below.
  • Device 502 may be useful for emergency personal, such as paramedics, or other clinicians.
  • clinicians with limited experience in reading the ECG signals themselves may gain a quick understanding of the patients current condition by observing the three-dimensional animation.
  • device 502 may provide the user control elements shown in Fig. 5 to enable the clinician to more carefully explore the beating heart and potentially identify any relevant information for treatment.
  • Fig. 6 illustrates a computer system 600 for generating an animated three- dimensional image of a beating heart.
  • the computer system 600 comprises a processor 601 connected to a program memory 602, a data memory 603, a communication port 604 and a user port 605.
  • the program memory 602 is a non-transitory computer readable medium, such as a hard drive, a solid state disk or CD-ROM.
  • Software, that is, an executable program stored on program memory 602 causes the processor 601 to perform the method in Fig. 3, that is, animated three-dimensional component models are stored on data memory 603, processor 601 determines a timing sequence and then displays the models according to the timing sequence.
  • the term “determining a timing sequence” refers to calculating one or more values that are indicative of the timing sequence. This also applies to related terms.
  • the processor 601 may then store the timing sequence on data store 603, such as on RAM or a processor register. Processor 601 may also send the determined timing sequence to another computer system, such as medical record system.
  • the processor 601 may receive data, such as ECG recordings , from data memory 603 as well as from the communications port 604 and the user port 605, which is connected to a display 606 that shows the animated image to a user.
  • the processor 601 receives the ECG recording from patient 501 via communications port 604 wirelessly, such as by using a Wi-Fi or Bluetooth network according to IEEE 802.11.
  • the Wi-Fi or Bluetooth network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network.
  • the processor 601 receives and processes the ECG data in real time. This means that the processor 601 determines the timing sequence every time a full heartbeat of ECG data is received from leads 503 and completes this calculation before the next ECG data update.
  • communications port 604 and user port 605 are shown as distinct entities, it is to be understood that any kind of data port may be used to receive data, such as a network connection, a memory interface, a pin of the chip package of processor 601, or logical ports, such as IP sockets or parameters of functions stored on program memory 602 and executed by processor 601. These parameters may be stored on data memory 603 and may be handled by-value or by-reference, that is, as a pointer, in the source code.
  • the processor 601 may receive data through all these interfaces, which includes memory access of volatile memory, such as cache or RAM, or non-volatile memory, such as an optical disk drive, hard disk drive, storage server or cloud storage.
  • volatile memory such as cache or RAM
  • non-volatile memory such as an optical disk drive, hard disk drive, storage server or cloud storage.
  • the computer system 600 may further be implemented within a cloud computing environment, such as a managed group of interconnected servers hosting a dynamic number of virtual machines.
  • any receiving step may be preceded by the processor 601 determining or computing the data that is later received.
  • the processor 601 determines a ECG recording and stores the ECG recording in data memory 603, such as RAM or a processor register.
  • the processor 601 requests the recording from the data memory 603, such as by providing a read signal together with a memory address.
  • the data memory 603 provides the data as a voltage signal on a physical bit line and the processor 601 receives the recording via a memory interface.
  • variables, parameters, sequences, timing information and the like refer to data structures, which are physically stored on data memory 603 or processed by processor 601. Further, for the sake of brevity when reference is made to particular variable names, such as “period of time” or “sequence” this is to be understood to refer to values of variables stored as physical data in computer system 600.
  • Fig. 3 is to be understood as a blueprint for the software program and may be implemented step-by-step, such that each step in Fig. 3 is represented by a function in a programming language, such as C++ or Java.
  • the resulting source code is then compiled and stored as computer executable instructions on program memory 602.

Abstract

This disclosure relates to generating an animated three-dimensional image of a beating heart. A computer stores multiple pre-configured animated three-dimensional models of components of the heart. Based on a sequence of events related to an electro-cardiogram, the computer determines a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart and displays the selection of the multiple pre-configured animated three-dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart. The selection of pre-configured animated models enables the real-time generation of the three-dimensional model. Otherwise, real-time simulation of the heart would be computationally difficult if attempted on a client device. Further, additional sequences of events can be added for a variety of different heart conditions without major work on the animation model.

Description

"Generating animated images"
Technical Field
[0001] This disclosure relates to generating an animated three-dimensional image of a beating heart.
Background
[0002] The heart is an essential organ for the survival of humans and animals. Accordingly, it is important that clinical practitioners have in-depth knowledge and understanding about this vital organ. While the anatomy of the heart is accessible through autopsy, it is difficult to access the dynamics of a beating heart. Therefore, electronic material, such as videos and images, can be provided. However, these forms of presentation are not interactive so that they do not enable exploration of the heart while it is beating.
[0003] One solution may be to provide an animation that is generated by simulating the heart in three dimensions. However, the heart is a complex organ including electrical conduction pathways and muscle fibres that contract in a complex pattern. While a three-dimensional model of these elements may be constructed, for many modes of delivery, such a simulation is not practical as it is computationally too complex. For example, in an online delivery, the simulation would be performed by a browser application which would result in a significant time lag between starting the simulation and seeing the produced result.
[0004] Any discussion of documents, acts, materials, devices, articles or the like which has been included in the present specification is not to be taken as an admission that any or all of these matters form part of the prior art base or were common general knowledge in the field relevant to the present disclosure as it existed before the priority date of each of the appended claims. [0005] Throughout this specification the word "comprise", or variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated element, integer or step, or group of elements, integers or steps, but not the exclusion of any other element, integer or step, or group of elements, integers or steps.
Summary
[0006] There is provided a computer-implemented method for generating an animated three-dimensional image of a beating heart. The method comprises: storing multiple pre-configured animated three-dimensional models of components of the heart; based on a sequence of events related to an electro-cardiogram (ECG), determining a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
[0007] It is advantage that the selection of pre-configured animated models enables the real-time generation of the three-dimensional model. This is particularly advantageous where the animated image is to be presented over a browser-based web- application. In those cases, a real-time simulation of the heart would be computationally difficult for the client device. It is a further advantage, that additional sequences of events can be added for a variety of different heart conditions without major work on the animation model. Instead, only the sequence needs to be defined and the animation will be generated accurately.
[0008] The components may comprise one or more of atria, ventricles and electrical pathways.
[0009] The pre-configured animated three-dimensional models may comprise animated three-dimensional models of muscle contractions. The pre-configured animated three-dimensional models may comprise animated three-dimensional models of electrical pathways. The pre-configured animated three-dimensional models may comprise animated three-dimensional wireframe models. The animated wireframe models may be based on nonuniform rational B splines.
[0010] Displaying the selection of the multiple pre-configured animated three- dimensional models of the heart may comprise rendering the animated three- dimensional image from the multiple pre-configured animated three-dimensional models of the heart.
[0011] The method may further comprise receiving user input indicative of a heart condition and determining the sequence of events based on the user input. Determining the sequence may comprise selecting the sequence from a pre-configured list of heart conditions. The method may further comprise determining the sequence of events from a recorded ECG signal.
[0012] The sequence of events may comprise a sequence of landmarks in an ECG waveform. The landmarks may comprise one or more of P wave, QRS complex, T wave, and ST segment. The sequence of events may comprise timing information of the landmarks in the ECG waveform. The timing information may comprise one or more of: a PR interval and a QT interval.
[0013] The method may further comprise generating a user interface comprising user control element for controlling the animated image. The user control elements may comprise user control elements for rotating the animated three-dimensional image in three dimensions and for playback of the animated image.
[0014] The method may be performed by a web server. The method may be performed by a browser application of a web client.
[0015] There is further provided a computer system for generating an animated three- dimensional image of a beating heart. The computer system comprises: memory to store multiple pre-configured animated three-dimensional models of components of the heart; a processor configured to: based on a sequence of events related to an electro-cardiogram (ECG), determine a timing sequence of a selection of the multiple pre -configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
[0016] Optional features described of any aspect of method, computer readable medium or computer system, where appropriate, similarly apply to the other aspects also described here.
Brief Description of Drawings
[0017] An example will be described with reference to the following drawings:
[0018] Fig. 1 illustrates a human heart.
[0019] Fig. 2 illustrates an electro cardiogram (ECG).
[0020] Fig. 3 illustrates a computer-implemented method for generating an animated three- dimensional image of a beating heart.
[0021] Fig. 4 illustrates a timing sequence related to the ECG in Fig. 2.
[0022] Fig. 5 illustrates a user interface for displaying an animated three-dimensional image of a heart.
[0023] Fig. 6 illustrates a computer system for generating an animated three- dimensional image of a beating heart. Description of Embodiments
[0024] Fig. 1 illustrates a human heart 100 and it is noted that the present disclosure equally applies to hearts of animals, such as domesticated animals including cattle, sheep and horses. The heart 100 includes a series of volumes comprising an aorta 101, left atrium 102, left ventricle 103, right ventricle 104 and right atrium 105. Heart 100 also includes a conduction system comprising an sinoatrial (SA) node (also ‘sinus’ node) 110, intermodal pathways 111, an atrioventricular (AV) node 112, a right bundle branch 113, a left bundle branch 114, and Purkinje fibres 115.
[0025] An electrical stimulus is generated by the SA node 110. This is a small mass of specialized tissue located in the right upper chamber (atria) 105 of the heart 100. The SA node 110 generates an electrical stimulus regularly, 60 to 100 times per minute under normal conditions. The atria 102, 105 are then activated. The electrical stimulus travels down through the conduction pathways 111 to AV node 112 and further along branches 113, 144 and causes the heart's ventricles to contract and pump out blood. The two atria 102, 105 are stimulated first and contract for a short period of time before the two ventricles 103, 104.
[0026] More particularly, the electrical impulse travels from the SA node 110 to the AV node 112. There, impulses are slowed down for a very short period, then continue down the conduction pathway via the bundles 113, 114 into the ventricles 103, 104.
The bundles 113, 114 divide into right and left pathways, called bundle branches, to stimulate the right and left ventricles. 103, 104, respectively.
[0027] Normally at rest, as the electrical impulse moves through the heart, the heart contracts about 60 to 100 times a minute, depending on a person's age. Each contraction of the ventricles represents one heartbeat. The atria contract a fraction of a second before the ventricles so their blood empties into the ventricles before the ventricles contract. [0028] Fig. 2 illustrates an electro cardiogram (ECG) 200 comprising the following sequence of ECG events: a P-wave 201, a PR interval 202, a QRS complex 203, an ST segment 204, a QT interval 205 and a T wave 206. ECG 200 is typically a recording of the electrical activity of the heart.
[0029] Under normal conditions, electrical activity is spontaneously generated by the SA node 110, the cardiac pacemaker. This electrical impulse is propagated throughout the right atrium 105, and to the left atrium 102, stimulating the myocardium of the atria to contract. The conduction of the electrical impulses throughout the atria is seen on the ECG as the P wave 201.
[0030] As the electrical activity is spreading throughout the atria 102, 105, it travels via pathways 111, from the SA node 110 to the AV node 112.
[0031] The AV node 112 functions as a critical delay in the conduction system. Without this delay, the atria 102, 105 and ventricles 103, 104 would contract at the same time, and blood would not flow effectively from the atria to the ventricles. The delay in the AV node 112 forms much of the PR segment 202 on the ECG 200, and part of atrial repolarization can be represented by the PR segment 202.
[0032] The distal portion of the AV node 112 is known as the bundle of His. The bundle of His splits into two branches in the interventricular septum: the left bundle branch 114 and the right bundle branch 113. The left bundle branch 114 activates the left ventricle 103, while the right bundle branch 113 activates the right ventricle 104.
[0033] The two bundle branches 113, 114 taper out to produce numerous Purkinje fibres 115, which stimulate individual groups of myocardial cells to contract. The spread of electrical activity through the ventricular myocardium produces the QRS complex 203 on the ECG 200. Atrial repolarization occurs and is masked during the QRS complex 203 by ventricular depolarization on the ECG. The last event of the cycle is the repolarization of the ventricles 103, 104. It is the restoring of the resting state. In the ECG 200, repolarization includes the ST segment 204, and T wave 206. [0034] An impulse (action potential) that originates from the SA node 110 at a relative rate of 60-100bpm is a normal sinus rhythm. If SA nodal impulses occur at a rate less than 60bpm, the heart rhythm is referred to as sinus bradycardia. If SA nodal impulses occur at a rate exceeding lOObpm, the consequent rapid heart rate is sinus tachycardia. These conditions are not necessarily bad symptoms, however. Trained athletes, for example, usually show heart rates slower than 60bpm when not exercising.
[0035] If the SA node fails to initialize, the AV junction can take over as the main pacemaker of the heart. The AV junction consists of the AV node, the bundle of His, and the surrounding area; it has a regular rate of 40 to 60bpm. These "junctional" rhythms are characterized by a missing or inverted P wave. If both the SA node and the AV junction fail to initialize the electrical impulse, the ventricles can fire the electrical impulses themselves at a rate of 20 to 40bpm and will have a QRS complex of greater than 120ms. This is necessary for the heart to be in good function.
[0036] Each event in the ECG 200 represents an activity in the heart 100 and the relative timing between the events in the sequence of ECG 200 defines the relative timing between activities in the heart 100. In that sense, ECG 200 defines a sequence of events including the events shown in Fig. 2.
[0037] The following description sets out how to create multiple animated three- dimensional models of components of the heart and combine them to represent the sequence of events defined by ECG 200. This way, it is possible to define multiple different heart conditions by way of their different sequence of events with associated timing. These multiple different sequences can then be used to combine the animated models into a single animation that simulates the actual beating heart in three dimensions. This means it is not necessary to create an individual animated version for each condition. It is also possible to provide a user interface to enable a user to select heart conditions, rotate the animated image and stop, start, accelerate, decelerate the animation. [0038] Fig. 3 illustrates a computer-implemented method 300 for generating an animated three-dimensional image of a beating heart as performed by a computer processor. An animated image in this context may comprise a sequence of images that are generated ‘on the fly’ as the user views a computer screen. Since computer screens are two-dimensional, the actual image is presented as a two-dimensional projection. However, reference is made herein to a three-dimensional (animated) image, which means that the two-dimensional representation or projection illustrates a three- dimensional object. So here, the three-dimensional object is the heart and the two- dimensional representation is the computer screen display. Therefore, the term “animated three-dimensional image” is used to highlight the three-dimensional nature of the object. More particularly, the user can manipulate the orientation of the three- dimensional object, which essentially changes the viewing direction and angle (the ‘view’). In response, the processor generates the images that represent that view. Therefore, it would not be practical to generate a single video or Graphics Interchange Format (GIF) image, as it would not be possible to change the view.
[0039] Here, the animated three-dimensional image shows the heart while it is beating, which means the animated image illustrates the movement and shape change (contraction) of the different components of the heart as it would look like if an equivalent real-world heart would beat in a human body to pump blood through that body.
[0040] As mentioned above, it is not practical to create video fdes for a particular view. Further, it is computationally too complex to simulate the entire heart or render the heart at a frame rate that is sufficient for a smooth animation, such as at least 10 frames per second. Instead, the computer stores 301 multiple pre-configured animated three-dimensional models of components of the heart. “Pre-configured” in this context means that the animation is configured at an earlier time and not ‘on the fly’ or in ‘real- time’. In other words, the relative movement of points of the model with respect to each other is set and stored before the method 300 is performed. [0041] For example, the computer stores an animated three-dimensional model of each of the components shown in Fig. 1 comprising aorta 101, atria 102, 105, ventricles 103, 104, SA node 110, pathways 111, AV node 112, bundles 113, 114 and fibres 115. The animated three-dimensional models of the atria 102, 105 and ventricles 103, 104 may represent a single contraction of each of these components, while the animated three-dimensional models for the SA node 110, pathways 111, AV node 112, bundles 113, 114 and fibres 115 may represent the conduction of an electrical signal through these components during a single heartbeat.
[0042] In one example, the animated three-dimensional models are created using Maya (Alias Wavefront, Toronto, Canada) in the form of nonuniform rational B-spline (NURBS) curves. A NURBS curve is defined by its order, a set of weighted control points, and a knot vector, and these parameters can be set so that the curve matches the shape of the heart optimally. These parameters can then be varied over time, such as by using a deformation tool, to create an animated model. This may be performed by hand by a human model designer, by matching fiduciary points to video recordings of a real-world heart, or by simulating the electrical conduction and muscle activation of the heart. The resulting three-dimensional model is essentially an animated polygonial wireframe that can be rotated or zoomed in or out to change the view. The processor can then use texture maps to render a realistic animated three-dimensional image.
[0043] While it may be possible to create a single animated three-dimensional model of the beating heart, it would not be practical to use that for multiple different heart conditions, since a different model would need to be created for each condition. This is one reason why here, each component has its own animated three-dimensional model for different activities, such as rest or contraction.
[0044] Then, the processor can retrieve the sequence of events related to electro cardiogram (ECG) 200. Based on that sequence, the processor determines 302 a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart. In other words, the processor determines at what time which animated three-dimensional model ought to be used for rendering the image to the user. For example, for P-wave 201 in Fig. 2, the processor selects an animated three- dimensional model of a contraction of the atria and an animate three-dimensional model of resting ventricles.
[0045] Fig. 4 illustrates a timing sequence 400 related to ECG 200. The sequence comprises a first time 401 tp, indicating the beginning of the P-wave and therefore contraction of the atria. A second time 402 IQRS, indicates the beginning of the QRS complex and therefore, the contraction of the ventricles. A third time 403 ΐt, indicates the beginning of the T-wave and therefore, the repolarisation of the ventricles. While only three times are shown in Fig. 4 for simplicity of illustration, there may be more times in the sequence. Further, the sequence may also comprise durations which may transform the timing of the pre -configured animated three-dimensional model. Again, for each time 401, 402, 403, the processor selects one of the pre-configured models for each component of the heart.
[0046] While the examples above mention muscle groups and contraction thereof, it is noted that other components of the heart are of electrical nature. Instead of contracting, these components conduct electrical signals and the corresponding animated three-dimensional models may illustrate this conduction by a light effect that travels along those components. For example, the current location of the electrical signal may be indicated by a brighter colour or an integrated light source or general illuminant at that location. During the animation, this location moves along to indicate the transmission of the signal.
[0047] Finally, the processor displays 303 the selection of the multiple pre-configured animated three-dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart. This is, the processor joins the animated three-dimensional models and ‘plays’ them according to the timing sequence and according to the current view set by the user. This may involve real-time rendering or ray tracing of the components. There may also be a connection between the NURBS of one component with the NURBS of another component, so that, for example, a relaxed ventricle shows a slight movement when the atria contract but this movement is not significant from a functional or clinical working of the heart and purely for the purpose of a realistic visualisation.
[0048] Displaying the animated models may be over a browser-based web- application. In that instance, the browser may execute a script, such as Java Script, that downloads the pre-configured animated three-dimensional models of the components and then renders the animated image based on the view selected by the user. In such an environment, it is particularly advantageous that the selection of pre-configured animated models enables the real-time generation of the three-dimensional model because a real-time simulation of the heart would be computationally difficult for the client device running the browser application. It is a further advantage, that additional sequences of events can be added for a variety of different heart conditions without major work on the animation model. Instead, only the sequence needs to be defined and the animation will be generated accurately.
[0049] Fig. 5 illustrates a user interface 500 as generated by the processor performing method 300 from Fig. 3. As set out above, user interface 500 may be displayed through a browser application running on a client machine, while a server provides the required Java Scripts and pre-configured models. In that sense, the method 300 in Fig. 3 may be performed by a web server, which then sends the resulting animated three-dimensional image to a web client. In another example, the web server sends the coded software instructions representing method 300 to the client, such as in a Java Script object, and the browser application of the web client executes the Java Script to perform method 300.
[0050] User interface 500 comprises an ECG panel 501 showing the measurements across the 12 standard ECG leads. User interface 500 further comprises an animation panel 502 where the animated three-dimensional image of the heart is displayed. User interface 500 also comprises a control panel 503, which includes a play/pause control 504, a zoom control 505, a decelerate control 506 for reducing the display speed of the animation, an accelerate control 507 for increasing the display speed of the animation, a resent speed control 508 to return the display speed back to the natural heart rate. Control panel 503 further comprises a display selection input 509 where the user can select which panels should be displayed. This way, the user can select, for example, to display only the animation panel 502 without the ECG panel 501.
[0051] In one example, the ECG panel 501 is also animated in the sense that, as the heart in the animation panel 502 beats, the ECG curve appears from left to right. In other words, each contraction may represent one animation and that animation may be repeated about every 1 s (depending on the heart rate). At the start of the 1 s animation, the waveforms in the ECG panel 501 are not there yet and as the animation progresses through the 1 s animation interval, the 12 waveforms gradually appear at the same time from left to right. This means that after 0.5 s, all 12 ECG waveforms are half present.
So as the QRS complex appears in all 12 waveforms, at the same time, the ventricles of the heart in the animation panel 502 contract. Of course, the above timings are examples only and would change in real time based on the user accelerating or decelerating the animation.
[0052] The user may also choose to display a text panel (not shown) which provides textual explanations of the currently selected heart condition for education purposes, for example. Control panel 503 also includes a full screen control 510 for toggling full screen display on and off and a rotate control 511. The rotate control 511 allows to rotate the animated three-dimensional image of the heart in the animation panel 502 to more clearly visualise the aspects that the user is interested in. For example, in some heart conditions, the operation of the atria is of particular importance and therefore, the user can bring the atria into view by selecting the rotation control 511. It is noted that the layout and details of user interface 500 are provided only for an exemplary illustration and a wide variety of different ways of implementing user interface 500 can be employed.
[0053] Finally, there is a control button 512 for selecting different ECG rhythms, which relate to different heart conditions. Activating control button 512 causes the processor to display a list of heart conditions (not shown). When the user selects one of the heart conditions for display, the processor then receives user input indicative of one of the heart condition and determines the sequence of events based on the user input. For example, the processor may store a list of heart conditions and for each heart condition, the processor stores the sequence of events, such as tp, tQRs, and ΐt, as explained with reference to Fig. 4. In this sense, the processor determines the sequence by selecting the sequence from a pre-configured list of heart conditions.
[0054] The heart conditions may include (among others):
• Normal Sinus Rhythm
• Sinus Arrhythmia
• Sinus Bradycardia
• Sinus Tachycardia
• Sinus Arrest
• Premature Atrial Contraction
• Atrial Fibrillation
• Atrial Flutter
• Supraventricular Tachycardia
• Paroxysmal Supraventricular Tachycardia
• Wolff Parkinson White
• Brugada Pattern Type 1
• Brugada Pattern Type 2
• Brugada Pattern Type 3
• Junctional Rhythm
• Premature Junctional Complex
• Junctional Bradycardia
• Accelerated Junctional Rhythm
• Junctional Tachycardia
• First Degree AV Block
• Second Degree AV Block Type 1 Mobitz type 1
• Second Degree AV Block Mobitz Type 2
• Third Degree - Complete Heart Block
• Right Bundle Branch Block • Left Bundle Branch Block
• Left Anterior Fascicular Block
• Left Posterior Fascicular Block
• Premature Ventricular Contraction
• Ventricular Bigeminy
• Ventricular Trigeminy
• Ventricular Tachycardia
• Ventricular Fibrillation
• Idioventricular Rhythm
• Accelerated Idioventricular Rhythm
• Agonal Rhythm
• Asystole
[0055] Fig. 5 illustrates a clinical application example scenario 500 comprising a patient 501 under examination and a medical device 502, connected to ECG leads 503. Device 502 records ECG signals measured via leads 503 and displays the ECG signals on ECG display 504.
[0056] Device 502 may determine the sequence of events from the recorded ECG signals. For example, device 502 may determine the time values tp, tQRS, and tr for the events of the ECG. In other words, the sequence of events comprises a sequence of landmarks in an ECG waveform, such as the P-wave, QRS complex, T-wave and ST- period, and device 502 determines this sequence with corresponding timing information. In one example, the timing information comprises the PR interval and the QT interval. Based on this determined sequence, device 502 can then display the sequence of accordingly selected pre-configured animated three-dimensional models to generate the animated three-dimensional image of the heart as described below.
[0057] Device 502 may be useful for emergency personal, such as paramedics, or other clinicians. In particular, clinicians with limited experience in reading the ECG signals themselves may gain a quick understanding of the patients current condition by observing the three-dimensional animation. Further, device 502 may provide the user control elements shown in Fig. 5 to enable the clinician to more carefully explore the beating heart and potentially identify any relevant information for treatment.
[0058] Fig. 6 illustrates a computer system 600 for generating an animated three- dimensional image of a beating heart. The computer system 600 comprises a processor 601 connected to a program memory 602, a data memory 603, a communication port 604 and a user port 605. The program memory 602 is a non-transitory computer readable medium, such as a hard drive, a solid state disk or CD-ROM. Software, that is, an executable program stored on program memory 602 causes the processor 601 to perform the method in Fig. 3, that is, animated three-dimensional component models are stored on data memory 603, processor 601 determines a timing sequence and then displays the models according to the timing sequence. The term “determining a timing sequence” refers to calculating one or more values that are indicative of the timing sequence. This also applies to related terms.
[0059] The processor 601 may then store the timing sequence on data store 603, such as on RAM or a processor register. Processor 601 may also send the determined timing sequence to another computer system, such as medical record system.
[0060] The processor 601 may receive data, such as ECG recordings , from data memory 603 as well as from the communications port 604 and the user port 605, which is connected to a display 606 that shows the animated image to a user. In one example, the processor 601 receives the ECG recording from patient 501 via communications port 604 wirelessly, such as by using a Wi-Fi or Bluetooth network according to IEEE 802.11. The Wi-Fi or Bluetooth network may be a decentralised ad-hoc network, such that no dedicated management infrastructure, such as a router, is required or a centralised network with a router or access point managing the network.
[0061] In one example, the processor 601 receives and processes the ECG data in real time. This means that the processor 601 determines the timing sequence every time a full heartbeat of ECG data is received from leads 503 and completes this calculation before the next ECG data update.
[0062] Although communications port 604 and user port 605 are shown as distinct entities, it is to be understood that any kind of data port may be used to receive data, such as a network connection, a memory interface, a pin of the chip package of processor 601, or logical ports, such as IP sockets or parameters of functions stored on program memory 602 and executed by processor 601. These parameters may be stored on data memory 603 and may be handled by-value or by-reference, that is, as a pointer, in the source code.
[0063] The processor 601 may receive data through all these interfaces, which includes memory access of volatile memory, such as cache or RAM, or non-volatile memory, such as an optical disk drive, hard disk drive, storage server or cloud storage. The computer system 600 may further be implemented within a cloud computing environment, such as a managed group of interconnected servers hosting a dynamic number of virtual machines.
[0064] It is to be understood that any receiving step may be preceded by the processor 601 determining or computing the data that is later received. For example, the processor 601 determines a ECG recording and stores the ECG recording in data memory 603, such as RAM or a processor register. The processor 601 then requests the recording from the data memory 603, such as by providing a read signal together with a memory address. The data memory 603 provides the data as a voltage signal on a physical bit line and the processor 601 receives the recording via a memory interface. In some examples, however, there may not be any recording since the processor 601 retrieve the sequence and timing information based on user input regarding a selection of a heart condition. In that case, communication port 604 and leads 503 may not be required.
[0065] It is to be understood that throughout this disclosure unless stated otherwise, variables, parameters, sequences, timing information and the like refer to data structures, which are physically stored on data memory 603 or processed by processor 601. Further, for the sake of brevity when reference is made to particular variable names, such as “period of time” or “sequence” this is to be understood to refer to values of variables stored as physical data in computer system 600.
[0066] Further, Fig. 3 is to be understood as a blueprint for the software program and may be implemented step-by-step, such that each step in Fig. 3 is represented by a function in a programming language, such as C++ or Java. The resulting source code is then compiled and stored as computer executable instructions on program memory 602.
[0067] It will be appreciated by persons skilled in the art that numerous variations and/or modifications may be made to the above-described embodiments, without departing from the broad general scope of the present disclosure. The present embodiments are, therefore, to be considered in all respects as illustrative and not restrictive.

Claims

CLAIMS:
1. A computer-implemented method for generating an animated three- dimensional image of a beating heart, the method comprising: storing multiple pre-configured animated three-dimensional models of components of the heart; based on a sequence of events related to an electro-cardiogram (ECG), determining a timing sequence of a selection of the multiple pre-configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
2. The method of claim 1, wherein the components comprise one or more of atria, ventricles and electrical pathways.
3. The method of claim 1 or 2, wherein the pre-configured animated three- dimensional models comprise animated three-dimensional models of muscle contractions.
4. The method of any one of the preceding claims, wherein the pre-configured animated three-dimensional models comprise animated three-dimensional models of electrical pathways.
5. The method of any one of the preceding claims, wherein the pre-configured animated three-dimensional models comprise animated three-dimensional wireframe models.
6. The method of claim 5, wherein the animated wireframe models are based on nonuniform rational B splines.
7. The method of any one of the preceding claims, wherein displaying the selection of the multiple pre -configured animated three-dimensional models of the heart comprises rendering the animated three-dimensional image from the multiple pre- configured animated three-dimensional models of the heart.
8. The method of any one of the preceding claims, wherein the method further comprises receiving user input indicative of a heart condition and determining the sequence of events based on the user input.
9. The method of claim 8, wherein determining the sequence comprises selecting the sequence from a pre-configured list of heart conditions.
10. The method of claim 8 or 9, wherein the method furher comprises determining the sequence of events from a recorded ECG signal.
11. The method of any one of the preceding claims, wherein the sequence of events comprises a sequence of landmarks in an ECG waveform.
12. The method of claim 11, wherein the landmarks comprise one or more of P wave, QRS complex, T wave, and ST segment.
13. The method of claim 11 or 12, wherein the sequence of events comprises timing information of the landmarks in the ECG waveform.
14. The method of claim 13, wherein the timing information comprises one or more of: a PR interval and a QT interval.
15. The method of any one of the preceding claims, wherein the method further comprises generating a user interface comprising user control element for controlling the animated image.
16. The method of claim 15, wherein the user control elements comprise user control elements for rotating the animated three-dimensional image in three dimensions and for playback of the animated image.
17. The method of any one of the preceding claims, wherein the method is performed by a web server.
18. The method of any one of claims 1 to 16, wherein the method is performed by a browser application of a web client.
19. A computer system for generating an animated three-dimensional image of a beating heart, the computer system comprising: memory to store multiple pre-configured animated three-dimensional models of components of the heart; a processor configured to: based on a sequence of events related to an electro-cardiogram (ECG), determine a timing sequence of a selection of the multiple pre -configured animated three-dimensional models of the heart; and displaying the selection of the multiple pre-configured animated three- dimensional models of the heart according to the timing sequence to generate the animated three-dimensional image of the heart.
PCT/AU2021/050749 2020-07-16 2021-07-14 Generating animated images WO2022011417A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2020902467 2020-07-16
AU2020902467A AU2020902467A0 (en) 2020-07-16 Generating animated images

Publications (1)

Publication Number Publication Date
WO2022011417A1 true WO2022011417A1 (en) 2022-01-20

Family

ID=79555891

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2021/050749 WO2022011417A1 (en) 2020-07-16 2021-07-14 Generating animated images

Country Status (1)

Country Link
WO (1) WO2022011417A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020016551A1 (en) * 1999-09-15 2002-02-07 Selvester Ronald H. Pictorial-display electrocardiographic interpretation system and method
US6694178B1 (en) * 1998-01-12 2004-02-17 Energy-Lab Technologies Gmbh Method and device for representing and monitoring functional parameters of a physiological system
WO2006126020A2 (en) * 2005-05-24 2006-11-30 Anglia Polytechnic University Higher Education Corporation Medical data signal processing systems
US20120283587A1 (en) * 2011-05-03 2012-11-08 Medtronic, Inc. Assessing intra-cardiac activation patterns and electrical dyssynchrony

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6694178B1 (en) * 1998-01-12 2004-02-17 Energy-Lab Technologies Gmbh Method and device for representing and monitoring functional parameters of a physiological system
US20020016551A1 (en) * 1999-09-15 2002-02-07 Selvester Ronald H. Pictorial-display electrocardiographic interpretation system and method
WO2006126020A2 (en) * 2005-05-24 2006-11-30 Anglia Polytechnic University Higher Education Corporation Medical data signal processing systems
US20120283587A1 (en) * 2011-05-03 2012-11-08 Medtronic, Inc. Assessing intra-cardiac activation patterns and electrical dyssynchrony

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
B. H. BRANHAM ET AL.: "A system for accurate interactive 3-D display of cardiac electrical activity", PROCEEDINGS COMPUTERS IN CARDIOLOGY, 1992, pages 335 - 338, XP010032071, DOI: 10.1109/CIC.1992.269363 *

Similar Documents

Publication Publication Date Title
US11344263B2 (en) Bootstrapping a simulation-based electromagnetic output of a different anatomy
CN106456269B (en) To the system and method for the real-time simulation of the specific cardiac electrophysiology of patient including the influence of the electrical conduction system of heart
US11806080B2 (en) Identify ablation pattern for use in an ablation
US10860754B2 (en) Calibration of simulated cardiograms
Ryzhii et al. A heterogeneous coupled oscillator model for simulation of ECG signals
TWI728832B (en) Methods and computing systems for presenting/providing information to assist evaluation of electromagnetic source
KR20210076165A (en) Magnification of image with root position
US11065060B2 (en) Identify ablation pattern for use in an ablation
WO2015200552A1 (en) Simulated cardiac model
WO2022011417A1 (en) Generating animated images
US20230049769A1 (en) Cardiogram collection and source location identification
JPWO2019206908A5 (en)
Gharpure A cellular automaton model of electrical wave propagation in cardiac muscle
Iravanian et al. A network-based cardiac electrophysiology simulator with realistic signal generation and response to pacing maneuvers
Albors Lucas Prediction of electrical activation patterns in porcine hearts with a meshless model Smoothed Particle Hydrodynamics: an silico study
Stárková Visualization of Myocardial Infarction in Android
Seemann et al. Simulation of surgical interventions: Atrial radio frequency ablation with a haptic interface
Singh et al. Formal Logic Based Heart-Model
WO2016124899A1 (en) Method and system for subject specific electrophysiological simulation of excitable tissue
Al Abed Electrical activity of atrial tissue: an experimental and computational study
Moin et al. PACEMAKER: An Insight Into the Artificial Heart Rhythm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21841337

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21841337

Country of ref document: EP

Kind code of ref document: A1