WO2002032035A2 - Procédé et dispositif de surveillance de systèmes dynamiques par affichage graphique intégré de repésentations multidimensionnelles des fonctions critiques - Google Patents

Procédé et dispositif de surveillance de systèmes dynamiques par affichage graphique intégré de repésentations multidimensionnelles des fonctions critiques Download PDF

Info

Publication number
WO2002032035A2
WO2002032035A2 PCT/US2001/031657 US0131657W WO0232035A2 WO 2002032035 A2 WO2002032035 A2 WO 2002032035A2 US 0131657 W US0131657 W US 0131657W WO 0232035 A2 WO0232035 A2 WO 0232035A2
Authority
WO
WIPO (PCT)
Prior art keywords
data
recited
display
monitoring
representation
Prior art date
Application number
PCT/US2001/031657
Other languages
English (en)
Other versions
WO2002032035A3 (fr
Inventor
Yinqi Zhang
Noah Syroid
James Agutter
Julio C. C. Bermudez
David Lee Strayer
Robert William Albert
Frank Drews
Michael E. Holmes
Stefano Foresti
Brent E. Lilly
Original Assignee
University Of Utah Research Foundation
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by University Of Utah Research Foundation filed Critical University Of Utah Research Foundation
Priority to AU2001296773A priority Critical patent/AU2001296773A1/en
Publication of WO2002032035A2 publication Critical patent/WO2002032035A2/fr
Publication of WO2002032035A3 publication Critical patent/WO2002032035A3/fr

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/74—Details of notification to user or communication with user or patient ; user input means
    • A61B5/742—Details of notification to user or communication with user or patient ; user input means using visual displays
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/026—Measuring blood flow
    • A61B5/029—Measuring or recording blood output from the heart, e.g. minute volume
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/083—Measuring rate of metabolism by using breath test, e.g. measuring rate of oxygen consumption

Definitions

  • This invention relates to the visualization, perception, representation and computation of data relating to the attributes or conditions constituting the health state of a dynamic system. More specifically, this invention relates to the graphic display of data, in which variables constituting attributes and conditions of a dynamic system can be interrelated and visually correlated in time as three-dimensional objects.
  • Another object of this invention is to provide a method and system that assists in the determination of .the functioning of a system by measuring the interaction among a set of "vital-signs" normally associated with the health of the system.
  • a still further object of this invention is to provide a method and system, which provides the gathering and use of sensor measured data, as well as the formatting and normalization of the data in a format suitable to the processing methodology.
  • a further object of this invention is to provide a method and system, which organizes a system's data into relevant data sets or critical functions as appropriate.
  • Another object of this invention is to provide a method and system, which provides both three-dimensional health-space as well as two-dimensional traditional space for mapping the system's data.
  • a further object of this invention is to provide a method and system where three-dimensional objects are built from three-dimensional object primitives, including: cubes, spheres, pyramids, n-polygon prisms, cylinders, slabs.
  • a still further object of this invention is to provide a method and system, wherein three-dimensional objects are placed within health-space based on the coordinates of their geometric centers, edges, vertices, or other definite geometric variables.
  • a still further object of this invention is to provide a method and system that permits the simultaneous display of the history of data objects.
  • Another object of this invention is to provide a method and system that provides for the selection of various user selectable viewports. It is a further object of this invention to provide a method and system that provides both a global and a local three-dimensional coordinate space.
  • a further object of this invention is to provide a method and system that provides viewpoints that can be selected to be perspective views, immersive Virtual Reality views, or any orthographic views.
  • Another object of this invention is to provide a method and system that permits the display of a layout of multiple time-space viewpoints.
  • a still further object of this invention is to provide a method and system that provides for zooming in and out of a time and/or space coordinate.
  • Another object of this invention is to provide a method and system that presents the data in familiar shapes, colors, and locations to enhance the usability of the data.
  • a still further object of the invention is to provide a method and system that uses animation, and sound to enhance the usefulness of the data to the user.
  • Figure la is a top-level representative diagram showing the data processing paths of the preferred embodiment of this invention.
  • Figure lb is a top-level block diagram of the data processing flow of the preferred embodiment of this invention.
  • Figure lc is a top-level block diagram of one preferred processing path of this invention.
  • Figure Id is a top-level block diagram of a second preferred processing path of this invention.
  • Figures 2a, 2b, 2c, and 2d are representative 3-D objects representing critical functions.
  • Figure 3 is a representation of data objects in H-space.
  • Figures 4a and 4b are representative views of changes in data objects in time.
  • Figures 5a, 5b, 5c, 5d, 5e, 5f, 5g and 5h are representative views of properties of data objects provided in the preferred embodiment of this invention.
  • Figure 6 shows a 3-D configuration of the objects in H-space in the preferred embodiment of the invention.
  • Figure 7 shows H-space with a time coordinate along with local-space coordinates.
  • Figures 8a and 8b show the global level coordinate system of the preferred embodiment of this invention.
  • Figures 9a and 9b show various viewpoints of the data within H-space in the preferred embodiment of this invention.
  • Figure 10 shows the transformation of an object in space in context, with a reference framework, in the preferred embodiment of this invention.
  • Figure 11a shows the zooming out function in the invention.
  • Figure 1 lb shows the zooming in function in the invention.
  • Figures 12a and 12b show a 3-D referential framework of normative values.
  • Figure 13 shows the interface modes of the preferred embodiment of this invention.
  • Figure 14 is a hardware system flow diagram showing various hardware components of the preferred embodiments of the invention.
  • Figure 15 is a software flow chart showing the logic steps of a preferred embodiment of the invention.
  • Figure 16 is a software block diagram showing the logic steps of the image computation and rendering process of a preferred embodiment of the invention.
  • Figure 17 is a photograph of the 3-dimensional display of a preferred embodiment of the invention.
  • Figure 18 is a close-up front view of the cardiac object and the associated reference grid of a preferred embodiment of the invention.
  • Figure 19 is a view of the front view portion of the display of a preferred embodiment of the present invention showing the cardiac object in the foreground and the respiratory object in the background.
  • Figure 20 is a view of the top view portion of the display of a preferred embodiment of the present invention showing the cardiac object toward the bottom of the view and the respiratory object toward the top of the view.
  • Figure 21 is a view of the side view portion of the display of a preferred embodiment of the present invention showing the cardiac object to the left and the respiratory object to the right.
  • Figure 22 is a view of the 3-D perspective view portion of the display of a preferred embodiment of the invention showing the cardiac object in the left foreground and the respiratory object in the right background.
  • Figure 23 is a top-level block diagram of the present preferred embodiment of the invention.
  • Figure 24 is a representative screen shot of the initialization of the present preferred embodiment of the invention.
  • Figure 25 is a block diagram showing the preferred UML description of VIZ of the invention.
  • Figure 26 is a representative cardiovascular display of the present preferred embodiment of this invention.
  • Figure 27 is a representative respiration system display of the present preferred. embodiment of this invention.
  • Figure 28 is a representative history display of the present preferred embodiment of this invention.
  • Figure 29 is a representative integrated global layout of the present preferred embodiment of this invention, showing a normal physiologic state.
  • Figure 30 is a representative integrated global layout of the present preferred embodiment of this invention, showing an anaphylaxis physiologic state.
  • Figure 31 is a detailed flowchart showing the preferred process steps of the integrated display of this invention.
  • FIG 32 is a detailed flowchart showing preferred process steps of the VIZ program component of this invention.
  • This invention is a method, system and apparatus for the visual display of complex sets of dynamic data.
  • this invention provides the means for efficiently analyzing, comparing and contrasting data, originating from either natural or artificial systems.
  • This invention provides n-dimensional visual representations of data through innovative use of orthogonal views, form, space, frameworks, color, shading, texture, transparency, sound and visual positioning of the data.
  • the preferred system of this invention includes one or a plurality of networked computer processing and display systems, which provide real-time as well as historical data, and which processes and formats the data into an audio-visual format with a visual combination of objects and models with which the user can interact to enhance the usefulness of the processed data.
  • An "artificial system” is an entity, process, combination of human designed parts, and/or environment that is created, designed or constructed by human intention.
  • artificial systems include manmade real or virtual processes, computer systems, electrical power systems, utility and construction systems, chemical processes and designed combinations, economic processes (including, financial transactions), agricultural processes, machines, and human designed organic entities.
  • a "natural system” is a functioning entity whose origin, processes and structures were not manmade or artificially created. Examples of natural systems are living organisms, ecological systems and various Earth environments.
  • the "health" of a system is the state of being of the system as defined by its freedom from disease, ailment, failure or inefficiency. A diseased or ill state is a detrimental departure from normal functional conditions, as defined by the nature or specifications of the particular system (using historical and normative statistical values).
  • the health of a functioning system refers to the soundness, wholeness, efficiency or well being of the entity. Moreover, the health of a system is determined by its functioning.
  • “Functions” are behaviors or operations that an entity performs. Functional fitness is measures by the interaction among a set of “vital-signs” normally taken or measured using methods well known in the art, from a system to establish the system's health state, typically at regular or defined time intervals.
  • H-space is the data representation environment that is used to map the data in three or more dimensions.
  • H-state is a particular 3-D configuration or composition that the various 3-D objects take in H-space at a particular time. In other words, H-state is a 3-D snapshot of the system's health at one point of time.
  • Life-space or “L-space” provides the present and past health states of a system in a historical and comparative view of the evolution of the system in time.
  • This 3-D representation environment constitutes the historical or Life-space of a dynamic system.
  • L-space allows for both continuous and categorical displays of temporal dependent complex data. In other words, L-space represents the health history or trajectory of the system in time.
  • Real-Time Representation is the display of a representation of the data within a fraction of a second from the time when the event of the measured data occurred in the dynamic system.
  • Real-Time User Interface is the seemingly instantaneous response in the representation due to user interactivity (such as rotation and zooming).
  • a “variable” is a time dependent information unit (one unit per time increment) related to sensing a given and constant feature of the dynamic system.
  • Vital signs are key indicators that measure the system's critical functions or physiology.
  • data is gathered using methods or processes well known in the art or as appropriate and necessary.
  • physiologic data such as heart rate, respiration rate and volume, blood pressure, and the like
  • Sensor-measured data is electronically transferred and translated into a digital data format to permit use by the invention.
  • This invention uses the received measured data to deliver real-time and/or historical representations of the data and/or recorded data for later replay.
  • this invention permits the monitoring of the health of a dynamic system in a distributed environment.
  • distributed environment it is meant that a user or users interacting with the monitoring system may be in separate locations from the location of the dynamic system being monitored.
  • the monitoring system of this invention has three major logical components: (1) the sensors that measure the data of the system; (2) the networked computational information systems that computes the representation and that exchanges data with the sensors and the user interface; and (3) the interactive user interface that displays the desired representation and that interactively accepts the users' inputs.
  • the components and devices that perform the three major functions of this invention may be multiple, may be in the same or different physical locations, and/or may be assigned to a specific process or shared by multiple processes.
  • FIG. la is a top-level representative diagram showing the data processing paths of the preferred embodiment of this invention operating on a natural system.
  • the natural system 101a is shown as a dynamic entity whose origin, processes and structures (although not necessarily its maintenance) were not manmade or artificially created. Examples of natural systems are living organisms, ecological systems, and various Earth environments.
  • a human being is the natural system whose physiology is being monitored. Attached to the natural system 101a are a number of sensors 102. These sensors 102 collect the physiologic data, thereby measuring the selected critical functions of the natural system. Typically, the data gathering of the sensors 102 is accomplished with methods or techniques well known in the art.
  • the sensors 102 are typically and preferably electrically connected to a digital data formatter 103.
  • this digital data formatter 103 is a high-speed analog to digital converter. Also, connected to the digital data formatter 103 is the simulator 101b.
  • the simulator 101b is an apparatus or process designed to simulate the physiologic process underlying the life of the natural system 101a.
  • a simulator 101b is provided to generate vital sign data in place of a natural system 101a, for such purposes as education, research, system test, and calibration.
  • the output of the digital data formatter 103 is Real-Time data 104.
  • Real- Time data 104 may vary based on the natural system 101a being monitored or the simulator 101b being used and can be selected to follow any desired time frame, for example time frames ranging from one-second periodic intervals, for the refreshment rates of patients in surgery, to monthly statistics reporting in an ecological system.
  • the Real-Time data 104 is provided to a data recorder 105, which provides the means for recording data for later review and analysis, and to a data modeling processor and process 108.
  • the data recorder 105 uses processor controlled digital memory
  • the data modeling processor and process 108 is one or more digital computer devices, each having a processor, memory, display, input and output devices and a network connection.
  • the data recorder 105 provides the recorded data to a speed controller 106, which permits the user to speed-up or slow-down the replay of recorded information.
  • Scalar manipulations of the time (speed) in the context of the 3-D modeling of the dynamic recorded digital data allows for new and improved methods or reviewing the health of the systems 101a,b.
  • a customize / standardize function 107 is provided to permit the data modeling to be constructed and viewed in a wide variety of ways according to the user's needs or intentions.
  • Customization 107 includes the ability to modify spatial scale, such modifying includes but is not limited to zooming, translating, and rotating, attributes and viewports in addition to speed.
  • the range of customization 107 permitted for monitoring natural systems 101a physiologic states is reduced and is heavily standardized in order to ensure that data is presented in a common format that leads to common interpretations among a diverse set of users.
  • the data modeling processor and process 108 uses the prescribed design parameters, the standardized/customize function and the received data to build a three-dimensional (3-D) model in real-time and to deliver it to an attached display.
  • the attached display of the data modeling processor and process 108 presents a representation 109 of 3-D objects in 3-D space in time to provide the visual representation of the health of the natural system 101a in time, or as in the described instances of the simulated 101b system.
  • FIG lb is a top-level block diagram of the data processing flow of the preferred embodiment of this invention operating on an artificial system.
  • An artificial system is a dynamic entity whose origin, processes and structure have been designed and constructed by human intention. Examples of artificial systems are manmade real or virtual, mechanical, electrical, chemical and/or organic entities.
  • the artificial system 110a is shown attached to a number of sensors 111. These sensors 111 collect the various desired data, thereby measuring the selected critical functions of the artificial system. Typically, the data gathering of the sensors 111 is accomplished with methods or techniques well known in the art.
  • the sensors 111 are connected to a data formatter 112, although alternative connection means including optical, RF and the like may be substituted without departing from the concept of this invention.
  • this digital data formatter 112 is a high-speed analog to digital converter. Although, in certain applications of the invention, namely stock market transactions, the data is communicated initially by people making trades. Also connected to the digital data formatter 112 is the simulator 110b.
  • the simulator 110b is an apparatus or process designed to simulate the process underlying the state of the artificial system 110a.
  • the simulator 110b is provided to generate vital data in place of the artificial system 110a, for such purposes as education, research, system test, and calibration.
  • the output of the digital data formatter 112 is Real-Time data 113.
  • Real-Time data 113 may vary based on the artificial system 110a being monitored or the simulator 110b being used and can be selected to follow any desired time frame, for example time frames ranging from microsecond periodic intervals, for the analysis of electronic systems, to daily statistics reported in an financial trading system.
  • the Real-Time data 113 is provided to a data recorder 114, which provides the means for recording data for later review and analysis, and to a data modeling processor and process 117.
  • the data recorder 114 uses processor controlled digital memory, and the data modeling processor and process 117 is one or more digital computer devices, each having a processor, memory, display, input and output devices and a network connection.
  • the data recorder 114 provides the recorded data to a speed controller 115, which permits the user to speed-up or slow-down, the replay of recorded information.
  • Scalar manipulations of the time (speed) in the context of the 3-D modeling of the dynamic recorded digital data allows for new and improved methods or reviewing the health of the system 110a,b.
  • a customize / standardize function 116 is provided to permit the data modeling to be constructed and viewed in a wide variety of ways according to the user's needs or intentions.
  • Customization 116 includes the ability to modify spatial scale (such modification including, but not limited to translating, rotating, and zooming), attributes, other structural and symbolic parameters, and viewports in addition to speed.
  • the range of customization form monitoring artificial systems' 110a,b states is wide and not as standardized as that used in the preferred embodiment of the natural system 101 a,b monitoring.
  • the symbolic system and display method is fully adaptable to the user's needs and interests.
  • this invention has a default visualization space, its rules, parameters, structure, time intervals, and overall design are completely customizable.
  • This interface mode customize/standardize function 116 also allows the user to select what information to view and how to display the data.
  • This interface mode customization 116 may, in some preferred embodiments, produce personalized displays that although they may be incomprehensible to other users, facilitate highly individual or competitive pursuits not limited to standardized interpretations, and therefore permit a user to look at data in a new manner.
  • the data modeling processor and process 117 uses the prescribed design parameters, the customize/standardized function 116 and the received real-time data 113 to build a three-dimensional (3-D) model in time and to deliver it to a display.
  • the display of the data modeling processor and process 117 presents a representation 118 of 3-D objects in 3-D space in time to provide the visual representation of the health of the artificial system 110a in time, or as in the described instances of the simulated 110b system.
  • Figure lc is a top-level block diagram of one preferred processing path of this invention.
  • Sensors 119 collect the desired signals and transfer them as electrical impulses to the appropriate data creation apparatus 120.
  • the data creation apparatus 120 converts the received electrical impulses into digital data.
  • a data formatter 121 receives the digital data from the data creation apparatus 120 to provide appropriate formatted data for the data recorder 122.
  • the data recorder 122 provides digital storage of data for processing and display.
  • a data processor 123 receives the output from the data recorder 122.
  • the data processor 123 includes a data organizer 124 for formatting the received data for further processing.
  • the data modeler 125 receives the data from the data organizer and prepares the models for representing to the user.
  • the computed models are received by the data representer 126, which formats the models for presentation on a computer display device.
  • Receiving the formatted data from the data processor 123 are a number of data communication devices 127, 130. These devices 127, 130 include a central processing unit, which controls the image provided to one or more local displays 128, 131.
  • the local displays may be interfaced with a custom interface module 129 which provides user control of such attributes as speed 131, object attributes 132, viewports 133, zoom 134 and other like user controls 135.
  • Figure Id is a top-level block diagram of a second preferred processing path of this invention.
  • a plurality of entities 136a,b,c are attached to sensors 137a,b,c which communicate sensor data to a data collection mechanism 138, which receives and organizes the sensed data.
  • the data collection mechanism 138 is connected 139 to the data normalize and formatting process 140.
  • the data normalize and formatting process 140 passes the normalized and formatted data 141 to the distributed processors 142.
  • the processing 142 is distributed over the Internet, although alternative communication networks may be substituted without departing from the concept of this invention.
  • Each processing unit 142 is connected to any of the display devices 143a,b,c and receives command control from a user from a number of interface units 144a,b,c, each of which may also be connected directly to a display devices 143a,b,c.
  • the interface units 144a,b,c receive commands 145 from the user that provide speed, zoom and other visual attributes controls to the displays 143a,b,c.
  • FIGS. 2a, 2b, 2c, and 2d are representative 3-D objects representing critical functions.
  • Each 3-D object is provided as a symbol for a critical function of the entity whose health is being monitored. The symbol is created by selecting the interdependent variables that measure a particular physiologic function and expressing the variable in spatial (x,y,z) and other dimensions.
  • Each 3-D object is built from 3-D object primitives (i.e., a cube, a sphere, a pyramid, a n-polygon prism, a cylinder, a slab, etc.).
  • the spatial dimensions are modeled after the most important physiologic variables based on (1) data interdependency relationships, (2) rate, type and magnitude of change in data flow, (3) geometric nature and perceptual potential of the 3-D object, for example a pyramid versus a cylinder, (4) potential of the object's volume to be a data- variable itself by modeling appropriate data into x, y and z dimensions (e.g., in one preferred application of the invention, cardiac output is the result of heart rate (x and y dimensions) and stroke volume (z)), (5) orthographic viewing potential (see viewport) and (6) the relationship with the normal values framework.
  • the first representative object 201 is an engine process.
  • the object 201 representing this process is provided on a standard x-y-z coordinate axis 202.
  • the correlation between temperature, shown in the xl-dimension 204, engine RPM, shown in the yl -dimension 205 and exhaust gas volume, shown in the zl -dimension 203 is shown by changes in the overall sizes and proportion of the object 201.
  • the engine gas volume 203 is large, when RPM 205 is low and the engine temperature 204 is in the middle range. This combination of values, even without specific identified values suggests an engine's starting point.
  • the second representative object 206 is an object representing cardiac function using stroke volume, in the y2-dimension 209, and the heart rate per second, shown as the x2, z2 dimensions.
  • the total cardiac volume is shown as the total spherical volume 208.
  • the third representative object 211 shown in figure 2c, represents the interaction between the number of contracts, shown in the y3-dimension 212, the average revenue per contract, shown in the z3-dimension 214, and the average time per contract, shown in the x3-dimension 213. Assessing the interaction among these variables is important in monitoring of a sales department's operations.
  • the fourth representative object 215 is shown in figure 2d, shows the respiratory function generated by the respiratory rate, shown in x4-dimension 216, the respiratory volume, shown in the y4-dimension 216, and inhalation / exhalations, shown in the z4-dimension 218.
  • FIG 3 is a representation of data objects in H-space 301.
  • Data sets are represented as 3-D objects of various characteristics and relationships within a 3-D representation space.
  • the data representation environment in this figure is used to map the physiologic data in 3-D and is what is referred to as "Health-space” or "H- space” 301.
  • the 3-D objects are placed within H-space on the 3 coordinates of their geometric centers. The coordinates for an object's geometric center depends on the relevant data associated to the particular critical function the object represents.
  • the cardiac function object shown as a spherical object 302
  • Oy 306 Mean Blood Pressure
  • Oxygen Saturation in the Blood shown as Oz 307.
  • the prism 309 is placed in H-space 301 depending on sales profit, shown as Py 312, and products in stock, shown as Pz, 311.
  • the location of 3-D objects in H-space 301 allows for the overall extension envelope of H-space, the relationship between 3-D objects and spaces within H-space 301, the viewport display areas and the departure from normative values.
  • the centers of the objects 302, 309 are located in the middle of the x-dimension of H-space 301.
  • Figures 4a and 4b are representative views of changes in data objects in time.
  • the x-coordinate 400 is used to measure the temporal dimension of an objects 402 trajectory.
  • the y-z plane 401a determines the location of an object's geometric center within H-space. Increases or decreases in data values associated with the coordinates of the object's geometric center that make that object's location change in time as shown in path line 401b.
  • the object 402 is presented in four different time intervals 403, 404, 405, 406, thereby creating a historical trajectory.
  • the time intervals at which the object 402 is shown are provided 407.
  • increases in size and proportion are presented, 408, 409, 410, 411 providing an example of changes in values. The monitoring of these changes in time assists the user establish and evaluate comparative relationships within and across H-states.
  • Figures 5a, 5b, 5c, 5d, 5e, 5f, 5g and 5h are representative views of properties of data objects provided in the preferred embodiment of this invention.
  • 3-D objects may present data value states by using other geometric, aesthetic, and aural attributes that provide for the mapping of more physiologic data.
  • Figure 5a shows changes in apparent volumetric density.
  • a solid object 501 is shown in relation to a void object 502 and an intermediate state 503 object.
  • Figure 5b shows changes in apparent 3-D enclosure.
  • An open object 504, a closed object 505, and an intermediate state 506 is shown.
  • Figure 5c shows the apparent degree of formal deformation.
  • a normal object 507, a distorted object 508, a transformed object 509, and a destroyed object 510 are shown in comparison.
  • Figure 5d shows secondary forms of the objects. "Needles" 513 protruding through a standard object 512 in combination 511 is shown in comparison with a boundary 515 surrounding a standard object 514 and a bar 517 protruding into the original form object 518 forming a new combination object 516 are shown providing additional combination supported in this invention.
  • Figure 5e shows the various degrees of opacity of the object's surface, showing an opaque object 519, a transparent object 520 and an intermediate state object 521.
  • Figure 5f shows the various degrees of texture supported by the object display of this invention, including a textured object 522, a smooth object 523 and an intermediate textured object 524.
  • Figure 5g is intended to represent various color hue possibilities supported for objects in this invention.
  • An object with color hue is represented 525 next to a value hue object 526 and a saturation hue object 527 for relative comparison.
  • Color are used rather than simply the representation of color shown in figure 5g.
  • Figure 5h shows the atmospheric density of the representation space possible in the display of objects in this invention.
  • An empty- clear space 528, a full-dark space 530 and an intermediate foggy space 523 are shown with 3-D objects shown within the representative space 529, 531, 533.
  • FIG. 6 shows the 3-D configuration of the objects in H-space in the preferred embodiment of the invention.
  • H-space 601 is shown within which the 3-D objects 602, 603, and 604 are located.
  • Object 602 represents the respiratory function of an individual. Its 602 x-y-z dimensions change based on the parameter-based dimensional correlation.
  • the object 603 represents the efficiency of the cardiac system by varying the x,y,z coordinates of the object.
  • the object 604 represents a human brain function, also with the x,y,z dimensions changing based on the parameter-based dimensional correlation.
  • the temporal coordinate i.e., periodic time interval for data capturing that defines how H-space is plotted in Live-space - see figure 7
  • the x-dimension of 605 of the H-space 601 can be mapped to another independent variable such as heart rate period, blood pressure or the like.
  • the location of an object in the y-dimension 606 of H-space 601 can be mapped to additional variables that are desired to be monitored such as Sa02 content, CaO2 content, or temperature in the blood.
  • the location of an object in the z- dimension 607 of the H-space 601 can also be mapped to additional variables that the user desires to monitor.
  • a hypothetical object 608 shows that the three coordinates are contextual to a particular object 608 and need not be the same for all objects, except in the object's 608 extension measuring properties.
  • Fixed x- and z-dimension values 609a and 609b are shown as constant.
  • the y-value 610 of this object 608 changes to fluctuating values or data type that results in the height of the object 608 increasing or decreasing.
  • This view shows another object 611 showing the relationship between the three dimensions. Constant x- and y-values 612a and 612b are shown.
  • the z-value 613 of this object 611 changes to fluctuating values or data types that result in the width of the object 611 increasing or decreasing.
  • FIG. 7 shows a series of H-spaces 701, 702, 703, 704, 705, 706 along a global time coordinate 708, and the local-space coordinates 707 that governs each H- space.
  • Each of these H-spaces represents progressive states of the dynamic system at pre-established temporal intervals (T 0 , T- 1; T- 2 , . . . T- n ) and the six 701, 702, 703, 704, 705, 706 together show the evolution of that system over time, demonstrating the historical representation of individual H-states within an overall "Life-space" or "L- space.”
  • T 0 , T- 1; T- 2 , . . . T- n the six 701, 702, 703, 704, 705, 706 together show the evolution of that system over time, demonstrating the historical representation of individual H-states within an overall "Life-space" or "L- space.”
  • L-space At the global level (or L-space), one of the coordinates, typically x, is always time.
  • the temporal coordinate is scaled based on the intervals at which a particular functions system's physiologic data are collected by the art or as appropriate.
  • This interval or module is fixed and constant across L-space and provides the necessary temporal frame of reference for comparing different H-spaces.
  • the fixed temporal interval also determines the maximum x-extension of the representation envelope of H-space.
  • the other two coordinates, y and z, provide L-space with extension and are not fixed. The three coordinates thus described provide a regulating 3-D environment within which the H-states can be visualized and related to each other.
  • Figures 8 a and 8b show the global level coordinate system of the preferred embodiment of this invention.
  • Figure 8a shows the L-space coordinate system 801 in its preferred embodiment.
  • the x-dimension 802 of L-space is mapped to a constant time interval, set by means standard in the art or otherwise as appropriate.
  • the present position of H-state is also indicated on the x-dimension 802.
  • the y-dimension 803 in both positive and negative extensions is measured, up and down from the x- axis.
  • This dimension 803 can be mapped to a data variable within particular 3D object in space.
  • the z-dimension 804 is shown in both positive and negative extensions measured forwards and backwards from the intersecting x-axis.
  • This dimension 804 can be mapped to a data variable within a particular 3D object in space.
  • a prismatic object 800 represents a critical function, whose evolution is being monitored in L-space, of a given dynamic system.
  • the front view 805 shows the different H-states of the prism/function 800 using a time T to T-n historical trend.
  • the level of intersection and separation between the front views of the prism indicate abnormal health states of the critical function the object 800 represents. No separation or intersection shows normal function conditions.
  • the trajectory in the y-dimension of the prism i.e., H-states of the critical function
  • the current state 806 of the prism is shown in this front view 805.
  • a top view of 809 of the three-dimensional L-space is shown, showing the evolution of the prism 800 backward in time and showing a T to T-N historical trend.
  • the level of intersection and separation indicate abnormal health states of the particular critical function the prism represents. No separation or intersection shows normal conditions.
  • the trajectory in the z-dimension of the object is mapped to a variable that causes their position to change in the + and -z dimension.
  • This top view shows both the z and y trajectories in one comprehensive view.
  • the perspective view 808 of L-space gives a comprehensive view of the interaction of the prisms (the H-states of the function) and their movement in all dimensions.
  • the side view 807 of L-space shows the prisms and their positions in L-space giving a simultaneous view of z and y trajectories.
  • Figures 9a and 9b shows various viewpoints in which the data may be visualized in the preferred embodiment of this invention.
  • This figure shows representations of a data object (a prism) and is provided to show that there are two basic types of viewports: orthographic and naval.
  • the orthographic viewports 906, 907, 908, of figure 9b use a parallel system of projection to generate representations of H-space that maintains dimensional constancy without deformation.
  • Some examples of orthographic views include traditional architectural or engineering views of objects, such as a top view, a front view, and a side view.
  • the orthographic viewport allows for accurate and focused 2-D expressions of the actual 3-D object.
  • the perspective viewport 909 uses a focal system of projection to generate depictions analogous to our perception of reality but at the cost of deformation and lack of dimensional constancy.
  • the top view 902 along with the side view 903 and the front view of 904 of the 3-D data object 901 are shown in figure 9a.
  • Figure 9b shows three orthogonal views 906, 907, 908 along with a perspective view 909 of the current data object.
  • the number and types of viewports used in a particular embodiment of the invention may range from one type, for example a perspective viewport allowing immerse virtual reality, to combinations of both types. In the preferred current embodiment, there are the four viewports shown in figure 9b.
  • Figure 10 shows the transform of an object in space in context, with a reference framework, in the preferred embodiment of this invention.
  • the referential framework 1010 is typically set based on population normals or patient normals. This framework assists the user to see deviations from normal very quickly.
  • An individual spherical object 1011 that represents cardiac function is shown located in L-space and in relation to the referential framework.
  • a side view 1012 is shown along with several cardiac objects. In this view the referential framework provides a center target point so that a user can make the necessary corrections to bring the object back to the ideal center of the framework.
  • a perspectivel view 1013 of the framework is also shown along with several cardiac objects.
  • the top view 1014 of the framework is shown with several spherical objects (representing cardiac states). This figure demonstrates the variety of viewports provided to the user by this invention, which provides enhanced flexibility of analysis of the displayed data.
  • Figure 11a shows the zooming out function in the invention.
  • This invention provides a variety of data display functions.
  • This figure shows the way views may be zoomed in and out providing the relative expansion or compression of the time coordinate.
  • Zooming out 1101 permits the user to look at the evolution of the system's health as it implies the relative diminution of H-states and the expansion of L-space.
  • This view 1101 shows a zoomed out view of the front view showing a historical view of many health states.
  • a side view 1102 zoomed out view is provided to show the historical trend stacking up behind the current view.
  • a 3-D perspectivel, zoomed out view 1103 showing the interaction of H-states over a significant amount of time is provided.
  • a zoomed out top view 1104 shows the interaction of H-states over a large amount of time.
  • Figure 1 lb shows the zooming in function of the invention.
  • the zooming in front view 1105 is shown providing an example of how zooming in permits a user to focus in on one or a few H-states to closely study specific data to determine with precision to the forces acting on a particular H-state.
  • a zoomed in side view 1106 is provided showing the details of specific variables and their interactions.
  • a zoomed in 3-D perspective view 1107 of a few objects is also shown.
  • a zoomed in top view 1108 showing the details of specific variables and their interaction.
  • Figures 12a shows a 3-D referential framework of normative values that is provided to permit the user a direct comparison between existing and normative health states, thereby allowing rapid detection of abnormal states.
  • the reference framework 1201 works at both the global L-space level and the local H-space level. "Normal" values are established based on average historical behavior of a wide population of systems similar to the one whose health is being monitored. This normal value constitutes the initial or by-default ideal value, which, if necessary may be adjusted to acknowledge the particular characteristics of a specific system or to follow user-determined specifications.
  • the highest normal value of vital sign "A" 1202 (+y) is shown, along with the lowest normal value of "B" 1203 (-z), the lowest normal value of vital sign "A” 1204 (-y) and the highest normal value of vital sign "B” 1205 (+z).
  • abnormal values of "A" and “B” are shown in an orthogonal view. An abnormally high value of "A” 1206, an abnormally low value of "B” 1207, an abnormally low value of "A” 1208 and an abnormally high value of "B” 1209 are shown.
  • Figure 13 shows a comparison of the interface modes of the preferred embodiment of this invention.
  • This invention provides two basic types of interface modes: (a) standardized or constrained customization; and (b) free or total customization. Each is directed toward different types of applications.
  • the standardized or constrained customization 1301 uses a method and apparatus for user interface that is set a-priori by the designer and allows little customization.
  • This interface mode establishes a stable, common, and standard symbolic system and displaying method that is "user-resistant".
  • the fundamental rules, parameters, structure, time intervals, and overall design of L-space and H-space are not customizable. Such a normalized symbolic organization creates a common interpretative ground upon which different users may arrive at similar conclusions when provided common or similar health conditions.
  • This interface method is intended for social disciplines, such as medicine in which common and agreeable interpretations of the data are highly sought after to ensure appropriate and verifiable monitoring, diagnosis and treatment of health states.
  • the customization permitted in this mode is minimal and is never threatening to render the monitoring device incomprehensible to other users.
  • the free or total customization interface mode 1302 provides a symbolic system and displaying method that is changeable according to the user's individual needs and interests. Although the invention comes with a default symbolic L-space and H-space, its rules, parameters, structure, time intervals, and overall design are customizable. This interface mode also permits the user to select what information the user wishes to view as well as how the user wishes to display it. This interface mode may produce personalized displays that are incomprehensible to other users, but provides flexibility that is highly desired in individual or competitive pursuits that do not require agreeable or verifiable interpretations. Examples of appropriate applications may include the stock market and corporate health data monitoring.
  • Figure 14 is a hardware system flow diagram showing various hardware components of the preferred embodiments of the invention in a "natural system" medical application.
  • a decision 1401 is made as to the option of using data monitored on a "real" system, that is a real patient, or data from the simulator, for anesthesiology training purposes. If the data is from a real patient, then the patient 1402 is provided with patient sensors 1404, which are used to collect physiological data. Various types of sensors, including but not limited to non-invasive BP sensors, ECG leads, SaO2 sensors and the like may be used. Digital sensors 1416 may also provide physiological data. An A/D converter 1405 is provided in the interface box, which receives the analog sensor signals and outputs digital data to a traditional patient monitor 1406. If the data is produced 1401 by the simulator 1403, a control box and mannequins are used.
  • the control box controls the scenarios simulated and the setup values of each physiological variable.
  • the mannequins generate the physiological data that simulates real patient data and doctors collect the data through different, but comparable sensors.
  • the traditional patient monitor 1406 displays the physiological data from the interface box on the screen. Typically and preferably, this monitor 1406 is the monitor used generally in an ICU.
  • a test 1407 is made to determine the option of where the computations and user interface are made, that is whether they are made on the network server 1408 or otherwise. If a network server 1408 is used, all or part of the data collection and computation may be performed on this computer server 1408.
  • An option 1409 is proved for running a real time representation versus a representation delayed or replayed from events that previously occurred.
  • a data buffer 1410 is provided to cache the data so that the representation is played in real time.
  • a data file 1411 provides the means for permanently storing the data so that visualization is replayed.
  • the visualization software 1412 runs on a personal computer and can display on its monitor or on remote displays via the internet or other networking mechanism. Typically the physiological data measured on either a real patient or the simulator are fed to the personal computer from the traditional data monitor.
  • a standard interface such as RS232, the Internet, or via a server, which receives data from the monitor, may serve as the communication channel to the personal computer running the visualization software 1412.
  • This program 1412 is the heart of the invention.
  • the program 1412 computes the representation and processes the user interface.
  • FIG. 15 is a software flow chart showing the logic steps of a preferred embodiment of the invention.
  • the preferred embodiment of this invention begins by reading the startup file 1501, which contains the name of the window and the properties associated with the invention.
  • the properties associated with the a window include formulas to set object properties, text that is to be rendered in the scene, the initial size of the window, the initial rotation in each window, zoom, lighting and patient data that describes the normal state of each variable.
  • Internal data tables are next initialized 1502. For each new window encountered in the startup file a new window object is made and this window object is appended to the list of windows.
  • the window object contains an uninitialized list of properties describing the state of the window, which is filled with data from the startup file.
  • the event loop is entered 1503. This is a window system dependent infinite loop from which the program does not exit. After some initialization, the program waits for user input and then acts on this input. The program then takes control of the event loop for continuous rendering that is if there is no interactivity in the program.
  • Initialization 1504 of windows is next performed. This involves calls to the window system dependent functions (these are functions that are usually different on different computational platforms) that creates the windows and displays them on the computer screen.
  • OpenGL is required, although alternative embodiments using other 3D application programming interfaces, such as PEX or DirectX, could be substituted without departing from the concept of this invention.
  • a personal computer graphics card is preferred in the personal computer so as to permit smooth animation with multiple windows. Although the lack of such a card is not absolutely required for operation of this invention.
  • New data is received 1509, typically from the data file 1506 or the data buffer 1507. This new data 1509 can come from any source that generates floating-point numbers.
  • the preferred line of data is composed of columns of floating point numbers separated by space. At this point the current time is also stored so that the next line of data can be obtained at the next user defined time interval, which is typically set at about 1 second.
  • Object properties are next computed 1510. This is performed by using formulas that are specified in the startup file to compute properties of objects. Data fields in the formulas are specified by writing the column number preceded by a dollar sign. For example, $1 / 20.0 would divide the first field by 20.0.
  • the specific properties in this application are: cardiac object dimensions, material properties, and position. Material properties can include the red, green, and blue components as they appear under ambient, diffuse, and specular light, as well as transparency. The cardiac object position includes the y and z positions as well as an x shift. If four or more lines of data have been acquired, the respiratory object properties are computed. A delay is necessary because a cubic spline is fitted, using four data points to do the fit, to the data points to generate a smooth respiratory object.
  • Cardiac object properties include material properties and the height of the color bands. Blood pressure object length and materials are the thin cylinders that go through the top and bottom of each ellipsoid.
  • reference grid properties are computed. All objects, except the cardiac object reference are stationary, in the current implementation. The cardiac object reference can move according to the movement of the cardiac object if the user specifies this movement in the startup file.
  • sounds are computed 1511 and made audible 1513.
  • Objects and reference grids are rendered 1512. Before rotation the newest object appears at the right side of the screen and oldest object is at the left side of the screen. Sound is produced 1513 next.
  • a test 1514 is next made to determine if smooth animation is selected. If smooth animation is selected the scene will scroll during the time the program is waiting to get new data. The program, using available computing resources, selects the minimum time increment so that the shift of the objects can be rendered within the increment, but limiting the increment to the smallest increment that human eyes can detect. If smooth animation is not selected, objects are shifted to the left 1515 such that the distance from the center of the newest cardiac object to that of the former cardiac object is equal to the inter-cardiac spacing. The process waits 1508 until the current time minus the time since data was last obtained equals the data acquisition period specified by the user. If the current time minus the time when the data was last acquired equals the user specified data acquisition period then a new line of data is acquired.
  • Figure 16 is a software block diagram showing the logic steps of the image computation and rendering process of a preferred embodiment of the invention. This process begins with acquiring the window identification 1601 of the current rendering context. Next, the data structure is found 1602 corresponding to the current window identification. After which, the view is set 1603. A rotation matrix is set 1604. A projection matrix is set 1605. Lights are set 1606. The back buffer is cleared 1607.
  • Object processing 1608 begins, and includes for each cardiac object, calling OpenGL to see material properties; shift left one inter-cardiac-object distance; push the modelview matrix, shift x,y, and z directions; call OpenGL utility toolkit to render the cardiac object; set the top cardiac object material properties, call OpenGL quadries function to render top cardiac object; set top cardiac object material properties, call OpenGL quadries function to render bottom cardiac object and pop modelview matrix.
  • the view is set 1609, as above.
  • the respiratory object is rendered 1610, by setting OpenGL to render quad strips, for each polygon strip set material properties, and send vertex to OpenGL.
  • Reference grids are rendered 1611 by setting material property of the cardiac reference grid.
  • the current position is set 1612 to be the ideal position of the newest cardiac object, that is the position corresponding to a patient in ideal health.
  • the cardiac object material properties are set 1613.
  • the OpenGL utility toolkit is called to render 1614 the cardiac object.
  • OpenGL is set to render quads 1615.
  • Vertices that compose the reference planes through the OpenGL pipeline are sent 1617 and buffers are swapped 1618. Buffer swap is a window system lawyer function.
  • Figure 17 is a photograph of the 3-dimensional display of a preferred embodiment of the invention.
  • the 3-D view shown at lower right 1706 provides a comprehensive, integrated and interactive view of all physiological data, and shows the interaction between the different objects in relation to the reference frame. This view can be manipulated by the user to fit specific application needs.
  • the front 1701, side 1704, 1705 and top views 1702 show how the same data appears from different vantage points.
  • these views 1701, 1702, 1704, 1705 show the interaction between the cardiac object, the reference frame and the respiratory object, with the side view 1704 providing a target for optimum efficiency of the cardiac system 1705 shows the level of gas concentration in the lungs and overall tidal volume in the respiratory system.
  • This figure 17 is a representation of a true 3-D model of the physiologic data.
  • the circle 1703 shown is the top view of the respiratory waveform showing CO2 content in the lungs and inspiration and expiration values.
  • the object grows and shrinks with each heartbeat. Its height is proportional to the heart's volume output and its width is proportional to heart rate.
  • the gridframe (or reference framework) shows the expected normal values for stroke volume and heart rate. The position of this object in the vertical direction of the display is proportional to the patient's mean blood pressure.
  • This graphic objects shape and animation provides a useful graphical similarity to a working heart.
  • the background is colored to show inspired and expired gases.
  • the height of the "curtain” is proportional to tidal volume, while the width is proportional to respiratory rate.
  • the colors, which are, displayed in the preferred display show the concentrations of respiratory gases. Time is set to move from right to left, with the present or current conditions shown at the "front" or right edge of each view. Past states remain to provide a historical view of the data.
  • Figure 18 is a close-up front view of the cardiac object and the associated reference framework of a preferred embodiment of the invention.
  • the upper limit of normal blood pressure value is shown 1800 on the reference frame.
  • the systolic blood pressure level is indicated by the bar 1801 penetrating the cardiac sphere 1806.
  • the height 1802 of the sphere 1806 is proportional to cardiac output, which shows the optimum efficiency of the heart.
  • the width of the sphere 1806 is proportional to 1/heart rate.
  • the elevation of the sphere 1806 is an indication of mean blood pressure, where the center reference gridline is a normal mean blood pressure 1803.
  • the lower limit, or diastolic blood pressure 1804 is shown by the length of the bar extending downward from the sphere 1806.
  • FIG. 19 is a view of the front view portion of the display of a preferred embodiment of the present invention showing the cardiac object in the foreground and the respiratory object in the background.
  • This view 1900 provides a more quantitative image of the hemodynamic variables, stroke volume, blood pressure 1901 and heart rate.
  • the "normal" reference lines are more apparent.
  • respiration is shown by changes in the background color.
  • Figure 20 is a view of the top view portion of the display 2000 of a preferred embodiment of the present invention showing the cardiac object toward the bottom of the view and the respiratory object toward the top of the view. Inhaled gas 2002 and exhaled gas 2003. CO2 concentrations and oxygen saturation of the arterial blood 2001 versus time are also shown.
  • Figure 21 is a view of the side view portion of the display of a preferred embodiment of the present invention showing the cardiac object to the left and the respiratory object to the right. Gas concentration in the lungs 2101, a calibrated scale for gas concentration 2103, blood pressure 2100, and oxygen saturation 2101 are shown.
  • the end view, shown here in figure 21, is especially useful during treatment, where the goal is to bring the variables back to the center or normal state. Functional relationships can be added to this view to predict how treatment can be expected to bring the variables back to normal.
  • Figure 22 is a view of the 3-D perspective view portion of the display of a preferred embodiment of the present invention showing the cardiac object in the left foreground and the respiratory object in the right background. This view 2200 provides a comprehensive, integrated and interactive view of nine physiological variables.
  • the sphere 2201 grows and shrinks with each heartbeat. Its height is proportional to the heart's stroke volume and its width is proportional to heart rate.
  • This graphic object 2201 offers useful similarity to a beating heart.
  • the gridframe 2202 shows the expected normal values for stroke volume and heart rate. The position of this object 2201 on the screen is proportional to the patient's mean blood pressure.
  • the ends of the bar 2203 drawn vertically through the center of the heart object show systolic and diastolic blood pressure.
  • the background 2204 is colored to show inspired and expired gases.
  • the height of the "curtain" 2205 is proportional to tidal volume.
  • the width of each fold 2206 is proportional to respiratory rate. In the preferred embodiment colors are used to show the concentrations of respiratory gases. Time moves from right to left with the present condition shown at the "front" or right edge of the view 2200. Past states 2207 remain to permit a historical view of the data.
  • Figure 23 shows a top-level block diagram of the present preferred embodiment of this invention.
  • This figure shows the data flow from its source in a body simulator 2301 to the data display (VIZ) 2305.
  • the physiological parameters generated by the body simulator 2301 are transferred to the graphic information display software process (VIZ) 2305 via the BodyDPC, inter process communication program 2303.
  • the body simulator 2301 is an anesthesia simulation software package, developed by Advanced
  • This software serves as a multi-media interactive anesthesia trainer that has been implemented on a personal computer. It simulates a patient, an anesthesia workstation, a ventilator and a gas delivery circuit, parts of the operating room, and even some operating room personnel.
  • the simulated data are exported in real time via an RS-232 serial communication port 2302.
  • the BodylPC 2303 program collects the body simulator 2301 data from the serial port 2302 and transmits the data to the VIZ software process 2305, typically over a network 2304 using the UDP protocol.
  • the body simulator 2301 also records the output of data of specific scenarios to a file. With BodylPC 2303, the recorded files can be replayed to repeat predefined scenarios for display evaluation.
  • BodylPC 2303 works as a multi-threaded application written in Visual C++ 6.0 under the Windows NT 4.0/Windows 98 platform.
  • a user interface thread configures the destination IP address, port number, data source (from serial communication port or from file) and the starting or stopping of the communication.
  • a second thread running as a background process, continually reads the serial communication port 2302 and sends the data to the UDP socket.
  • Figure 24 shows a representative screen shot 2400 of the initialization of the present preferred embodiment of the invention.
  • the destination IP Address 2402 is by default a loop back address so that the UDP packets are sent back to the same computer.
  • the data source port 2401 is defaulted to COM1 of the computer.
  • the destination port 2403 is set to 8888.
  • An exit button 2404, a start button 2405 and a stop button 2406 are provided for user control
  • Figure 25 shows a block diagram showing the preferred UML description of
  • VIZ of the invention.
  • the present VTS display program was developed using Visual C++ and OpenGL API under Microsoft Foundation Class (MFC) library application framework. This program is presently designed as a single thread application with one timer that will generate a time-out message every 100ms. During the response to the time-out message, the program reads data from the UDP socket, stores the data in a Standard Template Library (STL) vector, and passes a reference to the vector sot that all of the display component classes and their internal data representations are updated. Finally, the internal representations are refreshed on the computer screen.
  • STL Standard Template Library
  • VIZ was designed using an object oriented methodology. This figure shows the global structure of VIZ, represented by Unified Modeling Language (UML) [14].
  • UML Unified Modeling Language
  • Class CVizApp 2501 encapsulates the program entry point and has an instance of the CVizDlg class 2502.
  • Class CVizDlg 2502 provides the OpenGL rendering context, takes charge of the Windows message processing, and reads data from the UDP socket. The display screen is analyzed and structured into several components, which are then represented by several classes: CHeart 2504, CLung 2505, CDigital 2507 and CHistory 2506. These classes are all subclasses of the abstract class CDisplayObject 2503, whose virtual functions are called by the CVizDig class 2502.
  • CVizDig class 2502 has zero or more CDisplayObject 2503 pointers, which point to the instances of CDisplayObject subclasses 2503.
  • the CVizDig class 2502 calls the virtual functions of the CDisplayObject pointers, which are redirected, to the functions of the actual display component classes.
  • the adding or removing of display components causes very little change to the global program structure.
  • FIG. 26 shows a representative cardiovascular display of the present preferred embodiment of this invention.
  • the cardiovascular display 2600 is represented by a sphere 2601.
  • the width of the sphere is proportional to the time between consecutive heartbeats (R-R interval) 2602.
  • the height of the sphere 2601 is proportional to cardiac output, so the cross section of the sphere 2601 is proportional to the stroke volume.
  • the color of the sphere 2601 is coupled to arterial blood oxygen saturation (SaO2) as indicated by the palette, where high oxygen saturation is red and low oxygen saturation is typically displayed as blue. This is analogous to the color of saturated and desaturated blood.
  • SaO2 arterial blood oxygen saturation
  • the two ends of the vertical bar 2603 running through the heart icon represents the current systolic and diastolic aortic pressure.
  • the cage 2604 around the sphere 2601 shows the ideal state of the cardiac parameters and the background texture 2605 indicates the normal range of the blood pressure.
  • Figure 27 shows a representative respiration system display 2700 of the present preferred embodiment of this invention.
  • Functional Reserve Capacity FRC
  • FRC Functional Reserve Capacity
  • the gas concentrations in the lungs are mapped to the different colors on the bottom 2701 of the cylinder 2702.
  • the height 2703 and width 2704 of the top cylinder 2705 is proportional to the Tidal Volume (TV) and respiration period 2706 respectively.
  • the cage 2707 indicates the ideal tidal volume and respiration rate, which optimize the work done by the lungs.
  • the position of the cylinders is driven by airway pressure.
  • Figure 28 shows a representative history display 2800 of the present preferred embodiment of this invention.
  • the time scale 2801 is adjustable from 15 minutes to 2 hours.
  • Figures 29 and 30 shows a representative integrated global layout of the present preferred embodiment of this invention, showing a normal physiologic state and an anaphylaxis physiologic state respectively.
  • the objective of this display 2900 is to decrease the clinician's cognitive load by presenting information in a manner that complements the clinician's mental model.
  • the patient information is structured by physiological systems (mainly cardiac and respiratory systems).
  • the cardiac icon and pulmonary icon stand for their objects by virtue of resemblance and simplicity.
  • the alternative colors on the 3D curtain are used to indicate inspiration and expiration.
  • the respiration cycles are grouped with the airway pressure, which is shown on the top edge of the curtain.
  • the left column 2901 shows the short-term history of the parameters. Besides the three dimensional displays, features of traditional monitor displays have been maintained, including the digital display 2902 and the waveform display 2903.
  • Figure 29 shows the typical display when the subject is in a normal state and figure 30 shows the typical display when the subject is experiencing a serious incident.
  • Figure 31 shows a detailed flowchart showing the preferred process steps of the integrated BodylPC program of this invention.
  • the process starts 3101, following which the source of data is specified 3102.
  • the destination IP address and port number are specified 3103.
  • a socket for the UDP communication is created 3104. If the source of data is a file 3105, one row of data is read 3106 from the file. Data is sent out 3107 via UDP socket.
  • a test 3108 is made as to whether the user has specified a stop. If no stop is specified, the process repeats reading 3106 a row of data. If a stop is specified, the process ends 3114. If the source of the data is the serial port 3109, the COMl or COM2 port is opened 3110.
  • One packet of data is read 3111 from the COMl or COM2. Data is sent out 3112 via the UDP socket. A test 3113 is made as to whether the user has specified a stop. If no stop is specified, the process repeats reading 3111. If a stop is specified, the process ends 3114.
  • Figure 32 shows a detailed flowchart showing preferred process steps of the
  • VIZ program component of this invention starts 3201, following which C++ classes that represent different display components are created 3202.
  • a socket for UDP communication is created 3203.
  • the OpenGL render context is initialized 3204.
  • a 100ms time is set-up 3205 next.
  • a test 3206 is made to determine if a 100ms timeout has occurred. If the 100ms timeout has not timed-out, the process returns to the test 3206. If the 100ms timeout occurred, data is read 3207 from the UDP socket. Next, the data is transferred 3208 to all of the display components.
  • the display components update their data-shape mapping based on the new data. All of the display components are displayed 3209 and the process returns to the test 3206.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Surgery (AREA)
  • Primary Health Care (AREA)
  • Physics & Mathematics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Veterinary Medicine (AREA)
  • Molecular Biology (AREA)
  • Epidemiology (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • General Business, Economics & Management (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)
  • Digital Computer Display Output (AREA)

Abstract

La présente invention concerne un procédé, un système et un appareil pour la surveillance, le diagnostic et l'évaluation de l'état d'un système dynamique. Le procédé et le système fournissent les moyens de traitement pour la réception de données lues et/ou simulées, pour la conversion de telles données en format d'objets affichables et l'affichage de tels objets d'une façon propre à permettre que les interrelations ente les différentes variables puissent être corrélées et identifiées par un utilisateur. L'invention reçoit également et formate les données pour l'affichage traditionnel, numérique ou en forme d'ondes et intègre un tel affichage traditionnel dans la scène du format d'affichage de l'objet. Cette invention pourvoit à la prise en considération cognitive rapide de l'état d'ensemble d'une fonction critique par rapport à un système dynamique. Le système fait que pour les objets affichés, qui changent en temps réel montre les changements des fonctions du système. C'est un système hautement flexible qui fonctionne avec une grande variété d'applications (systèmes biologiques, environnementaux, d'ingénierie, économiques, mécaniques, chimiques, et analogue).
PCT/US2001/031657 2000-10-10 2001-10-10 Procédé et dispositif de surveillance de systèmes dynamiques par affichage graphique intégré de repésentations multidimensionnelles des fonctions critiques WO2002032035A2 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
AU2001296773A AU2001296773A1 (en) 2000-10-10 2001-10-10 Method and apparatus for monitoring dynamic systems using an integrated graphic display for the n-dimensional representations of critical functions

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US68920600A 2000-10-10 2000-10-10
US09/689,206 2000-10-10

Publications (2)

Publication Number Publication Date
WO2002032035A2 true WO2002032035A2 (fr) 2002-04-18
WO2002032035A3 WO2002032035A3 (fr) 2002-06-27

Family

ID=24767471

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2001/031657 WO2002032035A2 (fr) 2000-10-10 2001-10-10 Procédé et dispositif de surveillance de systèmes dynamiques par affichage graphique intégré de repésentations multidimensionnelles des fonctions critiques

Country Status (2)

Country Link
AU (1) AU2001296773A1 (fr)
WO (1) WO2002032035A2 (fr)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006050485A1 (fr) * 2004-11-02 2006-05-11 Lifescan, Inc. Procede et programme informatique d'analyse de modeles et d'etablissement de rapport de donnees de gestion d'etat pathologique chronique
WO2007142594A1 (fr) * 2006-06-02 2007-12-13 Gripping Heart Ab Système d'interface de machine d'états
WO2009072971A1 (fr) * 2007-12-03 2009-06-11 Inovacor Ab Système d'interface de validation et d'utilisateur de machine d'état

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5129722A (en) * 1991-05-16 1992-07-14 Tektronix, Inc. Expansion windowing system for a measurement test instrument
US5408603A (en) * 1992-03-31 1995-04-18 Dow Benelux N.V. Global process control information system and method
US5485600A (en) * 1992-11-09 1996-01-16 Virtual Prototypes, Inc. Computer modelling system and method for specifying the behavior of graphical operator interfaces
US5555366A (en) * 1991-07-18 1996-09-10 Mcc - Molecular Simulations Computer graphics system for selectively modelling molecules and investigating the chemical and physical properties thereof
US5602533A (en) * 1993-10-04 1997-02-11 Boverio; Antonello Device for sensing a state change in a mechanical system, method for monitoring the state of a mechanical system and use of said device
US5808918A (en) * 1995-04-14 1998-09-15 Medical Science Systems, Inc. Hierarchical biological modelling system and method
US5930154A (en) * 1995-01-17 1999-07-27 Intertech Ventures, Ltd. Computer-based system and methods for information storage, modeling and simulation of complex systems organized in discrete compartments in time and space
US6005548A (en) * 1996-08-14 1999-12-21 Latypov; Nurakhmed Nurislamovich Method for tracking and displaying user's spatial position and orientation, a method for representing virtual reality for a user, and systems of embodiment of such methods
US6154731A (en) * 1997-08-01 2000-11-28 Monks; Robert A. G. Computer assisted and/or implemented process and architecture for simulating, determining and/or ranking and/or indexing effective corporate governance using complexity theory and agency-based modeling
US6219046B1 (en) * 1998-09-17 2001-04-17 General Electric Company Man-machine interface for a virtual annunciator panel display
US6222547B1 (en) * 1997-02-07 2001-04-24 California Institute Of Technology Monitoring and analysis of data in cyberspace
US6222544B1 (en) * 1997-10-17 2001-04-24 Siemens Medical Systems, Inc. Graphical user interface for radiation therapy treatment apparatus
US6240372B1 (en) * 1997-11-14 2001-05-29 Arch Development Corporation System for surveillance of spectral signals

Patent Citations (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5129722A (en) * 1991-05-16 1992-07-14 Tektronix, Inc. Expansion windowing system for a measurement test instrument
US5555366A (en) * 1991-07-18 1996-09-10 Mcc - Molecular Simulations Computer graphics system for selectively modelling molecules and investigating the chemical and physical properties thereof
US5408603A (en) * 1992-03-31 1995-04-18 Dow Benelux N.V. Global process control information system and method
US5485600A (en) * 1992-11-09 1996-01-16 Virtual Prototypes, Inc. Computer modelling system and method for specifying the behavior of graphical operator interfaces
US5602533A (en) * 1993-10-04 1997-02-11 Boverio; Antonello Device for sensing a state change in a mechanical system, method for monitoring the state of a mechanical system and use of said device
US5930154A (en) * 1995-01-17 1999-07-27 Intertech Ventures, Ltd. Computer-based system and methods for information storage, modeling and simulation of complex systems organized in discrete compartments in time and space
US5808918A (en) * 1995-04-14 1998-09-15 Medical Science Systems, Inc. Hierarchical biological modelling system and method
US5808918C1 (en) * 1995-04-14 2002-06-25 Interleukin Genetics Inc Hierarchical biological modelling system and method
US6005548A (en) * 1996-08-14 1999-12-21 Latypov; Nurakhmed Nurislamovich Method for tracking and displaying user's spatial position and orientation, a method for representing virtual reality for a user, and systems of embodiment of such methods
US6222547B1 (en) * 1997-02-07 2001-04-24 California Institute Of Technology Monitoring and analysis of data in cyberspace
US6154731A (en) * 1997-08-01 2000-11-28 Monks; Robert A. G. Computer assisted and/or implemented process and architecture for simulating, determining and/or ranking and/or indexing effective corporate governance using complexity theory and agency-based modeling
US6222544B1 (en) * 1997-10-17 2001-04-24 Siemens Medical Systems, Inc. Graphical user interface for radiation therapy treatment apparatus
US6240372B1 (en) * 1997-11-14 2001-05-29 Arch Development Corporation System for surveillance of spectral signals
US6219046B1 (en) * 1998-09-17 2001-04-17 General Electric Company Man-machine interface for a virtual annunciator panel display

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006050485A1 (fr) * 2004-11-02 2006-05-11 Lifescan, Inc. Procede et programme informatique d'analyse de modeles et d'etablissement de rapport de donnees de gestion d'etat pathologique chronique
WO2007142594A1 (fr) * 2006-06-02 2007-12-13 Gripping Heart Ab Système d'interface de machine d'états
US8560057B2 (en) 2006-06-02 2013-10-15 Gripping Heart Ab State machine interface system
WO2009072971A1 (fr) * 2007-12-03 2009-06-11 Inovacor Ab Système d'interface de validation et d'utilisateur de machine d'état
US8943429B2 (en) 2007-12-03 2015-01-27 Gripping Heart Ab State machine user and validation interface system

Also Published As

Publication number Publication date
AU2001296773A1 (en) 2002-04-22
WO2002032035A3 (fr) 2002-06-27

Similar Documents

Publication Publication Date Title
US7603631B2 (en) Method and apparatus for monitoring dynamic systems using n-dimensional representations of critical functions
US7654966B2 (en) Method and apparatus for monitoring dynamic cardiovascular function using n-dimensional representatives of critical functions
US7413546B2 (en) Method and apparatus for monitoring dynamic cardiovascular function using n-dimensional representations of critical functions
US20080033661A1 (en) Method and apparatus for monitoring anesthesia drug dosages, concentrations, and effects using n-dimensional representations of critical functions
US7693697B2 (en) Anesthesia drug monitor
JP6883177B2 (ja) 解剖学的アイテムのコンピュータによる視覚化
US8131033B2 (en) Sensitivity lens for assessing uncertainty in image visualizations of data sets, related methods and computer products
US20130083032A1 (en) Biometric data display system and method
CN1685381B (zh) 用于产生一个虚拟解剖学环境的设备和方法
Sundstedt et al. A systematic review of visualization techniques and analysis tools for eye-tracking in 3D environments
CN116229793A (zh) 一种基于虚拟现实技术的培训考核系统
Ordóñez et al. An animated multivariate visualization for physiological and clinical data in the ICU
WO2002031642A1 (fr) Surveillance de la fonction cardiovasculaire dynamique par representation en n-dimensions
WO2002032035A2 (fr) Procédé et dispositif de surveillance de systèmes dynamiques par affichage graphique intégré de repésentations multidimensionnelles des fonctions critiques
Horn et al. Metaphor graphics to visualize ICU data over time
Bermudez et al. Data representation architecture. visualization design methods, theory and technology applied to anesthesiology
US20070211215A1 (en) Method for creating kaleidoscopic interfaces for use in biofeedback
Datey Experiments in the use of immersion for information visualization
US20220406017A1 (en) Health management system, and human body information display method and human body model generation method applied to same
US20240362866A1 (en) Health management system, and human body information display method and human body model generation method applied to same
Reis et al. Towards a virtual echocardiographic tutoring system
Hamza-Lup et al. Haptic user interfaces and practice-based learning for minimally invasive surgical training
Agutter et al. Architectural Potential of Data Representation: Modeling Physiological Processes
Novotny " Using Virtual Reality Effectively in Scientific Data Exploration-Perception, Usability and Design in Immersive Displays".
CN117910564A (zh) 一种用于虚拟培训考核的知识表示构建方法及系统

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

AK Designated states

Kind code of ref document: A3

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NO NZ PL PT RO RU SD SE SG SI SK SL TJ TM TR TT TZ UA UG UZ VN YU ZA ZW

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZW AM AZ BY KG KZ MD RU TJ TM AT BE CH CY DE DK ES FI FR GB GR IE IT LU MC NL PT SE TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
121 Ep: the epo has been informed by wipo that ep was designated in this application
REG Reference to national code

Ref country code: DE

Ref legal event code: 8642

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP