JP2016508049A - Measurement and enhancement in multi-modality medical imaging systems - Google Patents

Abstract

Systems and methods for multi-modality data processing are provided. Some embodiments relate to enhancing and measuring multi-modality medical data. In one embodiment, a method for enhancing medical data in a medical processing system includes receiving a reference set of medical data by the medical processing system. A reference set of medical data may be displayed on the first user display. A region identifier corresponding to the region to be highlighted is received with the highlight selection. A target set of data is identified based on the received region identifier and the received enhancement selection, and the medical processing system performs an enhancement corresponding to the enhancement selection on the target set of data. The enhanced target set of data may be displayed on a second user display, which is different from the first user display.

Description

  The present disclosure relates generally to the field of medical devices, and more particularly to data collection, manipulation, highlighting, display and annotation in multi-modality medical systems.

  Innovations in the diagnosis and validation of disease treatment success rates have shifted from external imaging processes to internal diagnostic processes. In particular, diagnostic devices and processes for diagnosing vascular occlusions and other vascular diseases with flexible elongated members such as catheters or microsensors placed on the tip of guidewires used in catheterization Has been developed. For example, known medical detection techniques include angiography, intravascular ultrasound (IVUS), forward vision IVUS (FL-IVUS), blood flow reserve ratio (FFR) identification method, coronary flow reserve (CFR) Specific methods, optical coherence tomography (OCT), transesophageal echocardiography, and image guided therapy. Each of these techniques may be well adapted for different medical situations. To increase the likelihood of successful treatment, a healthcare facility may have multiple imaging, treatment, diagnostic, and detection modalities at the hand of the catheter laboratory during the procedure. In recent years, processing systems have been designed that collect medical data from multiple different imaging, treatment, diagnostic and sensing tools and process multi-modality medical data. Such multi-component systems often include interdependent modules for information and system services.

  While existing multi-modality medical processing systems have proven useful, there remains a need for improvements in data handling and processing. For example, improvements to data identification have the potential to increase the ability to recognize, separate, index and classify related data. Improved methods and interfaces for presenting data collected in multiple modalities in a unified and coherent manner may allow an operator to draw accurate diagnostic conclusions. Further interface improvements may allow the operator to better refine, enhance and measure the data collected in the modality.

  Embodiments of the present disclosure provide an enhanced method and interface for data enhancement and measurement in a multi-modality processing system.

  In some embodiments, a method for enhancing medical data in a medical processing system is provided. The method includes receiving a reference set of medical data by a medical processing system. A region identifier corresponding to the region to be emphasized is received and an enhancement selection is also received. A target set of data is identified based on the received region identifier and the received enhancement selection, and the medical processing system applies an enhancement corresponding to the enhancement selection to the target set of data. The method also displays a reference set of medical data on a first user display and displays a targeted set of emphasized data on a second user display different from the first user display. And may be included.

  In some embodiments, a data measurement method in a medical processing system is provided. The method includes receiving a reference set of medical data by a medical processing system. A position marker associated with the reference set of medical data and the measurement selection is received, and a target set of data is identified based on the received reference set of medical data, the position marker, and the measurement selection. The medical processing system performs a measurement operation corresponding to the measurement selection on the target set of data to determine the measurement value. The reference set of medical data may correspond to a first modality, while the target set of data may include data corresponding to a second modality that is different from the first modality.

  In some embodiments, an apparatus is provided that includes a non-transitory computer readable storage medium that stores a plurality of instructions for execution by at least one computer processor. The apparatus displays a reference set of medical data on a first user display, receives user input for highlighting a portion of the reference set of medical data, and targets data based on the received user input Instructions are included for determining the set, highlighting a portion of the target set of data based on the received user input, and displaying the highlighted target set of data on the second user display. In various embodiments, the first and second user displays are the first and second display devices, the first and second portions of the display environment or desktop, and the display at the first and second time points. Correspond. The apparatus may include instructions for storing an emphasis transaction record of a portion of a target set of data and using the transaction record to emphasize a later data set based on previous emphasis.

  The systems and methods of the present disclosure perform on-demand enhancement and measurement of medical data in a multi-modality environment. In some embodiments, the operator may select a region or information to highlight to create a larger, clearer and more accurate view. This highlighted data and reference data may be displayed on separate parts of a single display or on separate display devices. This may be particularly useful in applications where user input may obscure some of the display, such as touch screen applications. An operator may combine data collected from multiple different modalities in a single view and thus present more information that is meaningful and easy to understand. In some embodiments, the selected enhancement leads to further data collection. In addition to operator instructions, the processing system may highlight and highlight the data of interest to attract the operator's attention. This processing system may help the operator to quickly locate critical structures and encourage the operator to develop conditions. The processing system may include an additional safety device for overload information. Some embodiments provide meaningful context for the rich data being displayed by allowing accurate measurements at the operator's request. Performing emphasis and measurement as needed ensures computing resources (computing resources), resulting in reduced system cost, size and power consumption, and improved system responsiveness Also good. Of course, it is understood that the above advantages are merely exemplary, and that no particular advantages are essential to any particular embodiment.

  Embodiments of the present disclosure provide an enhanced system and method for navigating a set of medical data in a multi-modality processing system.

  In some embodiments, a method for translating (interpreting) user input in a medical processing system is provided. The method includes receiving a reference set of medical data by a medical processing system, the medical data corresponding to a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS, and OCT. To do. The medical processing system also receives navigation commands. The highlighting and subset of the reference set of data is identified based on the navigation command. The medical processing system performs the selected enhancement of the subset and the highlighted subset is displayed. In one such embodiment, the navigation command specifies an additional set of medical data corresponding to a second modality that is different from the first modality. In such other embodiments, performing the enhancement includes performing a single axis zoom on the subset.

  In some embodiments, an apparatus is disclosed. The apparatus includes a non-transitory computer readable storage medium storing a plurality of instructions for execution by at least one computer processor, the plurality of instructions comprising a modality comprising FFR, iFR, pressure, flow, IVUS and OCT. Receiving a reference set of medical data corresponding to a first modality selected from the group of, receiving a navigation command, performing enhancement on a subset of the reference set identified by the navigation command, It is for giving an indication to a subset. In one such embodiment, the instruction for receiving the navigation command receives a user input sequence that indicates a first touch-based input that is concurrent with a second touch-based input, Contains instructions for translating into navigation commands. In the above embodiment, the navigation command specifies a first boundary of the subset corresponding to the first touch-based input and a second boundary of the subset corresponding to the second touch-based input.

  In some embodiments, a medical processing system is disclosed. The medical processing system includes a medical data interface configured to receive a reference set of medical data and a navigator engine. The navigator engine receives navigation commands from the controller, identifies enhancements to perform based on the navigation commands, identifies a subset of medical data reference sets to enhance based on the navigation commands, and is identified It is configured to perform the emphasis specified in the subset and provide a display for the emphasized subset. In one such embodiment, the navigator engine is further configured to provide updates for display of a reference set of medical data based on the navigation commands.

  The systems and methods of the present disclosure perform on-demand enhancement and measurement of medical data in a multi-modality environment. In some embodiments, the operator may select a region or information to highlight to create a larger, clearer and more accurate view. In some embodiments, a view that is enhanced by applying a single axis zoom to a subset of medical data is provided. Single axis zoom allows the data to be rescaled along the first axis without necessarily rescaling the data along the second axis. This is particularly useful for medical data sets that contain data values plotted over a range of time or distance, but without limitation, here the data values are periodic. In such data, the peak-to-peak extreme value along one axis may not change substantially as the scaling of the second axis is adjusted. Single axis zoom is just one enhancement, and by means of that single zoom, embodiments of the present disclosure provide an improved interface for navigating and manipulating data sets in a manner that corresponds to the nature of the underlying data. I will provide a. Of course, it is understood that the above advantages are merely exemplary, and that no particular advantages are essential to any particular embodiment.

  Embodiments of the present disclosure provide an enhanced method and interface for adaptive gesture recognition in a multi-modality processing system.

  In some embodiments, a method for translating user input in a medical processing system is provided. The method includes receiving a status indicator corresponding to an operating mode of the medical processing system, the operating mode including a value representing a modality selected from the group consisting of IVUS, OCT, pressure, and flow. A list of active commands is generated based on the received status indicator. A user input sequence is received from one or more user input devices. The medical processing system correlates the user input sequence and the commands in the list of active commands, which commands are used to control the operation of the system components. The list of active commands may include a subset of commands common to multiple modalities.

  In some embodiments, a method for controlling a medical processing system is provided. The method includes displaying medical data on a user display device, the set of medical data corresponding to the active modality of the medical processing system. The active modality is selected from a plurality of modalities of the medical processing system. The list of active commands is determined based on the active modality. The medical processing system receives a user input sequence from one or more user input devices. The user input sequence is adapted to a command selected from the list of active commands, and the selected command is used to control the operation of the components of the medical processing system.

  In some embodiments, an apparatus is provided that includes a non-transitory computer readable storage medium that stores a plurality of instructions for execution by at least one computer processor. The apparatus determines a medical processing system operating mode corresponding to a medical processing system modality, generates a list of active commands based on the operating mode, and generates a user input sequence from one or more user input devices. Instructions for correlating the user input sequence with commands in the list of active commands and using commands to control the behavior of the medical processing system. In one such embodiment, the list of active commands includes a subset of commands common to multiple modalities.

  Accordingly, the systems and methods of the present disclosure translate operator shortcuts, referred to as gestures in some embodiments, into commands for controlling the collection and manipulation of multi-modality medical data. Gestures enable multi-modality processing systems by allowing operators to select commands without GUI overhead, such as changing input devices, changing active windows or navigating complex menu structures Intuitive mechanism to control the speed quickly and accurately. The gesture based interface reduces GUI clutter by reducing unnecessary steps and replacing banks of command icons and deep menu trees. The natural flow of gestures results in less obstruction and interruption, especially in operational suits that may further improve procedure time and efficiency compared to navigating the GUI. Furthermore, some embodiments include a subset of commands that are common across different modalities. This commonality reduces the number of required gestures and further improves operator fluency and accuracy. Of course, it is understood that these advantages are merely exemplary and that certain advantages are not essential to any particular embodiment.

  Embodiments of the present disclosure provide an enhanced method and interface for data labeling in a multi-modality processing system.

  In some embodiments, a labeling method in a medical processing system is provided. The method includes receiving medical data that is labeled by a medical processing system. A first set of labels corresponding to the medical data is presented on the display device. A selection of a first label from the first set of labels is received from the interface device. The method also includes presenting a second set of labels corresponding to the medical data at the display device, where the second set of labels is based on the first label. A selection of a second label from the second set of labels is received from the interface device. The medical data is assigned a label based on the first and second labels.

  In some embodiments, an indexing method in a multi-modality medical processing system is provided. The method includes receiving, by a multi-modality medical processing system, medical data corresponding to a modality that is labeled. The set of labels is populated based on the modality. A set of labels is presented to the user, and user input is received based on the presented set of labels. A label is assigned to the medical data based on the received user input, and the medical data is stored such that the labeled medical data is indexed based on the assigned label.

  In some embodiments, a method for labeling data in a data processing system is provided. The method includes receiving medical data that is labeled by a data processing system. Procedure information is retrieved from the first database. A set of labels for medical data is retrieved from the second database, and the set of labels is based on procedural information. A set of labels is presented to the user at the user display device, and user input is received from the user interface based on the presented set of labels. The method also includes assigning a final label to the medical data based on the received user input.

  The systems and methods of the present disclosure provide a mechanism for selecting and assigning labels to medical data across multiple modalities. These labels allow the operator to annotate and index relevant patient data, thus improving data processing including retrieval and retrieval. For example, the assigned label may be used in determining which part of the medical data is stored, retained or archived. Labels allow data that is redundant or not critical to diagnosis to be easily identified. The assigned label may also be used to create a searchable index for quick and effective retrieval of important data. As such, by providing a list of standardized labels, some embodiments of the present disclosure recommend the use of standardized terms, while operators can use terms other than lists to cover special cases. It is also possible to select. This promotes searchability that is not unduly limited. In order to avoid overwhelming the operator, some embodiments refine the list according to previous selections, procedure information, patient information, other medical data and / or other criteria. This may allow the operator to recognize the most relevant labels, thereby improving labeling speed and accuracy. Of course, it is understood that the above advantages are merely exemplary, and that no particular advantages are essential to any particular embodiment.

  Further aspects, features and advantages of the present disclosure will become apparent from the following detailed description.

FIG. 1 is a schematic diagram illustrating a medical system including a multi-modality processing system according to some embodiments of the present disclosure. FIG. 2 is a functional block diagram of a portion of the medical system of FIG. 1, including a processing framework that executes in an embodiment of a multi-modality processing system. FIG. 3 is a functional block diagram of a portion of the medical system of FIGS. 1 and 2 including a user interface component that labels and / or indexes medical data according to some embodiments of the multi-modality processing system. . FIG. 4 is a diagram of an example user interface for label construction according to some embodiments of a multi-modality processing system. FIG. 5 is a flow diagram of a method for determining and attaching labels to medical data in a multi-modality processing system according to some embodiments of the present disclosure. FIG. 6 is a flow diagram of a method for handling an interruption to a labeling procedure in a multi-modality processing system according to some embodiments of the present disclosure. FIG. 7 is a functional block diagram of a portion of the medical system of FIGS. 1 and 2, including user interface components that perform on-demand data enhancement of medical data according to some embodiments of a multi-modality processing system. FIG. 8 is a diagram of an exemplary user interface for on-demand data enhancement according to some embodiments of a multi-modality processing system. FIG. 9 is a diagram of an example user interface for data measurement according to some embodiments of a multi-modality processing system. FIG. 10 is a flow diagram of an on-demand data enhancement method in a multi-modality processing system according to some embodiments of the present disclosure. FIG. 11 is a flow diagram of a data measurement method in a multi-modality processing system according to some embodiments of the present disclosure. FIG. 12 is a functional block diagram of a portion of the medical system of FIGS. 1 and 2 including a user interface component that distinguishes user input according to some embodiments of a multi-modality processing system. FIG. 13 is a diagram of an exemplary user interface displaying a gesture-based user input sequence according to some embodiments of a multi-modality processing system. FIG. 14 is a diagram of an exemplary user interface displaying a gesture-based user input sequence according to some embodiments of a multi-modality processing system. FIG. 15 is a diagram of an exemplary user interface displaying translated instructions according to some embodiments of a multi-modality processing system. FIG. 16 is a diagram of an exemplary user interface displaying a gesture-based user input sequence according to some embodiments of a multi-modality processing system. FIG. 17 is a diagram of an exemplary user interface displaying translated instructions according to some embodiments of a multi-modality processing system. FIG. 18 is a flow diagram of a gesture recognition method in a multi-modality processing system according to some embodiments of the present disclosure. FIG. 19 is a functional block diagram of a portion of the medical system of FIGS. 1 and 2 including user interface components for navigating a set of medical data according to some embodiments of the multi-modality processing system. FIG. 20 is a diagram of an exemplary user interface for data navigation and enhancement according to some embodiments of a multi-modality processing system. FIG. 21 is a flow diagram of a method for navigating and highlighting a set of medical data in a multi-modality processing system according to some embodiments of the present disclosure.

  For the purposes of promoting an understanding of the principles of the disclosure, reference is made to the embodiments illustrated in the drawings and specific language is used to describe the embodiments. Nevertheless, it is understood that no limitation to the scope of the present disclosure is intended. Any alternatives and further variations on the described apparatus, systems and methods, as well as any further applications of the principles of the present disclosure, are well contemplated, such applications would normally be envisioned by those skilled in the art to which this disclosure is related. Included within the scope of this disclosure. In particular, it is fully contemplated that features, components and / or steps described with respect to one embodiment may be combined with features, components and / or steps described with respect to other embodiments of the invention. . However, for the sake of brevity, many repetitions of these combinations are not described separately.

  FIG. 1 is a schematic diagram illustrating a medical system 100 including a multi-modality processing system 101 according to an embodiment of the present disclosure. In general, the medical system 100 acquires and is designed to be sensitive to various methods used to acquire and translate human biological physiology and morphological information and coordinate processing of various conditions. And coherent integration and integration of multiple forms of processing elements. More specifically, in system 100, multimodality processing system 101 is an integrated device for acquisition, control, translation and display of multimodality medical sensing data. In one embodiment, the processing system 101 is a computer system with hardware and software that acquires, processes, and displays multi-modality medical data, while in other embodiments, the processing system 101 processes medical data. Any other type of computing system that can be operated as such. In embodiments where the processing system 101 is a computer workstation, the system includes at least a processor, such as a microcontroller or dedicated central processing unit (CPU), and a hard drive, random access memory (RAM) and / or compact. A non-transitory computer readable storage medium such as a disk read only memory (CD-ROM), a video controller such as a graphics processing unit (GPU), an Ethernet controller or a wireless communication controller, etc. Network communication device. In that regard, in some specific embodiments, the processing system 101 is programmed to perform the steps associated with data acquisition and analysis described herein. Thus, any steps related to data acquisition, data processing, instrument control, and / or other processing or control aspects of the present disclosure are stored in a non-transitory computer readable storage medium accessible by the processing system. May be implemented by the processing system using instructions to In some embodiments, the processing system 101 is portable (eg, portable, on a rolling cart). Further, it is understood that in some embodiments, the processing system 101 includes a plurality of computing devices (computing devices). In that regard, it is specifically understood that the different processing and / or control aspects of the present disclosure may be implemented separately or within a predetermined group using multiple computing devices. Any division and / or combination of the processing and / or control aspects described below across multiple computing devices is within the scope of this disclosure.

  In the illustrated embodiment, the medical system 100 is placed in a catheter laboratory 102 having a management room 104 with a processing system 101 located in the management room. In other embodiments, the processing system 101 may be located at other locations, such as a catheter laboratory 102, a centralized area within a medical facility, or an off-site location (ie, within a cloud). The catheter laboratory 102 generally includes a sterilization area that surrounds the procedural area, but its associated management room 104 may or may not be sterilizable, depending on certain procedures and / or medical facility requirements. . Catheter laboratory and management room include angiography, intravascular ultrasound (IVUS), virtual history (VH), forward view IVUS (FL-IVUS), intravascular photoacoustic (IVPA) imaging, blood flow reserve Ratio (FFR) identification method, coronary flow reserve capacity (CFR) identification method, optical coherence tomography (OCT), computed tomography, intracardiac echocardiography (ICE), forward vision ICE (FLICE), intravascular It may be used to perform any number of medical sensing procedures on the patient, such as pulpography (Intravascular Palography), transesophageal echocardiography, or any other medical sensing modality known in the art. In addition, the catheter lab and management room may include one or more surgeries or radiofrequency ablation therapy (RFA), cryotherapy, atherectomy, or any other medical surgical procedure known in the art. It may be used to perform a treatment procedure on a patient. For example, in the catheter laboratory 102, a patient 106 may perform a multi-modality procedure as a single procedure or in combination with one or more sensing procedures. In any case, the catheter laboratory 102 includes a plurality of medical devices that include medical sensing devices that may collect medical sensing data from a patient 106 in a variety of different medical sensing modalities.

  In the embodiment shown in FIG. 1, the devices 108 and 110 are medical sensing devices that may be used by a clinician to obtain medical sensing data about the patient 106. In certain embodiments, device 108 collects medical sensing data in one modality and device 110 collects medical sensing data in different modalities. For example, each device may include pressure, flow (velocity), image (including images obtained using ultrasound (eg, IVUS), OCT, heat and / or other imaging techniques), temperature, and / or their One of the combinations may be collected. Devices 108 and 110 may be any type of device, instrument or probe that is positioned within a blood vessel, attached to the exterior of a patient, or sized and shaped to scan a patient from a distance. There may be.

  In the embodiment shown in FIG. 1, instrument 108 is an IVUS catheter 108 that may include one or more sensors, such as phased array transducers, to collect IVUS sensing data. In some embodiments, IVUS catheter 108 may allow multi-modality sensing such as IVUS and IVPS sensing. Further, in the illustrated embodiment, device 110 is an OCT catheter 110 that may include one or more optical sensors configured to collect OCT sensing data. In some embodiments, IVUS patient interface module (PIM) 112 and OCT PIM 114 couple IVUS catheter 108 and OCT catheter 110 to medical system 100, respectively. In particular, IVUS PIM 112 and OCT PIM 114 are operable to receive medical sensing data collected from patient 106 by IVUS catheter 108 and OCT catheter 110, respectively, and data received by processing system 101 of management room 104. Is operable to transmit. In one embodiment, PIMs 112 and 114 include analog to digital (A / D) converters that transmit digital data to processing system 101, while in other embodiments, PIM transmits analog data to the processing system. In one embodiment, IVUS PIM 112 and OCT PIM 114 transmit medical sensing data on a Peripheral Component Interconnect Express (PCIe) data bus connection, while in other embodiments they are USB connected, Thunderbolt. Data may be transmitted in a (registered trademark) connection, a FireWire (registered trademark) connection, or some other high-speed data bus connection. In other embodiments, the PIM is connected to the processing system 101 via a wireless connection using the IEEE 802.11 Wi-Fi standard, Ultra Wideband (UWB) standard, wireless fire wire, wireless USB, or other high speed wireless networking standards. May be.

  Further, in the medical system 100, an electrocardiogram (ECG) device 116 is operable to transmit an electrocardiogram signal or other hemodynamic data from the patient 106 to the processing system 101. In some embodiments, the processing system 101 is operable to synchronize the data collected by the catheters 108 and 110 using the ECG signal from the ECG 116. In addition, angiography system 117 is operable to collect x-rays, computed tomography (CT) or magnetic resonance imaging (MRI) of patient 106 and transmit them to processing system 101. In one embodiment, the angiography system 117 can be communicatively coupled to the processing system 101 via an adapter device. Such an adapter device may convert the data from a proprietary third party format into a format that can be used by the processing system 101. In some embodiments, the processing system 101 uses image data (eg, X-ray data, MRI data, CT data, etc.) from the angiography system 117 to sense and co-register the sensing data from the IVUS catheter 108 and the OCT catheter 110. It may be operable to co-register. As one feature of this, co-registration may be performed to generate a three-dimensional image using the sensing data.

  Bedside controller 118 is also communicatively coupled to processing system 101 and provides user control of the particular medical modality (s) used to diagnose patient 106. In this embodiment, the bedside controller 118 is a touch screen controller that provides user control and medical images on a single surface. However, in alternative embodiments, the bedside controller 118 may include both a non-interactive display and separate controls such as physical buttons and / or joysticks. In the integrated medical system 100, the bedside controller 118 is operable to present workflow control options and patient image data in a graphical user interface (GUI). As described in detail with respect to FIG. 2, the bedside controller 118 includes user interface (UI) framework services that may be executed by workflows associated with multiple modalities. Thus, the bedside controller 118 can display workflow and medical images for multiple modalities that allow clinicians to control the acquisition of multi-modality medical sensing data with a single interface device It is.

  The main controller 120 of the management room 104 is also communicatively coupled to the processing system 101 and is adjacent to the catheter laboratory 102 as shown in FIG. In this embodiment, the main controller 120 includes a touch screen and displays a number of GUI-based workflows corresponding to different medical sensing modalities via a UI framework service running on the touch screen. Similar to the bedside controller 118 in that it is operable. In some embodiments, the main controller 120 may be used to simultaneously execute different workflow workflow aspects than the bedside controller 118. In alternative embodiments, the main controller 120 may include non-interactive displays and stand-alone controls such as a mouse, keyboard, and the like.

  The medical system 100 further includes a boom display 122 communicatively coupled to the processing system 101. Boom display 122 may include an array of monitors, each of which is capable of displaying different information related to a medical sensing procedure. For example, during the IVUS procedure, one monitor of the boom display 122 may display a tomogram and one monitor may display a sagittal image.

  Further, multi-modality processing system 101 is communicatively coupled to data network 125. In the illustrated embodiment, the data network 125 is a TCP / IP-based local area network (LAN); however, in other embodiments, the data network 125 is Synchronous Optical Networking (SONET). Different protocols may be used, or a wide area network (WAN). The processing system 101 may connect to various resources via the network 125. For example, the processing system 101 may communicate with a Digital Imaging and Communications in Medicine (DICOM) system 126, a Picture Archiving and Communication System (PACS) 127 and a Hospital Information System (HIS) 128 via the network 125. Further, in some embodiments, the network console 130 may allow the physician or other medical personnel to remotely access aspects of the medical system 100 over the network 125 to enable the multi-modality processing system 101. You may communicate with. For example, a user of the network console 130 may access patient medical data such as medical images collected by the multi-modality processing system 101 or, in some embodiments, one in the catheter laboratory 102 in real time. One or more continuation procedures may be monitored or controlled. The network console 130 may be any type of computing device with a network connection, such as a PC, laptop, smartphone, tablet computer or other such device located inside or outside the healthcare facility. .

  Further, in the illustrated embodiment, the medical sensing tool in system 100 described above is shown as being communicatively coupled to processing system 101 via a wired connection, such as a standard copper link or fiber optic link. However, in alternative embodiments, the tool is via a wireless connection using the IEEE 802.11 Wi-Fi standard, ultra wideband (UWB) standard, wireless fire wire, wireless USB, or other high speed wireless networking standards. It may be connected to the processing system 101.

  Those skilled in the art will recognize that the medical system 100 described above is merely an exemplary embodiment of a system that is operable to collect medical data associated with multiple medical modalities. In alternative embodiments, different and / or additional tools may be communicatively coupled to the processing system 101 to provide additional and / or different functionality to the medical system 100.

  Referring now to FIG. 2, a functional block diagram of a portion of the medical system 100 of FIG. 1, including a processing framework 200 that executes in an embodiment of the multi-modality processing system 101 is shown. The processing framework 200 includes various independent and non-independent executable components that control the operation of the processing system 101, including acquisition, processing and display of multi-modality medical sensing data. In general, the processing framework 200 of the processing system 101 is modular and extensible. That is, the framework 200 consists of independent software and / or hardware components (or extensions) that are each associated with different functions and medical sensing modalities. This modular design extends the framework to accommodate additional medical sensing modalities and functionality without impacting existing functionality or requiring changes to the underlying architecture. Allows to be done. Furthermore, the internal messaging system facilitates independent data communication between modules within the framework. In one example, the processing framework 200 may be implemented as computer-executable instructions stored on a non-transitory computer-readable storage medium in the processing system 10. In other embodiments, the processing framework 200 may be a combination of hardware and software modules executing within the processing system 101.

  In general, in the embodiment shown in FIG. 2, the processing framework 200 receives medical sensing data from a plurality of medical sensing devices, processes the data, and performs a main controller 120, bedside controller 118, or other graphical representation. A plurality of components configured to output data as medical images via the display device; Framework 200 includes a plurality of system level components that manage the core system functions of processing system 101 and coordinate a plurality of modality specific components. For example, the framework 200 coordinates the startup (startup) and shutdown (shutdown) of multiple executable components of the processing framework 200 including hardware and software modules related to the acquisition and processing of patient care data. A system controller 202 is included. The system controller 202 may also be configured to monitor the status of components executing within the framework 202, for example, to determine whether any component has unexpectedly stopped execution. In addition, the system controller 202 provides an interface through which other framework components may obtain system configuration and state information. Because the software framework 200 is modular, the system controller 202 is independent of the components within the framework that manage such that errors and changes that occur in the components do not affect the execution or structure of the system controller. ing.

  As described above, the framework 200 is configured such that various extensions may be added and removed without changing the system architecture. In certain embodiments, an extension executing within framework 200 may include a plurality of executable components that together perform the full functionality of the extension. In such embodiments, the expansion may include an expansion controller similar to the system controller 202 that is operable to launch, deactivate and monitor various executable components associated with the expansion. For example, at system startup, the system controller 202 may initiate an expansion controller that corresponds to the medical modality, and then the expansion controller may further initiate executable components associated with the modality. . In one embodiment, extended controllers may not be assigned until system controller 202 associates them with a particular modality or other system task via parameters retrieved from a configuration mechanism such as a configuration file. .

  The processing framework 200 further includes a workflow controller component 204 that is generally configured to govern the execution of executable components of the framework 202 during a multi-modality medical sensing workflow. The workflow controller component 204 may dominate the workflow executed by the processing framework 200 in a variety of different ways.

  The processing framework 200 further includes an event logging component 206 configured for log messages received from various components of the processing framework. For example, during system startup, the system controller 202 may send a message about the status of the component that is initiated to the event logging component 206 that writes more messages to a log file in a standardized format. Further, the processing framework 200 is configured to manage limited system resource sharing among various executable components of the framework 202 during multi-modality medical sensing and / or processing workflows. An arbiter component 208 is included. For example, during a multi-modality workflow, two or more components associated with different modalities within the processing framework 202 may compete for the same system resources, such as a graphical display on the main controller 120. . The resource arbiter component 208 may coordinate limited system resource sharing in various ways, such as a lock system, a queue system, or a hierarchical collision management system.

  In one embodiment, the system controller 202, workflow controller component 204, event logging component 206, and resource arbiter component 208 are executed as processor executable software stored on a non-transitory computer readable storage medium. However, in alternative embodiments, these components may include dedicated microprocessors, field programmable gate arrays (FPGAs), microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), etc. It may be implemented as a hardware component. Alternatively, the components of the processing framework may be implemented as a combination of hardware and software. In certain embodiments where the executable components are implemented in the FPGA, the system controller 202 is configured to dynamically change the programmable logic within the FPGA to perform the various functions required at that time. May be. As an aspect of this, the processing system 101 may include one or more unassigned FPGAs that may be assigned by the system controller during system startup. For example, if the system controller detects an OCT PIM and a catheter coupled thereto upon processing system 101 startup, the system controller or expansion controller associated with the OCT function may receive and / or process OCT medical data. Programmable logic may be dynamically translated within one of the unassigned FPGAs to include the function to

  The processing framework 200 further includes a message delivery component 210 to facilitate intersystem communication between different hardware and software components in the multi-modality processing system 101. In one embodiment, message delivery component 210 receives a message from a component within framework 202, determines the intended purpose of the message, and delivers the message in a timely manner (ie, the message delivery component Active participation in distribution). In such embodiments, the message metadata is generated by a sending component that includes destination information, payload data (eg, modality type, patient data, etc.), priority information, timing information, or other such information. May be. In other embodiments, the message delivery component 210 receives a message from a component in the framework 202, temporarily stores the message, and makes the message available for retrieval by other components in the framework. (Ie, the message delivery component is a passive queue). In any case, the message delivery component 210 facilitates communication between executable components in the framework 200. For example, the system controller 202 may use the message delivery component 210 to examine the status of the component that starts during the system startup sequence, and subsequently writes to the log file when receiving status information. As may be the case, the message delivery component may be used to send status information to the event logging component 206. Similarly, the resource arbiter component 208 may use the message delivery component 210 to pass a resource token between components that require access to limited resources.

  In an exemplary embodiment where the message delivery component 210 is a passive queue, components in the framework 200 packetize incoming medical sensing data into a message, and then place the message in the message delivery component's queue. May be sent, where the message is retrieved by other components, such as an image data processing component. Further, in some embodiments, the message delivery component 210 is operable to make messages received in a first-in-first-out (FIFO) method available, and the message that reaches the queue first is , Will be removed from the queue first. In alternative embodiments, the message delivery component 210 may make the message available in different ways, for example by a priority value stored in the message header. In one embodiment, the message delivery component 210 is implemented in random access memory (RAM) in the processing system 101, but in other embodiments it is non-volatile RAM (NVRAM), secondary storage ( For example, it may be implemented in a magnetic hard drive, flash memory, etc.) or a network-based storage device. Further, in one embodiment, messages stored in message delivery component 210 may be accessed by software and hardware modules in processing system 101 using direct memory access (DMA).

  The processing framework 202 further includes a plurality of additional system components with core system functionality including a security component 212, a multi-modality case management (MMCM) component 214 and a database management component 216. In certain embodiments, the security component 212 is configured to provide various security services in the overall processing framework or in individual components. For example, a component that implements an IVUS data acquisition workflow may use an encrypted application programming interface (API) exposed by a security component 212 that encrypts IVUS data before being sent over a network connection. . In addition, the security component 212 restricts access to the processing framework for authenticated users and prevents other security features such as system level authentication and authorization services that prevent the extension of untrusted components within the extensible framework. A service may be provided. A multi-modality case management (MMCM) component 214 is configured to coordinate and consolidate medical data associated with multiple medical modalities into an integrated patient record that may be more easily managed. Such integrated patient records may be stored more efficiently in the database and may be more flexible by archiving and retrieving data. In that regard, the database management component 216 is configured to present a transparent database service to other components in the framework 200 so that database connection and management details are hidden from the other components. For example, in certain embodiments, the database management component 216 may expose an API that includes database storage and search functionality to components of the framework 200. In other words, the medical sensing workflow component can send clinical data to a local and / or remote database such as a DICOM or PACS server through the database component without knowing the details of the database connection It may be. In other embodiments, the database management component 216 is operable to perform additional and / or different database services, such as a data formatting service that creates medical data for database archiving. Also good.

  As described above, the processing framework 200 of the multi-modality processing system 101 is operable to receive and process medical data related to multiple modalities. In that regard, the processing framework 200 includes a plurality of modular acquisition components and workflow components that are each associated with different medical sensing and medical modalities. For example, as shown in the exemplary embodiment of FIG. 2, the processing framework 200 is configured to receive and process IVUS medical sensing data from the IVUS PIM 112, respectively, and an IVUS acquisition component 220 and an IVUS workflow component. 222 is included. Modality “N” acquisition component 224 and modality that any number of additional acquisition and workflow components acquire and process data from modality “N” PIM 228 according to the modular and extensible nature of processing framework 200. It may be added independently to the framework represented by the “N” workflow component 226. For example, in certain embodiments, the processing system 101 may be communicatively coupled to an OCT PIM 114, an ECG system 116, a blood flow reserve ratio (FFR) PIM, a FLIVUS PIM, and an ICE PIM. In other embodiments, additional and / or different medical sensing devices, processing devices or medical devices may be coupled to the processing system 101 via additional and / or different data communication connections known in the art. . In such a scenario, in addition to the IVUS acquisition module 220, the processing framework 200 includes an FFR acquisition component that receives FFR data from the FFR PIM, a FLIVUS acquisition component that receives FLIVUS data from the FLIVUS PIM, and an ICE PIM to ICE. An ICE acquisition component that receives data and an OCT acquisition component that is operable to receive OCT data from an OCT PIM. In this regard, medical data communicated between medical devices (eg, PIMs, catheters, etc.) that are communicatively coupled to the executable components of the processing framework 200 is the data collected by the sensors, control It may also include signals, power levels, device feedback, and other medical data related to sensing, processing or diagnostic procedures. Further, in certain embodiments, the patient treatment device includes a radiofrequency ablation therapy (RFA), a device associated with cryotherapy or atherectomy, a PIM or other control device associated with such treatment procedure, etc. The processing system 101 may be communicatively coupled. In such an embodiment, the modality “N” acquisition component 224 and the modality “N” workflow component 226 relay control signals, relay power levels, receive device feedback, and sensors included in the therapy device. May be configured to communicate with and control the therapy device, such as by receiving data collected by the.

  In one embodiment, when data is received from a medical sensing device to which acquisition components 220 and 224 are connected, this component packetizes the data into messages to facilitate intersystem communication. Specifically, this component may be operable to generate multiple messages from an incoming digital data stream, each message being part of the digitized medical sensing data. And header. The message header includes metadata related to medical sensing data included in the message. Further, in some embodiments, acquisition components 220 and 224 were operable to manipulate medical sensing data digitized in some manner before being sent to other parts of framework 200. May be. For example, the acquisition component may compress the sensing data to make intersystem communication more efficient, normalize, scale, or filter the data to assist in later processing of the data. In some embodiments, this operation may be modality specific. For example, IVUS acquisition component 220 may identify and discard redundant IVUS data before it is passed to save processing time in a later step. Acquisition components 220 and 224 detect which medical sensing device is connected to processing system 101 in response to an interruption caused by a data bus (eg, PCIe, USB) and are connected to the medical sensing device. A plurality of tasks related to data acquisition may be further performed, including retrieving information about, storing sensing device specific data, and allocating resources to the data bus. As mentioned above, the data acquisition components are independent of each other and may be installed or removed without disturbing data acquisition by other components. Further, the acquisition component is independent of the underlying data bus software layer (eg, using an API) and thus facilitates acquisition of data from third party medical sensing devices. It may be generated by a third party.

  Processing frame workflow components, such as IVUS workflow component 222, receive raw medical sensing and / or medical data from their respective acquisition components via message delivery component 210. In general, workflow components start and stop data collection at calculated times, display acquired and processed patient data, and facilitate the analysis of patient data acquired by clinicians, etc. Configured to control data acquisition. In this aspect, the workflow component is operable to convert raw medical data collected from the patient into a clinical image or other data format that allows the clinician to assess the patient's condition. It is. For example, the IVUS workflow component 222 may translate the IVUS data received from the IVUS PIM 112 and convert this data into a human readable IVUS image. In one embodiment, the software stack in the framework exposes a set of APIs that allow the workflow component 222 and other workflow components in the framework to compute resources, message delivery. It may be invoked to access component 210 and system resources such as communication resources. After processing the acquired data, the modality-centric workflow component may send one or more messages containing the processed data to other components in the framework 200 via the message delivery component 210. In some embodiments, before sending such a message, those components may insert a flag in the header indicating that the message contains processed data. Further, in some embodiments, after processing the medical sensing data, those components can process the processed data into a locally attached mass storage device or an archiving system such as a network-based PACS server 127. The database management component 216 may be used to transmit. In accordance with the modular architecture of the processing framework 200, the workflow components 222 and 226 are independent of each other, may be installed or removed without disturbing other components, and may be written by a third party. Furthermore, because of their independence, they may be operable to process signal data and image data from multiple medical sensing devices simultaneously.

  The processing framework 200 acquires data from any number of data collection tools 234, processes them, and co-registers the acquired data with data acquired by one of the other acquisition components in the framework. It further includes a co-registration interface component 230 and a co-registration workflow component 232 that are configured to do so. More specifically, the co-registration interface component 230 operates to communicatively interface with a medical data acquisition tool associated with any number of modalities, such as the ECG device 116 or angiography system 117 of FIG. It may be possible. In certain embodiments, interface component 230 is adapted to standardize and / or transform incoming modality data so that it may be co-registered with other sensing data obtained by processing system 101. It may be operable. Because the medical data has been acquired by the co-registration interface component 230, the co-registration workflow component 232 synchronizes the data collection between the medical sensing devices spatially or temporally, spatially or temporally. Co-registered clinical images or other human-readable data that aligns two or more acquired data sets based on the statistical registration markers and allows the clinician to assess the patient's condition It is configured to facilitate the co-registration of data from different modalities, such as by generating. Further, in other embodiments, the co-registration workflow component 232 uses a previously generated two-dimensional (2D) image or three-dimensional (3D) model to spatially capture catheter collection data in 2D or 3D space. May be operable to co-register. For example, a catheter-based sensing tool may include criteria that are tracked to generate location data during the sensing procedure, and the co-registration workflow component 232 may be based on previously acquired MRI data. This position data may be registered. In addition, the co-registration workflow component 232 facilitates the co-registration of multi-modality data acquired by native acquisition components within the framework 200, such as the IVUS acquisition component 220 and the modality “N” acquisition component 224. May be. Further, in some embodiments, a real time clock may be integrated into the co-registration workflow component 232. In US Provisional Application No. 61 / 473,591, the title of the invention is “DISTRIBUTED MEDICAL SENSING SYSTEM AND METHOD”, the time-sequential medical sensing data collection is disclosed in detail. Is included herein by reference.

  As described above with respect to FIG. 1, a clinician using the processing system 101 may control the workflow via the main controller 120 and the bedside controller 118 to view medical images. Main controller 120 and bedside controller 118 include user interface (UI) framework services 240 and 242 that support a plurality of user interface (UI) extensions (or components), respectively. In general, UI extensions supported by UI framework services 240 and 242 correspond to medical sensing modalities, respectively, and provide a user interface for controlling the associated acquisition workflow and display of processed sensing data. It is operable to render. Similar to processing framework 200, UI frameworks 240 and 242 are extensible in that they support UI extensions that are independent of each other. That is, the modular design allows UI frameworks 240 and 242 to add additional medical sensing modalities without affecting or requiring changes to the existing user interface relative to the underlying UI architecture. Allows to be extended to accommodate a user interface. In the illustrated embodiment, the main controller 120 includes a system UI extension 244 that renders a user interface including core system controls and configuration options. For example, the clinician may launch, shut down, or manage the processing system 101 using a user interface rendered by the system UI extension 244. In one embodiment, the main controller 120 component may consider a portion of the processing framework 200. IVUS UI extensions 246 and 248 render the user interface for main controller 120 and bedside controller 118, respectively. For example, the IVUS UI extensions 246 and 248 may render and display touch screen buttons that are used to control the IVUS workflow, or view the IVUS medical image generated by the IVUS workflow component 222. It may be rendered and displayed. Similarly, modality “N” UI extensions 250 and 252 render controls and images associated with the modality “N” workflow.

  In one embodiment, UI framework services 240 and 242 expose an API that a UI extension may call to access system resources such as look and feel toolboxes and error handling resources. Also good. The look and feel toolbox API allows UI extensions to present a standardized user interface with common buttons, parallel workflow formats and data presentation schemes for different modality workflows. In this way, the clinician may more easily switch between acquisition modalities without additional user interface training. Further, the co-registration UI extension may present and / or combine image or signal data processed from multiple modalities. For example, the UI extension may display an electrocardiogram (ECG) waveform adjacent to the IVUS image data, or may display the IVUS image superimposed on a previously drawn boundary on the OCT image. Good. Further, in some embodiments, UI framework services 240 and 242 may include a multitasking framework to simultaneously adjust existing UI extensions. For example, if the processing system 101 is acquiring data related to more than one modality simultaneously, the UI framework services 240 and 242 may present a modality selection screen to the user, on which The desired user interface may be selected.

  The UI framework service 240 communicates with the components of the processing framework 200 via the message delivery component 210. As shown in the exemplary embodiment of FIG. 2, the bedside controller 118 may be communicatively coupled to the processing framework 200 via a network connection 254. Network connection 254 may be any type of wired or wireless network connection, such as an Ethernet connection or an IEEE 802.11 WiFi connection. Alternatively, one or both of main controller 120 and bedside controller 118 may be local (PCIe) such as a data bus connection, USB connection, thunderbolt connection, fire wire connection, or some other high speed data bus connection. -It may communicate with the processing framework 200 via a bus connection. Further, in the exemplary embodiment of FIG. 2, the bedside controller is configured to facilitate message-based communication between the UI extension in bedside controller 118 and the components in processing framework 200. Message delivery component 256. In certain embodiments, the message delivery component 256 may extract medical image data from the network communication packet in order for the medical image data to reach the network connection 254.

  The processing framework 200 includes additional components that allow the clinician to access and / or control the workflow that runs on the multi-modality processing system 101. For example, the framework 200 includes a remote access component 260 that communicatively couples the network console 130 (FIG. 1) to the processing framework 200. In one embodiment, the remote access component 260 is operable to export the processing system 101 control functions to the network console 130, and thus the network console presents workflow control functions in the user interface. May be. In certain embodiments, remote access component 260 may receive a workflow command from network console 130 and forward it to remote access workflow component 262. The remote access workflow component 262 may dominate a set of commands and medical data, which may be accessed by the remote user via the network console 130. In addition, legacy control component 264 and legacy control workflow component 266 provide a level of access to modality workflow control and data to users of legacy console 268 (eg, button console, mouse, keyboard, standalone monitor). provide.

  In one embodiment, additional components such as core system components and modality related components of processing framework 200 may be implemented as processor executable software stored on non-transitory computer readable storage media, In embodiments, these components are implemented as hardware components such as dedicated microprocessors, Field Programmable Gate Arrays (FPGAs), microcontrollers, graphics processing units (GPUs), digital signal processors (DSPs), etc. May be. Alternatively, the components of the processing framework may be implemented as a combination of hardware and software.

  Those skilled in the art will appreciate that the processing framework 200 of FIG. 2 is merely an exemplary embodiment, and in alternative embodiments, the framework may be configured to perform various medical sensing workflows, different and / or Additional components may be included. For example, the processing framework 200 may further include an executable component configured for assessment of human blood vessel stenosis or configured to facilitate control of computer-assisted surgery or remote-controlled surgery.

  Referring now to FIG. 3, the medical system of FIGS. 1 and 2 that includes a user interface component 300 that labels and / or indexes medical data according to some embodiments of the multi-modality processing system 100. Some functional block diagrams are shown. In various embodiments, the user interface component 300 presents a list of nested indexes or labels that are used to identify medical data. The amount of data collected during surgery or treatment can be quite large. Adding multiple modalities only exacerbates the problem. The labeling and / or indexing related portion of the data set facilitates review, searching, auditing, statistical analysis and other uses of the data.

  The user interface component 300 includes a label builder module 302 that constructs an associated label based on user input and applies the label to a portion of medical data. The label builder module 302 includes one or more of a label builder engine 306, a medical data interface 308, a label set database 310 and a procedure database 312. The procedural database 312 includes information related to the procedure being performed, such as an operable course of procedure or surgery, patient identification, patient demographics, patient medical history, and / or status of the processing system 100. In some embodiments, the label set database 310 and / or the procedure database 312 are external to the label builder module 302, and thus the module 302 is a database database coupled to the respective database 310 or 312. Includes interface.

  The user interface component includes a controller 304 that itself includes a user input device 314 and a user display device 316. Examples of suitable user input devices 314 include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, gesture-based interfaces, speech and speech recognition interfaces, adaptive interfaces, cameras, motion sensing interfaces , And other user input devices known to those skilled in the art, but are not limited thereto.

  Some of the user interface components 300 may be processor executable software stored in non-transitory computer readable storage media and / or such as dedicated microprocessors, FPGAs, microcontrollers, graphics processing units and DSPs. It may be implemented in whole or in part as a hardware component. In some embodiments, a portion of the user interface component 300 is incorporated into the components of the multi-modality processing system 100 described with reference to FIGS. For example, in some such embodiments, the controller 304 is a component of the bedside controller 118, main controller 120, boom display 122, and / or network console 130 described with reference to FIG. is there. As a further example, in some such embodiments, the label builder module 302 may include the UI framework services 240 of the main controller 120, the bedside controller described with reference to FIG. 118 UI framework services 242 and / or embedded in UI extensions such as IVUS UI extension 246 or IVUS UI extension 248. In other embodiments, the label builder module 302 is a separate and separate component of the multi-modality processing system 100.

  The label builder module 302 constructs labels that are applied to units of medical data, such as IVUS image frames, OCT tomographic image segments, FFR measurement ranges or other parts of the data set. In some embodiments, the label builder module 302 receives medical data that is labeled via the medical data interface 308. In other embodiments, the label builder module 302 receives an identifier corresponding to the medical data in addition to or as an alternative to receiving the medical data itself. For example, in one such embodiment, other components of the multi-modality processing system 100 that are separate from the label builder module 302 may provide the user display device without providing the data to the label builder module 302. The data is displayed at 316. In other such embodiments, the medical data is distributed over the network.

  The labeled data may be any suitable medical data collected by one or more modalities implemented by the multi-modality processing system. In some embodiments, the medical data is raw medical data and a modality acquisition component (eg, IVUS acquisition component 220, forward view IVUS acquisition component, FFR acquisition component, CFR acquisition component, OCT acquisition component of FIG. 2). And / or a transesophageal echocardiography component). In some embodiments, the medical data is processed medical data and includes workflow components (eg, IVUS workflow component 222, forward view IVUS workflow component, FFR workflow component, CFR workflow component of FIG. 2). , OCT workflow component and / or transesophageal echocardiographic workflow component). In some embodiments, the medical data is collected from multiple modalities and may be provided by the MMCM workflow component 214 of FIG.

  To build the labels, in some embodiments, the label builder engine 306 presents the set of valid labels arranged and arranged on the user display device 316 for selection by the operator. In various exemplary embodiments, the set of labels may include a surgical or procedural name, body part, anatomical structure such as a vascular name or vascular segment, patient data, procedural data, and / or other suitable Derived from the identifier. In determining the set of labels to be presented, the label builder engine 306 may determine in the procedural database 312 and / or a label related to the processing environment to determine information related to the current processing environment. In addition, the label set database 310 may be queried. In one such embodiment, the label builder engine queries the procedural database 312 to determine the current operating environment status, and then based on that status to determine the associated label, Queries the label set database 310. In some embodiments, the label builder engine 306 selects a set of labels based on medical data or other corresponding medical data collected by other modalities. As one example, the set of labels for IVUS data may include labels corresponding to blood pressure or pulse measured by other sensor modalities connected to system 100. In that regard, data between modalities is coordinated by the multi-modality processing system using time stamps or other suitable techniques.

  The label builder engine 306 presents a set of labels to the system operator via the user display device 316 of the controller 304. Using the user input device 314, the operator selects an appropriate label from this set. In some embodiments, the operator may supply alternative labels that are not included in this set in addition to or as an alternative to the selection of labels from this set. This user selection is received by the label builder engine 306 and the corresponding label is stored. The label builder engine 306 may perform multiple iterations of building a set of labels, presenting the set, and receiving a selection. In some embodiments, the selected label determines the components of the subsequent label set. In such embodiments, any iteration may depend on the response received in any previous iteration. For example, in one such embodiment, the parent set of labels is derived from the name of the blood vessel and is “LAD” (corresponding to “left front descending branch”), “LCX” (corresponding to “left turning branch”). And “RCA” (corresponding to “right coronary artery”). In this example, the operator selects “RCA” and a response is received at the label builder engine 306. Based on this response, the label builder engine 306 builds a dependent set of labels based on the relevant segment of the right coronary artery. The label builder engine 306 then presents a dependent set of labels via the user display device 316.

  In some embodiments, the label builder engine 306 supports multiple branch dependencies. In such embodiments, the label set may depend on any patient set or may not depend on any patient set, and iterations may be added or removed based on previous responses. May be.

  Further, the label builder engine 306 may receive a request at any time via the user input device 314 to change the previous response and repeat any dependent iterations. The label builder engine 306 may repeat the dependent iterations out of order and / or without repeating other intervening iterations. For example, a change to a response related to the second set of labels does not necessarily result in a change to the first or third set of labels, and thus the first or third set of labels is not necessarily repeated. Instead, changes may be made to the fourth set of labels.

  In embodiments that incorporate a dependent set of labels, the selected label is stored and the dependent set of labels is generated by the label builder engine 306. The generation of subsequent label sets that include dependent sets may be substantially similar to the generation of the parent set. In that regard, subsequent sets may be populated by querying the label set database 310 and / or the procedure database 312. Subsequent sets of labels may be derived from surgical or procedure names, body parts, anatomical structures such as vessel names or vessel segments, patient data, procedure data, and / or other suitable identifiers. . In some embodiments, the label builder engine 306 selects a subsequent set of labels based on medical data or other corresponding medical data collected by other modalities.

  In some embodiments, the label builder engine 306 is attached to other medical data sets, as well as how such subsequent sets may depend on labels selected from previous sets. The set may be selected and populated based on the label that was selected. For example, the first data set may be labeled with “POST” indicating that data was collected after a point in the procedure. From this label, the label builder engine 306 may assume that data has been collected after the critical point, and therefore select a label set for subsequent data sets with “POST”. Good.

  In some embodiments, the label builder engine 306 preselects labels from a given set as the default selection. In such embodiments, the operator may be given the opportunity to ignore the preselected label. As one example, the label builder engine 306 may preselect the current time from a time related label set. The operator may then have the ability to select an alternative time. Preselection may be based on system defaults, operable courses of surgery, patient information, user preferences and / or other criteria.

  When the final iteration ends, the label builder engine 306 builds a final label based on the received response. The final label may also include other relevant information obtained from sources other than user responses. This information may be retrieved from the procedural database 312 and from other sources, such as patient names or other patient identifiers, patient statistical data, operator names, facility identifiers, additional annotations and / or other relevant information. May be included. In some embodiments, this label includes data based on medical data provided by other modalities of the system 100. For example, a label for imaging data may include the corresponding patient's pulse rate or blood pressure.

  The label builder engine 306 gives the final label to the medical data. In some embodiments, the label builder engine 306 modifies the medical data to add this label and returns the updated data to the storage device via the medical data interface 308. In other embodiments, the label builder engine 306 provides a label separate from the corresponding medical data without changing the corresponding medical data.

  FIG. 4 is a diagram of an example user interface 400 for label construction according to some embodiments of the multi-modality processing system 100. This user interface 400 may be displayed on a user display, such as the user display 316 described with reference to FIG. The user interface 400 shows one effective arrangement for displaying information presented by the multi-modality processing system 100, specifically the label builder module 302 of the system. Those skilled in the art will recognize that alternative arrangements are contemplated and provided.

  In the illustrated embodiment, the user interface 400 includes four label set panes (panes 402a, 402b, 402c, and 402d), but other embodiments incorporate any number of panes. In some embodiments, the number of panes increases or decreases based on the selected label. Each label set pane displays an associated label set, eg, label set 404, and provides both an area for selecting a label from the label set and an identifier 406 for the currently selected label. The interface 400 may also include a current selection pane 408 that displays one or more currently selected labels. In some embodiments, the current selection pane 408 allows the operator to select a label. For example, pane 408 allows the use of typing or other inputs. The user interface 400 may also include one or more command elements that send instructions to the label builder module 302. An exemplary instruction includes a selected label, applies the final label (ie, OK button 410), erases the currently selected label (ie, CLEAR button 412), and interrupts without assigning the label ( That is, the CANCEL button 414).

  FIG. 5 is a flow diagram of a method 500 for determining and assigning labels for medical data in the multi-modality processing system 100 according to some embodiments of the present disclosure. Additional steps may be provided before, during and after the steps of the method 500, and some of the steps shown may be replaced or eliminated for other embodiments of the method It is understood.

  At block 502, the medical data to be labeled is received by a module of the system 100, such as the label builder module 302 described with reference to FIG. In some embodiments, an identifier corresponding to the medical data is received in addition to or as an alternative to receiving the medical data itself. The medical data identifier may include some or all of the medical data, and / or may include pointers to data located anywhere in the system or distributed over the network. In block 504, the module selects and populates the first set of labels. In some embodiments, this includes querying one or more databases, such as label set database 310 or procedure database 312. In various exemplary embodiments, the set of labels may include a surgical or procedure name, a body part, a anatomical structure such as a vessel name or vessel segment, patient data, procedure data, and / or any other Some or all of these are derived from the appropriate identifiers. The set of labels may also be derived in part or in whole from information about patient information, treatment regimens, statistical information and any other context information. In some embodiments, the module selects a set of labels based on labeled medical data or other corresponding medical data collected by other modalities. As one example, the set of labels for IVUS data may include labels corresponding to blood pressure or pulse measured by other sensor modalities connected to system 100.

  In block 506, the module presents the first set of labels to a display device, such as the user display device 316 described with reference to FIG. In block 508, the module receives a first label selection based on the first set of labels. The response may be input via an input device such as the user input device 314 described with reference to FIG. In some embodiments, the operator may supply alternative labels that are not included in the first set in addition to or as an alternative to the selection of labels from this set. The label supplied by the operator may be stored for future use, such as reporting or including in a future label set. The module receives the selection and the selected label may be stored in the system 100. At block 510, a second set of labels may be generated by the label builder engine. Subsequent sets of components, including the second set, may be substantially similar to the first set of components. In that regard, subsequent sets may be retrieved from the database and may include surgical or procedure names, body parts, anatomical structures such as vessel names or vessel segments, patient data, procedure data, and / or any May be derived from any other suitable identifier. Subsequent sets may also be populated based on contextual information that may be further retrieved from the database. In some embodiments, the module selects a subsequent set of labels based on medical data or other corresponding medical data collected by other modalities.

  With further reference to block 510, the module may also select and populate a subsequent set of labels (eg, a second set) based on the previously selected label for the medical data. For example, in one such embodiment, the first set of labels is derived from the name of the blood vessel and corresponds to “LAD” (corresponding to “left front descending branch”), “LCX” (corresponding to “left turning branch”). And “RCA” (corresponding to “right coronary artery”). In this embodiment, the operator selects “RCA” and a response is received. Based on this response, a second set of labels is populated based on the corresponding segment of the right coronary artery.

  The module may also select the second set based on the labels selected for or given to other previous medical data sets. For example, the previous data set may be labeled with the patient's name. In some embodiments, the module populates the second set to include the patient's name based on the label attached to the previous data.

  In some embodiments, at block 512, a particular label or value from the second set of labels is preselected. The particular label may be preselected based on any suitable criteria such as system defaults, surgical routine flow, patient information and / or other criteria. As an example, in some embodiments, a time stamp from data acquisition is preselected from a set of time related labels.

  At block 514, a second set of labels is presented on the display device. In block 516, the module receives a second label selection from the user based on the second set of labels. The response may be input via an input device such as the user input device 314 described with reference to FIG. In some embodiments, the second label selection replaces a preselected label or value.

  At block 518, it is determined whether the label query is complete. If the label query is not finished, subsequent iterations of building a set of labels, presenting the set, and receiving a selection may be performed at block 520. Any iteration may depend on the response received in any previous iteration. Of course, any iteration may be independent of any or all previous iterations. At any time, a request may be received via the input device to change the previous response and repeat any dependent iterations. Dependent iterations may be performed out of order and / or without repeating other intervening iterations. For example, a change to a response related to the second set of labels does not necessarily result in a change to the first or third set of labels, and therefore the first or third set of labels is not necessarily repeated. Changes may be made to the fourth set of labels without doing so.

  When the label query is finished, at block 522, the final label is constructed from the received response. The final label is other relevant information obtained from sources other than the user response, such as patient name or other patient identifier, patient statistical data, operator name, facility identifier, additional annotations and / or other relevant information. May be included. In some embodiments, the label includes data based on medical data provided by other modalities. For example, a label for imaging data may include the corresponding patient's pulse rate or blood pressure.

  At block 524, a final label is attached to the medical data. In some embodiments, the medical data is modified to add a label, and the updated data is subsequently stored and / or displayed. In other embodiments, the label is separate from the corresponding medical data and no data modification is performed.

  The systems and methods of the present disclosure provide a mechanism for selecting medical data in a modality and assigning labels to the medical data. Assigning labels to medical data may significantly improve data processing, including search and retrieval. In some embodiments, this assigned label determines which portions of the medical data are stored, retained, and / or archived. For example, in an embodiment, unlabeled data is discarded after a certain amount of time while labeled data is retained. This allows important data to be maintained and avoids storing redundant data. In other embodiments, the labeled data is uploaded to other data processing systems for further analysis. In other embodiments, the data processing system requires medical data labeled with one or more important terms. To further facilitate retrieval, in some embodiments, a searchable index of medical data is generated from assigned labels for quick and effective retrieval. By providing a list of valid labels, some embodiments facilitate the use of standardized terms while still allowing the operator to select external terms from the list to cover special cases. to enable. To avoid overwhelming the operator, some embodiments refine the list with previous selections, procedural information, patient information, other medical data, and / or other criteria. This allows the operator to view most relevant labels, thereby improving labeling speed and accuracy. Of course, it is understood that the above advantages are merely exemplary, and that no particular advantages are essential to any particular embodiment.

  FIG. 6 is a flow diagram of a method 600 for handling interruptions to a labeling procedure within the multi-modality processing system 100 according to some embodiments of the present disclosure. In some embodiments, the labeling procedure supports interrupt commands such as pause or abort. The user may pause the labeling procedure while other tasks are being performed. For example, the window used for the display label set may be minimized or hidden while the underlying window is visible. In another embodiment, the modality connected to processing system 100 provides an interrupt as part of an error procedure that causes labeling to be suspended. When the system or user is ready to continue, the labeling procedure may be resumed from a previous state.

  At block 602, a request to pause the labeling task is received. This request may be made by a user, by a component of the multi-modality system 100, by an external device or system including a modality coupled to the multi-modality system 100, and computing devices networked to the system 100, or any other May be provided by any suitable source. In some embodiments, it is determined whether to allow the source of the request to pause the labeling task. At block 604, the current state of the labeling task is saved. At block 606, the labeling task is aborted. At block 608, it is determined whether a resume command has been received. If a resume command has not been received, the labeling task remains suspended at block 606. The labeling task is resumed at block 610 when a resume request is received. By providing a mechanism for safe interruption of the labeling task, the method allows the operator to be interested in higher priority items without data loss.

  Referring now to FIG. 7, one of the medical systems of FIGS. 1 and 2, including a user interface component 700 for performing on-demand enhancement of medical data according to some embodiments of the multi-modality processing system 100. A functional block diagram of the part is shown. Examples of enhancements include measurements including zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution adjustment, resampling, interpolation, gain adjustment, and distance, area, volume and rate of change measurements. Further emphasis combines data collected by multiple modalities to refine the processing parameters attached to the medical data. Further exemplary enhancements perform 3d reconstruction. Emphasis may also include labeling, highlighting or annotating portions of medical data, such as the labeling described with reference to FIGS. Other emphasis is considered and provided. Such multi-modality enhancement assists the operator in navigating through large amounts of data and in deriving meaningful and accurate diagnostic phases. In some embodiments, the user interface component 700 performs a measurement decision based on the received position marker. The interface component 700 allows an operator to specify highlighting, marking and / or measuring points. This gives the operator context about the displayed images and data that connect the patient to the data.

  The user interface component 700 includes a measurement zoom module 702 coupled to a controller 704 or operator interface. The measurement zoom module 702 includes one or more of a measurement zoom engine 706, a memory device 708 for storing transaction records, a modality feedback interface 710 and a medical data interface 712. The controller 704 includes a user input device 714 and one or more user display devices (shown as a first user display device 716 and a second user display device 718). User input device 714 may be substantially similar to user input device 314 described with respect to FIG. In that regard, examples of suitable user input devices 714 include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, gesture-based interfaces, word and voice recognition interfaces, adaptive interfaces, cameras, These include, but are not limited to, motion sensing interfaces and other user input devices known to those skilled in the art.

  Some of the user interface components 700 may be processor executable software stored in non-transitory computer readable storage media and / or such as dedicated microprocessors, FPGAs, microcontrollers, graphics processing units and DSPs. It may be implemented in whole or in part as a hardware component. In some embodiments, a portion of the user interface component 700 is incorporated into the components of the multi-modality processing system 100 described with reference to FIGS. For example, in some such embodiments, controller 704 is a component of bedside controller 118, main controller 120, boom display 122, and / or network console 130 described with reference to FIG. is there. As a further example, in some such embodiments, the measurement zoom module 702 may include the UI framework services 240 of the main controller 120, the bedside controller 118 described with reference to FIG. Embedded in a UI extension such as a UI framework service 242 and / or IVUS UI extension 246 or IVUS UI extension 248. In other embodiments, the measurement zoom module 702 is a separate and separate component of the multi-modality processing system 100.

  In some embodiments, the measurement zoom module 702 performs on-demand enhancement of medical data. Examples of suitable enhancements include zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution adjustment, resampling, interpolation, gain adjustment, and measurements including distance, area, volume and rate of change measurements. Emphasis may also include labeling or annotating portions of the medical data, such as the labeling described with respect to FIGS. Other emphasis is considered and provided. The measurement zoom module 702 first receives a reference set of medical data via the medical data interface 712. In some embodiments, an identifier corresponding to the medical data is received in addition to or as an alternative to receiving the medical data. The medical data identifier may include some or all of the medical data and / or may include pointers to data located anywhere in the system or distributed over the network. The reference data may also be displayed on a display device (eg, first user display device 716) of controller 704 for viewing by the operator. Display and data processing control may be performed directly by the measurement zoom module 702 and / or by other components of the multi-modality processing system 100 remote from the measurement zoom module 702.

  The reference data may be any suitable medical data. In some embodiments, the medical data includes raw medical data and a modality acquisition component (eg, IVUS acquisition component 220, forward view IVUS acquisition component, FFR acquisition component, CFR acquisition component, OCT acquisition component of FIG. 2). And / or a transesophageal echocardiography component). In some embodiments, the medical data includes processed medical data and includes workflow components (eg, IVUS workflow component 222, forward view IVUS workflow component, FFR workflow component, CFR workflow component of FIG. 2). , OCT workflow component and / or transesophageal echocardiographic workflow component). In some embodiments, the medical data includes data collected from multiple modalities and may be provided by a multi-modality component.

  The measurement zoom module 702 may receive an appropriate region identifier to determine the data set or portion of the data set to enhance. In various embodiments, the region identifier takes the form of a set of boundary coordinates, labels, measurements, and / or bookmarked locations. The measurement zoom module 702 receives region identifiers from the user input device 714 and / or from other system components such as modality workflow components. For example, in the illustrative embodiment, the measurement zoom module 702 receives the region identifier to highlight from the IVUS workflow component 222 via the modality feedback interface 710. In this embodiment, the region identifier is configured to highlight, display, or focus on the region of interest detected by the workflow component. For example, the region of interest may be a vessel wall, a stent, and / or a characterized tissue such as a plaque or lesion. Such automatic highlighting allows the system components to be alerted to potentially important data that may otherwise be unaware.

  In some embodiments, the region identifier is converted from an initial format. For example, the region identifier may include a set of orthogonal coordinates corresponding to pixels of the first display device 716, and the reference data may be represented in polar coordinates on different scales. Accordingly, in various embodiments, the measurement zoom engine 706 of module 702 converts the region identifier from an initial format to an appropriate data format. This may include one or more of scale transformations, coordinate transformations, shape transformations and other suitable transformations known to those skilled in the art.

  Measurement zoom module 702 may also receive an enhancement selection that identifies one or more enhancement functions to perform. The highlight selection may be received from the user input device 714 and / or other components of the system via the modality feedback interface 710.

  Based on one or more of the enhancement selection, region identifier, and reference data, measurement zoom engine 706 determines a target data set for enhancement. The target data set includes, in some embodiments, a portion of reference data that is used to identify the region. The target data set may also include alternative data sets in addition to or instead of the reference data set. In such an embodiment, the data sets included in the goal set correspond to different modalities, single modalities at different times, single modalities in different operating modes, and other combinations of modalities and operating conditions. Also good. In some enhancement implementations, the measurement zoom engine 706 may combine the data sets directly or indirectly to form a reference data set. Accordingly, in one embodiment, the measurement zoom engine 706 can be configured to execute a command at the modality feedback interface 710 that instructs the MMCM component to combine at least a portion of a data set collected by two or more modalities. I will provide a. In a related embodiment, the measurement zoom engine 706 receives data sets collected by two or more modalities and performs a combination within the engine 706. Similarly, in some enhancement implementations, the measurement zoom engine 706 may select a data set to improve responsiveness to reduce the load on storage, networking and / or processing resources. In one such embodiment, the measurement zoom engine 706 selects data other than the region identifier.

  In some embodiments, the particular emphasis includes data collection. The measurement zoom engine may collect additional data sets to add to the target data set, either directly or indirectly. For example, the measurement zoom engine 706 may collect commands from attached equipment and issue commands via a modality feedback interface 710 that instructs the multi-modality processing system to capture subsequent data sets. Good. Subsequent data sets may correspond to the same modality as the reference data set, may correspond to one or more different modalities, and / or a combination of modalities that includes the modality of the reference data set It may correspond to. In this manner, the measurement zoom engine 706 can enhance existing data and guide subsequent data collection.

  If a target data set is identified, the measurement zoom engine 706 may perform enhancement of the target data. Some examples of enhancements include measurements including zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution increase / decrease, resampling, interpolation, gain adjustment, and distance, area, volume and rate of change measurements. . As a further example in some embodiments, the measurement zoom engine 706 changes one or more processing parameters imparted to the target data. In some embodiments, the measurement zoom engine 706 applies the modified parameters directly to the data. In some embodiments, the measurement zoom engine 706 indirectly imparts the changed parameters to the data by feeding back the changes at the modality feedback interface 710 used by the appropriate system component. In some embodiments, engine 706 applies the modified parameters directly or indirectly. In one exemplary embodiment, the measurement zoom engine 706 indirectly determines the focal length parameter used by the IVUS workflow component (eg, the IVUS workflow component 222 of FIG. 2) to generate focused IVUS data. change. In doing so, the measurement zoom engine 706 directs the workflow component to change the focal length parameter to increase the data granularity and resolution in the region corresponding to the region identifier. Provide a command. In a further exemplary embodiment, the measurement zoom engine 706 changes the mode of operation and power flow analysis (eg, Chromaflo® imaging (registered trademark of Volcano Corporation)) within the region corresponding to the region identifier. A command is provided at the modality feedback interface 710 that instructs the IVUS workflow component to execute.

  In some embodiments, the reference data and the enhanced target data are displayed on different user display devices (eg, first user display device 716 and second user display device 718, respectively). In some embodiments, these display devices correspond to a single physical display device at different times. Thus, the reference data set may be displayed on the monitor at the first time point and the highlighted data may be displayed on the monitor at a later time. In some embodiments, these display devices use a representation, such as a “window” of a display “desktop” or a region of a display workspace, of a single display environment shown on a single monitor or display. Corresponds to some. In some embodiments, the first and second display devices correspond to first and second separate physical displays (eg, first and second monitors, respectively). Displaying the reference data and the enhanced target data on different user display devices may be particularly useful in applications where user input may obscure a portion of the display, such as touch screen applications.

  The measurement zoom module 702 may also provide an interface for storing transaction records. Transaction records include region identifiers, highlight selections, data set identifiers, parts of reference data sets, parts of highlighted target data sets, modified processing parameters, and / or other appropriate elements of the transaction May be included. Transaction records are stored in the transaction record database 708 or provided to the medical data interface 712 for transmission and / or storage by system components such as the database management component 216 described in connection with FIG. May be. In some embodiments, the transaction record includes a bookmark that records aspects of the highlight operation that include a description of the data set, region, highlight, and / or other parameters. This bookmark can be retrieved and used to restore a previous state and apply previous data sets, region selections, enhancements and / or parameters to subsequent data sets. This bookmark may serve as a label for indexing, as described above in connection with FIGS. 3-6, or may include such a label.

  Measurement zoom module 702 may also perform on-demand measurements designed to provide additional details, thereby facilitating clinical analysis. For example, an operator may request a detailed measurement at a particular point, referred to as a pinpoint measurement. In such an embodiment, the measurement zoom module 702 receives a location marker specifying this point and performs an analysis of the available medical data to determine the requested measurement. This analysis may include data interpolation, predictive analysis, cross-referencing of medical data sets in modalities, and / or other forms of data analysis. In other examples, the operator may request measurements between two or more reference points. In such an embodiment, the measurement zoom module 702 receives a reference point, such as a distance measurement, volume measurement, area measurement, time measurement, rate of change measurement, and / or other suitable measurement associated with the reference point, etc. Perform measurements. These measurements are merely exemplary and other types of measurements are contemplated or provided.

  As described above, the reference data set may be displayed on a user display device (eg, first user display device 716). The measurement zoom module 702 receives one or more position markers based on the reference data. In various embodiments, position markers are received from a user input device (eg, user input device 714 of FIG. 7) and / or a component of a multi-modality processing system. For example, the workflow component of the multi-modality system 100 may make the operator aware of a subject's diagnostic features or areas such as vessel walls, stents, plaques, lesions, other characterized tissues, and / or other subject structures. A position marker to be displayed may be designated. In some embodiments, the position marker is determined by a combination of user input and system component feedback. For example, the user may identify a first location that has been adjusted or snapped to a nearby point of interest identified by a component or module of the multi-modality processing system.

  Measurement zoom module 702 may also receive measurement selections from user input devices and / or components of the multi-modality processing system. The measurement selection may correspond to one or more measurement functions such as distance measurement, area measurement, volume measurement, time measurement, rate of change measurement, and / or other suitable measurement associated with the reference point.

  Based on the one or more position markers, measurement selections and / or reference data sets, the measurement zoom engine 706 identifies a target data set to measure. In some embodiments, the target data set includes a portion of a reference data set. The target data set may also include other data sets in addition to or in place of the reference data set. In such an embodiment, the data sets included in the goal set correspond to different modalities, a single modality at different times, a single modality in different modes of operation, and other combinations of modalities and operating states. Also good.

  If target data is identified, the measurement zoom engine 706 performs one or more measurements corresponding to the measurement selection. In the exemplary embodiment, the measurement selection specifies a distance measurement. In this example, engine 706 determines the pixel distance between position markers and converts this pixel distance to a physical distance between corresponding points, but other methods of determining distance measurements are contemplated and provided. The In doing so, the measurement zoom engine 706 can use a second format (eg, millimeter-based) associated with physical distance from a first format (eg, pixel-based increments) associated with a user display device. (Increment) may be performed, and conversion from a first coordinate system (for example, polar coordinates) to a second coordinate system (for example, orthogonal coordinates) may be performed. Accordingly, the measurement zoom engine 706 may receive the transform coefficients via the modality feedback interface 710 based on the operational parameters of the associated modality. Continuing with the illustrative embodiment, the measurement zoom engine 706 receives a sample conversion factor per millimeter and a sample conversion factor per pixel. From this, the measurement zoom engine 706 determines the number of pixels per millimeter.

  As described above, in some embodiments, the system includes multiple displays. Thus, the reference data, target data, position markers and measurement values may be displayed on any one or more of those display devices. In one such embodiment, the reference data is displayed on the first user display device 716. An operator superimposes the first user display device 716 and inputs a position marker using a touch sensor (a kind of user input device 714). The measurement zoom engine 706 updates the first user display device 716 with icons indicating position markers and measurement values, and also updates the second user display device 718 with icons indicating position markers and measurement values. Displaying position markers and measurements on multiple user display devices is particularly useful in applications, such as touch screen applications, where user input may obscure part of the display.

  In some embodiments, the measurement zoom module 702 includes an interface that stores elements of measurement operations such as position markers and measurements. For example, measurement zoom engine 706 may include information regarding position markers, measurement selections, measurements, part of a reference data set, part of a target data set, and / or other suitable elements of a transaction. A transaction record may be stored or provided for storage. Transaction records may be stored in the transaction record database 708 or provided in a medical data interface 712 for transmission and / or storage by system components such as the database management component 216 described in connection with FIG. Good. In some embodiments, the transaction record includes a bookmark that records aspects of the measurement operation including a description of the data set, position marker, measurement type, measurement value, or other parameter. This bookmark can be retrieved and used to restore a previous state and attach subsequent data sets to previous data sets, position markers, measurement types and / or other parameters. This bookmark may serve as a label for indexing, as described above in connection with FIGS. 3-6, or may include such a label.

  FIG. 8 is a diagram of an exemplary user interface 800 for on-demand data enhancement according to some embodiments of the multi-modality processing system 100. The user interface 800 may be presented on a single user display or multiple displays, such as the first and second user displays 716 and 718 described with reference to FIG. The user interface 800 shows one effective arrangement for displaying information presented by the multi-modality processing system, specifically the system's measurement zoom module 702. Those skilled in the art will recognize that alternative arrangements are contemplated and provided.

  Interface 800 may be different display devices, a single physical display device at different times, different parts of a single display environment on a single display device (a “window” on a “display” desktop or an area of a display workspace). A first data display window 802 and a second data display window 804 that may correspond to different physical displays (ie, different monitors). In the illustrated embodiment, the first data display window 802 presents the reference data set in the first window on the desktop, and the second data display window 804 is the target highlighted in the second window on the desktop. Present the data set.

  The interface 800 may also include a region selection tool 806 that allows an operator to specify a region or data portion. In various non-limiting examples, the operator can use mouse input, touch-based input, or digital input using a reference point specified by mouse input, touch-based input, or digital pen input using the typed coordinates. Regions may be specified using geometric shapes specified by pen input and / or other methods of data entry. In that regard, the region selection tool 806 allows the operator to select a region or data portion by analyzing the user input provided.

  This interface may also include a toolbar 808. In various embodiments, the toolbar 808 is used to select commands for the multi-modality processing system. Exemplary commands include selecting a region to highlight, selecting a highlight to perform, and a data store and load command. The interface may also include a plurality of command elements 810 for quickly selecting commands to send to the multi-modality processing system. Any command that may be selected from the toolbar is suitable for selection using the command element 810. In that regard, in the exemplary embodiment, interface 800 includes a command element 810 configured to select an emphasis to perform.

  FIG. 9 is a diagram of an example user interface 900 for data measurement according to some embodiments of the multi-modality processing system 100. The user interface 900 may be presented on a single user display or multiple displays, such as the first and second user displays 716 and 718 described with reference to FIG. The user interface 900 presents one possible arrangement for displaying information presented by the multi-modality processing system, more specifically, the measurement zoom module 702 of the system. Those skilled in the art will appreciate that alternative arrangements are contemplated and provided.

  In many respects, the user interface 900 is substantially similar to the user interface 800 of FIG. In that regard, the user interface 900 may include different display devices, a single physical display device at different times, different portions of a single display environment on a single display device (a “window” of a display “desktop”, or A first data display window 802 and a second data display window 804 corresponding to different physical displays (ie, different monitors) and / or representations such as areas of the display workspace). User interface 900 may also include a toolbar 808 and a command element 810. The user interface 900 may also include icons corresponding to the markers 902, i.e., position markers provided to the system used for data measurement. The position marker may be provided by the operator via the user input device 1212. In some embodiments, the operator specifies and provides position markers using typed coordinates, using mouse input, using touch-based input, and / or using digital pen input. The position marker may also be provided by other components of the system 100 via the module interface 1208 in addition to or instead of the position marker provided by the operator. Although two position markers are shown, in other embodiments, any number of position markers may be provided. In the illustrated embodiment, the marker 902 is displayed in the first data display window 802 and the second data display window 804, but in a further embodiment, the first data display window 802 or the second data display window 804 is displayed. This marker is displayed in the data display window 804. The user interface may also include a measurement output area 904 that displays measurement values determined based on the position markers. In some embodiments, the measurement values are displayed in one or more of the first data display window 802 and the second data display window 804 in addition to or instead of the measurement output area 904. . While the illustrated measurement output area 904 displays the distance, in other embodiments, the measurement output area 904 is an area measurement, volume measurement, time measurement, rate of change measurement, and / or other suitable reference point related reference point. Any type of measurement used by the system may be displayed, including various measurements.

  FIG. 10 is a flow diagram of an on-demand data enhancement method 1000 within the multi-modality processing system 100 according to some embodiments of the present disclosure. It will be appreciated that additional steps may be provided before, during and after the steps of method 1000 and that some of the steps shown may be replaced or eliminated in other embodiments of the method.

  At block 1002, a reference set of medical data is received by the multi-modality processing system 100. In some embodiments, the identifier corresponding to the medical data is received in addition to or as an alternative to receiving the medical data itself. The medical data identifier may include some or all of the medical data and / or may include pointers to data located anywhere in the system or distributed over the network. The reference set of medical data may be any suitable medical data. For example, the medical data may correspond to raw medical data provided by the modality acquisition component, processed medical data provided by the workflow component, and / or multiple modalities and provided by the multi-modality component Aggregate data may be included. At block 1004, a reference set of medical data may be displayed on the first display device.

  At block 1006, a region identifier is received. The region identifier corresponds to a subset of the reference set of medical data. In various embodiments, the region identifier takes the form of a boundary identifier, a region identifier such as a grid index, gestures, commands, labels, measurements, bookmark positions, and / or a set of appropriate formats. In some embodiments, the region identifier is received from a user input device, such as the user input device 714 described with reference to FIG. Examples of suitable user input devices include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, gesture-based interfaces, speech and speech recognition interfaces, adaptive interfaces, cameras, motion sensing interfaces Light-based sensing mechanisms (eg, infrared sensing) and other user input devices known to those skilled in the art, but are not limited thereto. In some embodiments, the region identifier is received from another system component, such as a modality workflow component. In one such embodiment, the region identifier is configured to highlight, display, or focus on the structure of interest. Such automatic emphasis allows the system components to be alerted to potentially important data that may otherwise not be noticed.

  In some embodiments, the region identifier is converted from an initial format. For example, the region identifier may include a set of orthogonal coordinates corresponding to the pixels of the display device, and the reference data may be represented in polar coordinates on different scales. Thus, in various embodiments, the region identifier is converted from an initial format to an appropriate data format. This may include one or more of scale transformations, coordinate transformations, shape translations and other suitable transformations known to those skilled in the art.

  At block 1008, an enhancement selection is received. This selection identifies one or more actions to perform and is received from a user input device and / or system component.

  At block 1010, a target data set is identified based on one or more of the highlight selection, region identifier, and reference data. In some embodiments, the target data set includes a portion of a reference data set. The target data set may also include alternative data sets in addition to or instead of the reference data set. In such an embodiment, the data sets included in the goal set correspond to different modalities, a single modality at different times, a single modality in different modes of operation, and other combinations of modalities and operating states. Also good. In some embodiments, the method includes combining data sets directly or indirectly to form a target data set. Similarly, in some embodiments, the method includes selecting a portion of the target data set. In one such embodiment, the method includes selecting data other than the region corresponding to the region identifier.

  At block 1012, enhancement is performed on the target data set by the system 100. In various embodiments, this enhancement is determined by the enhancement selection, region identifier, and / or target data set. Examples of enhancements include measurements including zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution increase / decrease, resampling, interpolation, gain adjustment, and distance, area, volume and rate of change measurements. Further emphasis combines data collected by multiple modalities to refine the processing parameters attached to the medical data. For example, in one such embodiment, multi-modality processing system 100 modifies the focus range parameter provided by the IVUS workflow component of system 100 to provide high resolution within the region specified by the region identifier. To do. In other such embodiments, the multi-modality processing system changes the operating mode of the workflow component of system 100.

  Emphasis may also include labeling or annotating portions of the medical data, such as the labeling described with respect to FIGS. Other emphasis is considered and provided.

  At block 1014, the highlighted target data set is displayed on the second user display. The first and second user displays may correspond to different user display devices (eg, first user display device 716 and second user display device 718 in FIG. 7). In some embodiments, these display devices correspond to a single physical display device at different times. Thus, the reference data set may be displayed on the monitor at the first time point and the highlighted data set may be displayed later on this monitor. In some embodiments, the different user display device, the second user display, corresponds to a portion of a single display environment on a single display device (display “desktop” “window” or display screen). Using expressions such as workspace areas). In some embodiments, the different display devices correspond to first and second physical displays (eg, first and second monitors, respectively).

  At block 1016, the enhanced transaction record is stored. In various embodiments, this record may include a region identifier, highlight selection, data set identifier, part of the reference data set, part of the highlighted target data set, modified processing parameters, and / or transactions. Including other suitable elements. In some embodiments, storing includes providing transaction records at a system interface for a remote storage device. The transaction record may include a bookmark that records aspects of the highlight operation, including a description of the data set, region, highlight or other parameters. This bookmark can be retrieved and used to restore a previous state and apply previous data sets, region selections, enhancements and / or parameters to subsequent data sets. This bookmark may serve as a label for indexing as described above in connection with FIGS. 3-6, or may include such a label.

  FIG. 11 is a flow diagram of a data measurement method 1100 within the multi-modality processing system 100 according to some embodiments of the present disclosure. It will be appreciated that additional steps may be provided before, during and after the steps of method 1100 and that some of the steps shown may be replaced or eliminated in other embodiments of the method.

  At block 1102, a reference set of medical data is received by the multi-modality processing system 100. In some embodiments, the identifier corresponding to the medical data is received in addition to or as an alternative to receiving the medical data itself. The medical data identifier may include some or all of the medical data and / or may include a pointer to data located anywhere in the system or distributed over the network. The reference set of medical data may be any suitable medical data. At block 1104, a reference set of medical data may be displayed on the first display device. At block 1106, a position marker is received. The location marker may be received from a component of the user interface, such as the user input device 714 of FIG. 7, or from a component of the multi-modality system 100, such as a workflow component. As an example of the latter, in an embodiment, the workflow component of the multi-modality system 100 may include associations such as vessel walls, stents, plaques, lesions, other characterized tissues, and / or other subject structures. Specify a position marker that warns the operator of the diagnostic feature to be performed. In some embodiments, the position marker is determined by a combination of user input and system component feedback. For example, the user may identify a first location that has been adjusted or snapped to a nearby point of interest identified by a component or module of the multi-modality processing system.

  At block 1108, a measurement selection is received. This selection identifies one or more measurements to perform. For example, this selection may specify distance measurements, area measurements, volume measurements, rate of change measurements, pinpoint measurements, and / or other suitable measurements.

  At block 1110, a target data set is identified. The target data set may depend on one or more position markers, measurement selections and / or reference data sets. In some embodiments, the target data set includes a portion of a reference data set. The target data set may also include other data sets in addition to or as an alternative to the reference data set. In such an embodiment, the data sets included in the goal set correspond to different modalities, single modalities at different times, single modalities in different operating modes, and other combinations of modalities and operating conditions. Also good.

  At block 1112, measurements are performed by the system 100 on the target data set. In various embodiments, the particular measurement performed is determined by measurement selection, one or more position markers, and / or a target data set. In the exemplary embodiment, the measurement is a distance measurement and a second coordinate system (eg, millimeter-based) that represents a physical distance from a first coordinate system (eg, pixel-based coordinates) that represents the user display device. Conversion to the coordinates of. In a particular embodiment, a sample conversion factor per millimeter and a sample conversion factor per pixel are received. From this, the number of pixels per millimeter is determined. Subsequently, based on the pixel distance between the position markers, a physical distance can be determined between corresponding points. In a further exemplary embodiment, the selected measurement action is a pinpoint measurement, ie a detailed measurement at a specific point. The analysis may include data interpolation, predictive analysis, cross-referencing of medical data sets in modalities, and / or other forms of data analysis. The measurement process for determining pinpoint data values may include data interpolation, predictive analysis, cross-referencing of medical data sets in modalities, and / or other forms of data analysis. Referring to the measurements in block 1112, other measurement operations are contemplated and provided, including area measurements, volume measurements, rate of change measurements, cross-modality measurements, and other measurements known to those skilled in the art.

  In block 1114, the value determined by the measurement operation is displayed. As described above, in some embodiments, the system includes multiple displays. Thus, the reference data, target data, position markers and measurement values may be displayed on any one or more display devices. At block 1116, a transaction record is generated and stored. The transaction record may include information regarding location markers, measurement selections, measurements, part of the reference data set, part of the target data set, and / or other suitable elements of the transaction. In some embodiments, storing includes providing transaction records at a system interface for a remote storage device. Transaction records may include bookmarks that record characteristics of measurement operations including descriptions of data sets, position markers, measurement types, specific values, or other parameters. The bookmark can be retrieved and restored to the previous state and used to attach the previous data set, position marker, measurement type and / or other parameters to subsequent data sets. The bookmark may serve as a label for indexing as described above in connection with FIGS. 3-6, or may include such a label.

  Referring now to FIG. 12, some functions of the medical system of FIGS. 1 and 2, including a user interface component 1200 for distinguishing user input according to some embodiments of the multi-modality processing system 100 A block diagram is shown. User interface component 1200 converts operator input in the form of gestures into commands for a medical system. Gestures provide an intuitive mechanism for controlling data acquisition, manipulation and viewing. In some embodiments, gesture recognition allows an operator to select a command without changing the input device, without switching active windows, and / or without navigating complex menu structures. to enable. Further, in some embodiments, gesture recognition eliminates unnecessary steps and reduces GUI clutter. An efficient GUI that results in good patient results and reduced costs reduces procedure time.

  The user interface component 1200 may further include one or more of a gesture recognition engine 1204, a gesture database 1206, and a module interface 1208 for communicating with other components of the medical system. 1202 is included. The user interface component 1200 also includes a controller 1210 or operator interface. The controller 1210 includes a user input device 1212 and one or more user display devices 1214. User input device 1212 may be substantially similar to user input device 314 described with respect to FIG. In that regard, examples of suitable user input devices 1212 include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, gesture-based interfaces, word and voice recognition interfaces, adaptive interfaces, cameras , Motion sensing interfaces and other user input devices known to those skilled in the art, but are in no way limited thereto.

  Some of the user interface components 1200 may be processor executable software stored on non-transitory computer readable storage media and / or such as dedicated microprocessors, FPGAs, microcontrollers, graphics processing units and DSPs. All or some of the hardware components may be implemented. In some embodiments, a portion of the user interface component 1200 is incorporated into the components of the multi-modality processing system 100 described with reference to FIGS. For example, in some such embodiments, the controller 1210 is a component of the bedside controller 118, main controller 120, boom display 122, and / or network console 130 described with reference to FIG. is there. As a further example, in some such embodiments, the gesture recognition module 1202 may include the UI framework service 240 of the main controller 120, the UI of the bedside controller 118 described with reference to FIG. It is incorporated into a UI extension, such as framework service 242, and / or IVUS UI extension 246 or IVUS UI extension 248. In other embodiments, the gesture recognition module 1202 is a separate and different component of the multi-modality processing system 100.

  The gesture recognition module 1202 interprets user input based on the rules retrieved from the gesture database 1206 and translates the user input into commands. As such, module 1202 allows an operator to use gesture-based input to issue commands. Valid gestures include body gestures and verbal commands. The body gesture may or may not include manipulating the input device, such as moving the input device or performing a key press. Gestures may also include other interface methods known to those skilled in the art. For reference, gestures include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, speech and voice recognition interfaces, adaptive interfaces, cameras, motion sensing interfaces, and others known to those skilled in the art. May be captured by any suitable user input device, including other user input devices.

  The gesture recognition module 1202 receives user input sequences from one or more user input devices 1212. The user input sequence is divided into discrete elements representing data collected by the input device 1212, such as button presses, key presses, movements, captured audio, captured video and other input data known to those skilled in the art. May be. The user input sequence may include formatting data including device identifier, device status, device polling and delimiter. In some embodiments, the user input sequence combines multiple types of data collected from multiple input devices. In such an embodiment, the sequence includes data from a keyboard and mouse. In other such embodiments, the sequence includes pen-based input and data from a virtual keypad. An exemplary sequence includes a mouse button down event followed by position data, followed by a mouse button up event. Further exemplary input sequences include a touch start event, a touch move event, and a touch end event.

  The gesture recognition engine 1204 of module 1202 builds a list of active commands for comparison with the user input sequence. In some embodiments, the module 1202 receives a system status indicator corresponding to the state or mode of operation of the multi-modality processing system. This status indicator may indicate a modality. In that regard, exemplary system status indicators include current data for IVUS, FL-IVUS, FFR identification method, coronary flow reserve capacity (CFR) identification method, optical coherence tomography (OCT), transesophageal tract. Indicates corresponding to one or more of echocardiography, pressure, flow and / or other suitable modalities. The status indicator may also indicate the status of the hardware component whether medical data is being collected or reviewed, the list of running software components, the status of the software component, the mode of the software component and / or any of the system Other suitable characteristics may be exhibited.

  Based on the state indicator, the gesture recognition engine 1204 determines an active command, which is supported in the current state of the system. In building the list of active commands, engine 1204 may query gesture database 1206 based on the system status indicator. The record of the gesture database 1206 may include one or more of command name, command format, interphase system state, component receiving the command, identification of relevant gesture characteristics, and other suitable information. In some embodiments, the list of active commands depends on one or more modalities corresponding to the current set of medical data. For example, certain commands may be active during data collection and / or review for forward view IVUS images (FL-IVUS), while other commands may collect and / or review OCT data. May be active during. Further, in some embodiments, the subset of active commands is common to multiple mobility. For example, some commands related to manipulating images may be uniform across different imaging modalities. Having uniformity across multiple modalities helps simplify the interface and reduces the learning curve and operator burden.

  When this list is constructed, the gesture recognition engine 1204 matches one or more elements of the user input sequence to the list of active commands. Elements may be adapted based on the identification of active command characteristics. If a match is found, the sequence is translated into a corresponding command or series of commands and displayed on the module interface 1208 used by the appropriate component. In addition to identifying associated commands, the translation may include converting one or more elements of the user input sequence into command parameters. For example, a set of coordinates obtained from a user input sequence may be passed as a command parameter. These coordinates are followed by the modality extension including measurement zoom module 702, UI framework service 240, system UI extension 244, IVUS UI extension 246 and modality UI extension 250 described with respect to FIG. / Or may be used by components of the system, such as other suitable components of the system 100.

  As a non-limiting example, one gesture-based command, i.e., a poke command is intended to select a single point, followed by a travel distance below a threshold for unconscious movement. Characterized by a selection event. The exemplary poke command passes a set of coordinates derived from a gesture input sequence as a parameter. Another exemplary command, i.e., a diameter measurement command, is characterized by a selection event followed by a travel distance greater than a threshold for unintentional operation, where the movement is within the tolerance of a linear path threshold. Is in range. An exemplary diameter measurement command passes start and end coordinates derived from a gesture input sequence as a parameter. Another exemplary command, an area measurement command, is characterized by a selection event followed by a travel distance greater than a threshold for unconscious movement, where the movement is within the tolerance of the linear path threshold. Exceed. The example area measurement command passes a geometric shape approximating the path followed by the gesture input sequence as a parameter.

  When the translation is complete, the gesture recognition engine 1204 presents the translation command on the module interface 1208 used by the appropriate component. In some embodiments, the gesture recognition module 1202 displays the translated command to the operator via the controller 1210 for confirmation prior to presenting the command at the module interface 1208. If the operator confirms the command, the command may be displayed on the module interface 1208 for use in the system 100. If the operator is not satisfied with the translated command, the gesture recognition module 1202 may cancel and / or change the command at the operator's request. In some embodiments, the operator may change the elements of the command with additional gestures and inputs. In some such embodiments, the gesture recognition module 1202 presents the operator with reference points that the operator can drag or manipulate to adjust the position or path. In other such embodiments, the gesture recognition module 1202 presents suggested changes to the current command to the operator. The suggestion may be based on system defaults, user preferences and / or other criteria. In other such embodiments, the gesture recognition module 1202 presents the operator with a list of alternative commands for the operator to select. U.S. Provisional Application No. 61 / 560,677 entitled "MEDICAL SENSING CONTROL SYSTEM AND METHOD" also discloses gestures and gesture translation, and is incorporated herein by reference in its entirety. .

  An exemplary embodiment is described with reference to FIGS. FIG. 13 is a diagram of an example user interface 1300 that displays a gesture-based user input sequence according to some embodiments of the multi-modality processing system 100. FIG. 14 is a diagram of an example user interface 1400 that displays a gesture-based user input sequence according to some embodiments of the multi-modality processing system 100. FIG. 15 is a diagram of an example user interface 1500 that displays translated instructions according to some embodiments of the multi-modality processing system 100. FIG. 16 is a diagram of an example user interface 1600 that displays a gesture-based user input sequence according to some embodiments of the multi-modality processing system 100. FIG. 17 is a diagram of an example user interface 1700 that displays translated instructions according to some embodiments of the multi-modality processing system 100.

  With reference to FIG. 13 in more detail, in the illustrated embodiment, a set of medical data is displayed on a first user display 1302 and a second user display 1304. The gesture recognition module 1202 receives a status indicator that identifies the modality corresponding to the set of medical data and the operating mode. The gesture recognition engine 1204 of the module 1202 then queries the gesture database 1206 based on the status indicator and receives an active command. This active command identifies the feature used to recognize the associated gesture and is added to the list.

  In some examples, the gesture recognition engine 1204 receives a user input sequence that includes a touch initiation event. In the illustrated embodiment, the marker 1310 is displayed based on the touch start event. The user input sequence includes position data and a touch end event (displayed with marker 1312). Markers 1310 and 1312 may be displayed on any one or more suitable user display devices including first user display 1302 and second user display 1304 of FIG. The gesture recognition engine 1204 adapts the user input sequence to gestures based on the identification of active command characteristics. In the example of FIG. 13, the poke command is intended to select a single point if the action appears to be small or unconscious and ignores the movement of the input device. A threshold may be established for unconscious actions. Thus, the identification of a poke command feature includes a touch start event followed by a travel distance less than a length threshold followed by a touch end event. In the illustrated embodiment, the user input sequence is interpreted as a poke command.

  As part of the translation, the gesture recognition engine 1204 determines a set of coordinates to pass as parameters for the poke command. In various embodiments, these coordinates are based on the coordinates of the touch start event, the touch end event, or an intermediate point between the touch start event coordinates and the touch end event coordinates. In some embodiments, these coordinates may be adjusted by a position determined by the system 100 or snapped to that position. For example, in embodiments where the system 100 supports boundary detection, a user-specified location may be snapped to neighboring boundaries and the coordinates determined accordingly. Boundary detection in a medical imaging context is described in US Pat. No. 6,200,268, issued on March 13, 2001 under the name “VASCURALAR PLAQUE CHARACTERIZATION”, “INTRAVASCULAR ULTRASONIC ANALYSTY USING ACTIVE COUNTY COUNTY COUNTY CONTOWN COUNTY COUNTY COUNTY COUNTY COUNTY COUNTY TOWN US Pat. No. 6,381,350 issued April 30, 2002, entitled “SYSTEM AND METHOD OF CHARACTERIZING VASCULAR TISSUE”, issued July 11, 2006. No. 7,074,188, “NON-INVASIVE TISSUE CHARACTER” U.S. Patent No. 7,175,597 issued on February 13, 2007 under the name "ZATION SYSTEM AND METHOD" and issued on May 8, 2007 under the name "SYSTEM AND METHOD FOR VASCULAR BORDER DETECTION" US Pat. No. 7,215,802, issued in the name of “SYSTEM AND METHOD FOR IDENTIFYING A VASCULAR BORDER”, US Pat. No. 7,359,554, issued April 15, 2008, and “ U.S. Pat. No. 7,463,759 issued on Dec. 9, 2008 under the name "SYSTEM AND METHOD FOR VASCULAR BORDER DETECTION" The teachings of which are incorporated herein by reference in their entirety.

  The translated “poke” command is a modality extension that includes the measurement zoom module 702, UI framework service 240, system UI extension 244, IVUS UI extension 246, and modality UI extension 250 described with respect to FIG. And / or a module interface 1208 used by system components, such as other suitable components of system 100.

  Referring to FIG. 14, in the illustrated embodiment, the gesture recognition engine receives a different user input sequence after displaying a set of medical data on a first user display 1302 and a second user display 1304. To do. An exemplary user input sequence includes a touch start event (shown by marker 1410), position data (shown by trace 1412), and a touch end event (shown by marker 1414). Markers 1410 and 1414 and trace 1412 may be displayed on any one or more suitable user display devices 1214 including first user display 1302 and second user display 1304 of FIG. The gesture recognition engine 1204 adapts the user input sequence to gestures based on the identification of active command characteristics. In the example of FIG. 14, the diameter measurement command is intended to measure the distance in a straight line between two specified points. Thus, the identification of the command features includes unintentional movement and a movement distance that is greater than the threshold for the movement path, within the tolerance of the linear path threshold. In the illustrated embodiment, the user input sequence is interpreted as a diameter measurement command.

  The gesture recognition engine 1204 may perform further translation, such as determining coordinate endpoints based on user input. In various embodiments, the coordinate endpoint is based on the coordinates of the touch start event and / or the touch end event. In some embodiments, the coordinate endpoints may be adjusted by an area determined by the system 100 or snapped to that area. For example, in embodiments where the system 100 supports boundary detection, user-specified locations may be snapped to neighboring boundaries and associated coordinates. The gesture recognition engine 1204 may also correct or calculate anomalies that affect the diameter measurement.

  Referring to FIG. 15, in the illustrated embodiment, the translated diameter measurement command described with reference to FIG. 14 includes one or more of the first display device 1302 and the second display device 1304. Displayed on one or more of the user display devices 1214. This gives the operator the opportunity to verify the translation. In the illustrated embodiment, the first user display 1302 and the second user display 1304 include a marker 1510 that indicates the start of the command, a marker 1514 that indicates the end of the command, and a trace that indicates the linear path of the command. 1512 is used. The operator may confirm the translated command via the user input device 1212. If this command is confirmed by the operator, this command may include measurement zoom module 702, UI framework service 240, system UI extension 244, IVUS UI extension 246 and modality UI extension 250 described with respect to FIG. Provided by a module interface 1208 used by each component of the system, such as a modality extension including: and / or other suitable components of the system 100. If the operator is not satisfied with the translated command, the gesture recognition module 1202 may cancel and / or change the command at the operator's request. For example, the gesture recognition module 1202 may allow the operator to drag the endpoint, may present the operator with other reference points that the operator can manipulate, and suggest changes to the current command. May be presented to the operator and / or a list of alternative commands for the operator to select may be presented.

  Referring now to FIG. 16, in the illustrated embodiment, the gesture recognition engine performs other operations based on the set of medical data displayed on the first user display 1302 and the second user display 1304. The user input sequence is received. An exemplary user input sequence includes a touch start event (shown by marker 1610), position data (shown by trace 1612), and a touch end event (shown by marker 1614). Markers 1610 and 1614 and trace 1612 may be displayed on any one or more suitable user display devices 1214 including first user display 1302 and second user display 1304 of FIG. The gesture recognition engine 1204 adapts the user input sequence to the gesture based on the identification of the active command characteristics. In the example of FIG. 16, the area measurement command is intended to determine the area within the specified boundary. Thus, identifying the features of the command includes unintentional movement and a movement distance that is greater than the threshold for the movement path that exceeds the tolerance of the linear path threshold. In the illustrated embodiment, the user input sequence is interpreted as an area measurement command.

  As part of the translation, the gesture recognition engine may perform further interpretation, such as determining coordinate endpoints for the command. In various embodiments, the coordinate endpoint is based on the coordinates of the touch start event and / or the touch end event. The gesture recognition engine 1204 may also determine a smooth geometric path, such as a circular, oval, spline or polygonal path, for the corresponding command. The position, shape and nature of the geometric path may be determined by a portion of the position and structure detected and recognized by the system 100. In the exemplary embodiment, the geometric path represented by the circle is resized and moved through the boundary points determined by the system 100. In a further exemplary embodiment, the oval geometric path is replaced with a circular geometric path so that it passes closer to one or more boundary points determined by the system. In another exemplary embodiment, the spline breakpoint is adjusted to snap to a boundary point determined by the system.

  Referring to FIG. 17, in the illustrated embodiment, the translated area measurement command described with reference to FIG. 16 provides one or more user display devices 1214 that provide an opportunity to verify the translation. Are displayed on one or more of them. In the illustrated embodiment, the first user display 1302 and the second user display 1304 use a trace 1712 that shows the smooth geometric path of the command. The operator may confirm the translated command via the user input device 1212. If this command is confirmed by the operator, this command may include measurement zoom module 702, UI framework service 240, system UI extension 244, IVUS UI extension 246 and modality UI extension 250 described with respect to FIG. Provided by a module interface 1208 used by each component of the system, such as a modality extension including: and / or other suitable components of the system 100. If the operator is not satisfied with the translated command, the gesture recognition module 1202 may cancel and / or change the command at the operator's request. For example, the gesture recognition module 1202 may present the operator with a reference point that can be dragged or manipulated to adjust the endpoint or midpoint, and presents suggested changes to the current command to the operator. And / or a list of alternative commands for the operator to select may be presented.

  FIG. 18 is a flow diagram of a gesture recognition method 1800 in the multi-modality processing system 100 according to some embodiments of the present disclosure. It will be appreciated that additional steps may be provided before, during and after the method 1800 and that some of the steps shown may be replaced or eliminated in other embodiments of the method.

  At block 1802, a status indicator is received by a component of the system 100, such as the gesture recognition module 1202 of FIG. This status indicator may indicate a modality. In that regard, the exemplary system status indicator shows that current data includes IVUS, FL-IVUS, FFR identification, coronary flow reserve ratio (CFR) identification, optical coherence tomography (OCT), intracardiac echocardiography. , Corresponding to one or more of pressure, flow and / or other suitable modalities. The status indicator may also indicate the status of the hardware component on which medical data is collected or reviewed, the list of running software components, the status of the software component, the mode of the software component, and / or the system Any other suitable identification of may be indicated.

  At block 1804, a list of active commands is constructed. In some embodiments, the list of active commands is based on a status indicator. For example, the list may be based on modalities and may include a subset of active commands that apply multiple modalities. In some embodiments, the construction of the list includes querying an electronic database, ie, the gesture database 1206, using a status indicator. The record of the gesture database 1206 may include one or more of a command name, command format, interphase system state, component receiving the command, identification of associated gesture characteristics, and other suitable information. .

  At block 1806, a user input sequence is received. This sequence consists of a keyboard, keypad, mouse, trackball, digital pen, touch-based interface, speech and speech recognition interface, adaptive interface, camera, motion sensing interface and other user input devices known to those skilled in the art May be received from a suitable user input device. In some embodiments, the user input sequence combines multiple sequences from multiple input devices, eg, keyboard and mouse. This sequence may take any suitable form known to those skilled in the art. In the exemplary embodiment, the elements of this sequence represent device status, events, position data such as operational data, audio samples, delimiters and / or other sequence elements.

  At block 1808, the user input sequence is correlated with commands from the supported command list. This may include comparing one or more sequence elements of the user input sequence to characteristic criteria of the active command. It may also include mapping one or more sequence elements to corresponding command parameters. At block 1810, a command mapped to the user input sequence is generated. This may include generating values for one or more parameters of this command based on a user input sequence.

  At block 1812, the command is displayed on the user display device for confirmation by the operator. This may include receiving an acknowledgment via user input device 1212. If the operator is not satisfied with the translated command, at block 1814, the command may be canceled and / or changed at the operator's request. In some embodiments, this includes presenting options for changes to the command and / or receiving further user input specifying the changes to apply. In some such embodiments, a reference point is presented that the operator can drag or manipulate to adjust the position or path. In other such embodiments, suggested modifications to the current command are presented and a selection is received. This suggestion may be based on system defaults, user preferences and / or other criteria. In other such embodiments, a list of alternative commands is presented and a selection is received. When the operator confirms the command, at block 1816, the command is presented to the module interface 1208 for use by the appropriate component of the system.

  FIG. 19 is a functional block diagram of a portion of the medical system of FIGS. 1 and 2 including a user interface component 1900 for navigating a set of medical data according to some embodiments of the multi-modality processing system 100. is there. The user interface component 1900 presents a high-level display of a reference set of medical data from which the operator can select a subset to explore in the detailed display. In some embodiments, the user interface component 1900 performs a uniaxial zoom so that the detailed display is generated, and the subset of data is large along a first axis that is independent of the second axis. Will be changed. This is particularly useful for medical data sets that contain plotted data over a range of time or distance, or when the data values are periodic. In such data, the peak-to-peak extrema along one axis may not change substantially when the scaling of the second axis is adjusted.

  One example of a data set suitable for uniaxial zoom enhancement is blood flow reserve ratio (FFR) data. FFR is a currently accepted technique for assessing the severity of stenosis in blood vessels, including ischemic lesions. FFR is the calculated ratio of the peripheral blood pressure measurement (taken on the distal (distal) side of the stenosis) to the proximal blood pressure measurement (taken on the proximal side of the stenosis). FFR provides an indication of the severity of a lesion that allows a determination as to whether an obstruction restricts blood flow in a blood vessel so that surgery is required. A normal value for FFR in healthy blood vessels is 1.00, while values less than about 0.80 are generally considered severe and require surgery. Disclosed for further measurements such as InstantWave-Free Ratio (TM) functional data (iFR (TM) function) (both are trademarks of Volcano Corp.) and the use of pressure ratios that can be used without hyperemic agents The measurement described in US patent application Ser. No. 13 / 460,296, entitled “DEVICES, SYSTEMS, AND METHODS FOR ASSESSING A VESSEL” is also suitable for such enhancement. The iFR® and FFR data sets may include pressure measurements, pressure ratios, and / or values corresponding to pressure differences measured over a time range. By allowing the operator to adjust the scaling of the time axis independent of the pressure axis or ratio axis, the user interface component 1900 can provide a more useful display of the selected subset to the user. .

  In addition, the user interface component 1900 allows an operator to specify multiple data operations including data selection, data enhancement and bookmarking using advanced and / or detailed views. In that regard, the user interface component 1900 may include a measurement zoom module 702 described with respect to FIG. 7, a gesture recognition module 1202 described with respect to FIG. 12, and / or other systems 100 disclosed herein. Some or all of the component functionality may be incorporated. Accordingly, in various embodiments, the user interface component 1900 may include zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution increase / decrease, resampling, interpolation, gain adjustment, and distance, area, volume and rate of change. Various data enhancements can be performed on a subset of data including measurements including As such, the user interface component 1900 includes an improved interface for navigating and manipulating data sets in a manner that depends on the nature of the underlying data.

  User interface component 1900 includes a measurement navigator module 1902 that further includes a navigator engine 1904, a medical data interface 1906 for receiving a set of medical data, and transactions, bookmarks and annotations. One or more of the electronic databases 1908 may be included. User interface component 1900 also includes a controller 1910 or an operator interface. The controller 1910 may include one or more user display devices 1914 and user input devices 1912. User input device 1912 may be substantially similar to user input device 314 described with respect to FIG. In that regard, examples of suitable user input devices 1912 include keyboards, keypads, mice, trackballs, digital pens, touch-based interfaces, gesture-based interfaces, word and voice recognition interfaces, adaptive interfaces, cameras, Including but not limited to motion sensing interfaces and other user input devices known to those skilled in the art.

  Some of the user interface components 1900 may be processor executable software stored in non-transitory computer readable storage media and / or such as dedicated microprocessors, FPGAs, microcontrollers, graphics processing units and DSPs. It may be implemented as a hardware component. In some embodiments, a portion of the user interface component 1900 is incorporated into the components of the multi-modality processing system 100 described with reference to FIGS. For example, in some such embodiments, the controller 1910 may be within the bedside controller 118, main controller 120, boom display 122, and / or network console 130 described with reference to FIG. It is a component. As a further example, in some such embodiments, the measurement navigator module 1902 includes the UI framework service 240 of the main controller 120, the bedside controller 118 described with reference to FIG. It is incorporated into a UI extension, such as UI framework service 242 and / or IVUS UI extension 246 or IVUS UI extension 248. In other embodiments, the measurement navigator module 1902 is a separate and separate component of the multi-modality processing system 100.

  The measurement navigator module 1902 receives a reference set of medical data via the medical data interface 1906. In some embodiments, the identifier corresponding to the medical data is received in addition to or as an alternative to receiving the medical data itself. The medical data identifier may include some or all of the medical data and / or may include a pointer to data located anywhere in the system or distributed in the network. The reference data set may also be displayed on the display device (eg, user display device 1914) of the controller 1910 viewed by the operator. Display control and data manipulation may be performed directly by the measurement navigator module 1902 and / or by other components of the multi-modality processing system 100 separate from the measurement navigator module 1902. In some embodiments, the reference data is presented as a graph, such as a timeline or a distance-based plot, but other forms of displaying the data are contemplated, including 2D and 3D images.

  The reference set of medical data may be any suitable medical data. In some embodiments, the medical data includes raw medical data, and a modality acquisition component (eg, a reserve blood flow ratio acquisition component, an iFR acquisition component, a pressure acquisition component, a flow acquisition component, the IVUS acquisition component of FIG. 220, an anterior view IVUS acquisition component, an FFR acquisition component, a CFR acquisition component, an OCT acquisition component, and / or a transesophageal echo acquisition component). In some embodiments, the medical data includes processed medical data and a workflow component (eg, blood flow reserve ratio workflow component, iFR workflow component, pressure workflow component, flow workflow component, IVUS workflow). Component, forward vision IVUS workflow component, FFR workflow component, CFR workflow component, OCT workflow component, and / or transesophageal echocardiographic workflow component). In some embodiments, medical data includes data collected from multiple modalities and may be provided by a multi-modality workflow component.

  Measurement navigator module 1902 includes one or more navigation commands, such as data selection, highlight selection, bookmark commands, zoom in and zoom out commands, and other suitable navigation commands that represent a portion of a reference set of medical data. Also receive. Based on the navigation command and / or the reference set of medical data, the navigator engine 1904 determines a subset of the reference medical data to highlight. This subset may include all or part of the reference set. This subset may also include alternative data sets in addition to or instead of the reference data set. In such embodiments, the data sets included in the subset may also correspond to different modalities, a single modality at different times, a single modality in different operating modes, and other combinations of modalities and operating conditions. Good. For some types of data selection commands, navigation commands may be converted from an initial format to a data format. For example, the navigation command may include a set of Cartesian coordinates corresponding to pixels of the user display device 1914. Thus, in one embodiment, the navigator engine 1904 of the module 1902 converts pixel coordinates to data values. This may include one or more transformations such as scale transformations, coordinate transformations, shape translations and other suitable transformations known to those skilled in the art.

  In some embodiments, the navigation command is an operator command and is received via user input device 1912. In one such embodiment, a gesture interface that communicates with the user input device 1912 and the measurement navigator module 1902 (eg, the gesture recognition module 1202 of FIG. 12) receives the gesture from the user input device 1912 and sends the gesture. Translate to navigation commands and provide the navigation commands to the measurement navigator module 1902. In further such embodiments, the measurement navigator module 1902 incorporates some or all of the gesture recognition module 1202. Navigation commands may also be received from other system components, such as modality workflow components. For example, in the illustrative embodiment, the measurement navigator module 1902 receives a data selection indicating a portion of medical data from the FFR workflow component. This data selection highlights, displays, and / or displays values of interest detected by that workflow component, such as peak FFR values, values that exceed thresholds, and / or values that indicate sensor anomalies. It may be configured to be noted. In another exemplary embodiment, the measurement navigator 1902 receives bookmarks from the FFR workflow component. This bookmark may be configured to provide a reference point for quickly identifying the value of interest detected by the workflow component. Such behavior allows components of the system to allow the operator to focus on potentially important data that may not be noticed.

  As described above, the measurement navigator module 1902 may receive navigator commands that specify one or more enhancement functions to perform. Examples of suitable enhancements include zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution increase / decrease, resampling, interpolation, gain adjustment, and measurements including distance, area, volume and rate of change measurements. Emphasis may also include labeling or annotating portions of the medical data, such as the labeling described with respect to FIGS. Other suitable enhancements include uniaxial zoom. Unlike biaxial zoom, navigator engine 1904 resizes and resamples data along a first axis, such as a time axis, independently of a second axis, such as a pressure axis or velocity axis, or It may be emphasized. In one such embodiment, the reference data set corresponding to the FFR measurement includes a plurality of pressure data points over a range of time. The operator chooses to perform a uniaxial zoom and specifies the start and end time boundaries and the resulting new range. In response, the navigator engine 1904 scales the subset of data represented by the new range along the time axis, but not necessarily along the pressure axis. Instead, in the exemplary embodiment, the pressure axis is resized based on the maximum and minimum pressure values in the new range. In a related embodiment, the navigator engine 1904 scales the subset of data along the time axis, but does not scale the pressure axis from that of the reference set. In a further set of embodiments, the measurement navigator engine 1904 performs a uniaxial zoom of iFR® imaging data. The operator selects uniaxial zoom and start and end time boundaries. The navigator engine 1904 scales the subset represented by the start and end time boundaries along the time axis, but in various embodiments, the pressure axis is scaled independently or relative to the reference set. Do not scale the pressure axis. By supporting uniaxial operation in addition to or instead of biaxial operation, navigator engine 1904 provides a more useful representation of a particular type of data corresponding to a particular modality. In that regard, in various embodiments, the uniaxial operation may be performed so that FFR data, iFR data, pressure data, flow data and / or other suitable data sets are presented so as to present a more useful representation of the enhanced data. Executed with respect to.

  As with data specification commands, commands that specify the type of emphasis to be made may also be received from user input device 1912 and / or from other components of the system. The specified enhancement includes a measurement navigator module 1902 and / or a modality extension including a measurement zoom module 702, a UI framework service 240, a system UI extension 244, an IVUS UI extension 246 and a modality UI extension 250. Because it may be executed by other components of the system, such as departments and / or other suitable components of system 100, measurement navigator module 1902 may forward the highlight selection command to the appropriate system component. For example, in some enhancement implementations, the navigator engine 1904 may combine multiple data sets directly or indirectly to form a subset. Thus, in one embodiment, navigator engine 1904 provides a command to instruct the MMCM component to combine data sets collected by two or more modalities. In a related embodiment, navigator engine 1904 receives data sets collected by two or more modalities and combines within engine 1904. Similarly, in some emphasis implementations, the navigator engine 1904 may select a data set to improve responsiveness and / or reduce the load on storage, networking and / or processing resources. Good. In one such embodiment, navigator engine 1904 selects data outside the selected subset.

  If the type of enhancement to be performed is determined, the measurement navigator module 1902 may subsequently perform enhancement on the subset of medical data instructed by the navigation command. Thereafter, the highlighted subset may be displayed on the user display device. In some embodiments, the reference data and the subset of highlighted data may be displayed on different user display devices (eg, each of the first user display device and the second user display device). . In some embodiments, these display devices correspond to a single physical display device at different times. Thus, the reference data set may be displayed on the monitor at the first time point and the highlighted data may be displayed on the monitor at a later time. In some embodiments, these display devices correspond to a portion of a single display environment on a single display device (such as a “window” of a display “desktop” or an area of a display workspace). make use of). In some embodiments, the first and second display devices correspond to first and second physical displays (eg, first and second monitors, respectively). Displaying reference data and an enhanced subset of data on different user display devices may be particularly useful in applications where user input may obscure some of the display, such as touch screen applications. In some embodiments, the measurement navigator module 1902 updates the display of the reference data set based on the navigation command. This may include adding labels, bookmarks, selected subset boundaries and icons indicating other points of interest. The display may also be an update that includes shading that shows data within or outside the selected subset. As described above, the generation of display signals corresponding to the reference data and the highlighted subset can be performed directly by the measurement navigator module 1902 and / or other components of the multi-modality processing system 100 remote from the measurement navigator module 1902. May be executed.

  The measurement navigator module 1902 may also include an interface for storing transaction records. The transaction record may include a region identifier, highlight selection, data set identifier, part of the reference data set, part of the highlighted subset, modified processing parameters, and / or other appropriate elements of the transaction. Good. Transaction records may be stored or provided in the transaction record database 1908 for transmission and / or storage by components of the system, such as the database management component 216 described in connection with FIG. In some embodiments, the transaction record includes a bookmark that records aspects of the highlight operation that include a description of the data set, region, highlight or other parameter. This bookmark can be retrieved and used to restore a previous state and apply previous data sets, region selections, enhancements and / or parameters to subsequent data sets. This bookmark may serve as a label for indexing, as described above in connection with FIGS. 3-6, or may include such a label.

  FIG. 20 is a diagram of an example user interface 2000 for data navigation and enhancement according to some embodiments of the multi-modality processing system 100. This user interface 2000 may be presented on a single user display or multiple displays. The user interface 2000 shows an effective configuration for displaying information presented by the multi-modality processing system, specifically, the measurement navigator module 1902 of the system. Those skilled in the art will recognize that alternative configurations are contemplated and provided.

  Interface 2000 is a different display device, a single physical display device at different times, different parts of a single display environment on a single display device (a “window” of a display “desktop” or an area of a display workspace). A first data display window 2002 and a second data display window 2004 that may correspond to different physical displays (ie, different monitors). In the illustrated embodiment, the first data display window 2002 presents the reference data set in the first window of the desktop, and the second data display window 2004 displays the reference data set in the second window of the desktop. Present a highlighted subset of. In the illustrated embodiment, the highlighted subset has undergone a uniaxial zoom procedure. In other words, the highlighted subset was scaled along the first axis (time axis) independent of the second axis (pressure axis).

  In some embodiments, one or more of the first data display window 2002 and the second data display window 2004 include a bookmark icon 2006, subset boundary icons 2008 and 2010, and other annotations. The boundary icons 2008 and 2010 may mark the outer boundary of the subset along the axis (in the illustrated embodiment, the time axis). In some embodiments, the subset is identified by shading regions inside or outside the subset.

  Interface 2000 may also include one or more auxiliary display areas 2012 that display medical data, such as values from a reference set of data and / or highlighted subsets. In some embodiments, interface 2000 includes a toolbar 2014. In various such embodiments, the toolbar 2014 is used to select commands for the multi-modality processing system. Exemplary commands include selecting subsets to highlight, selecting highlights to perform, and data store and load commands. The interface may also include a plurality of command elements 2016 for quickly selecting commands to send to the multi-modality processing system. Any command that may be selected from the toolbar is suitable for selection using command element 2016. In that regard, in the exemplary embodiment, interface 2000 is configured to adjust and recover subset boundaries and set bookmarks, label data values, and reference set and / or subset zoom in or zoom out. Command element 2016.

  FIG. 21 is a flow diagram of a method 2100 for navigating and highlighting a set of medical data within a multi-modality processing system according to some embodiments of the present disclosure. It will be appreciated that additional steps may be provided before, during and after the steps of method 2100 and that some of the steps shown may be replaced or eliminated in other embodiments of the method.

  At block 2102, a reference set of medical data is received by the multi-modality processing system 100. The reference set of medical data may be any suitable medical data. For example, the medical data may correspond to raw medical data provided by the modality acquisition component, processed medical data provided by the workflow component, and / or multiple modalities and provided by the multi-modality component Aggregate data may be included. In some embodiments, the identifier corresponding to the medical data is received in addition to or instead of receiving the medical data itself. The medical data identifier may include some or all of the medical data and / or may include a pointer to data located anywhere in the system or distributed in the network. At block 2004, a reference set of medical data may be displayed on the display device.

  At block 2106, a navigation command is received. The navigation command may define a subset to highlight, select the highlight to perform, set and restore a bookmark, give a label, and other suitable navigation -A task may be executed. In some exemplary embodiments, a gesture recognition module (eg, gesture recognition module 1202 of FIG. 12) that is incorporated into and / or communicates with measurement navigator module 1902 is touch-based from user input device 1912. Receive the input sequence. The exemplary touch-based input sequence shows a first touch at a first location and a second touch at a second location at the same time. The gesture recognition module translates the input sequence into commands to perform a uniaxial zoom to the subset corresponding to the region between the first touch position and the second touch position. Thus, the navigation command specifies an area for the subset having a first boundary corresponding to the first touch position and a second boundary corresponding to the second touch position. The gesture recognition module provides translated commands to the measurement navigator module 1902. While uniaxial zoom is provided as an example, any type of data enhancement may be performed using this method.

  At block 2108, the display of the reference set of medical data may be updated based on the received navigation command. For example, the display may be updated to include icons indicating labels, bookmarks, boundaries of selected subsets, and other points of interest. In some embodiments, this display is updated to include shading that shows data internal or external to the selected subset.

  At block 2110, the enhancement to be performed may be identified. Emphasis may be based on one or more navigation commands and / or a reference set of data. At block 2112, a subset of the reference set of data to be enhanced is identified. As with emphasis, the subset may depend on one or more navigation commands and / or a reference set of data. The subset of data may include some or all of the reference set, and may include alternative data sets in addition to or as an alternative to the reference data set. In such embodiments, the data sets included in the subset may also correspond to different modalities, a single modality at different times, a single modality in different operating modes, and other combinations of modalities and operating conditions. Good. In some embodiments, the emphasis includes data registration that corresponds to a plurality of different modalities. For co-registration in different modalities, see US Pat. No. 7,930,014 issued April 19, 2011 under the name “VASCULAR IMAGE CO-REGISTRATION” and “THREE DIMENSIONAL CO-REGISTRATION FOR INTRAVASCULAR”. Further details are disclosed in US Patent Application Publication No. 2007/0038061, issued June 23, 2006, entitled "DIAGNOSIS AND THERAPY", the teachings of which are hereby incorporated by reference in their entirety. Incorporated into the book. In some embodiments, the method includes combining data sets directly or indirectly to form a subset. Similarly, in some embodiments, the method includes selecting a portion of the subset. In one such embodiment, the method includes selecting data outside the selected region.

  At block 2114, enhancement is performed by the system 100 on a subset of the data set. In various embodiments, highlighting is determined by a reference set of one or more navigation commands and / or data. Appropriate enhancements include, but are not limited to, zooming, brightness adjustment, opacity adjustment, color mask adjustment, resolution increase / decrease, resampling, interpolation, gain adjustment, and measurements including distance, area, volume and rate of change measurements. It is not limited. Emphasis may also include labeling or annotating portions of the medical data, such as the labeling described in connection with FIGS. Other emphasis is considered and provided. As a further example, in some embodiments, enhancement includes a uniaxial zoom and the subset of data is resized along the first axis, independent of the second axis.

  At block 2116, the highlighted subset of data is displayed. The reference data and the highlighted subset may be displayed on different user display devices (eg, a first user display device and a second user display device). In some embodiments, these display devices correspond to a single physical display device at different times. Thus, the reference data set may be displayed on the monitor at the first time point and the highlighted subset may be displayed later on the monitor. In some embodiments, these display devices correspond to a portion of a single display environment on a single display device (such as a “window” of a display “desktop” or an area of a display workspace). make use of). In some embodiments, the first and second display devices correspond to first and second physical displays (eg, first and second monitors, respectively).

  At block 2118, the enhanced transaction record is stored. In various embodiments, this record may include region identifiers, highlight selections, data set identifiers, parts of reference data sets, parts of highlighted subsets, modified processing parameters, and / or other suitable transactions. Contains elements. In some embodiments, storing includes providing transaction records at a system interface for a remote storage device. Transaction records may include bookmarks that record aspects of the highlight operation, including descriptions of data sets, regions, highlights or other parameters. This bookmark can be retrieved and used to restore previous states and apply previous data sets, region selections, enhancements and / or parameters to subsequent data sets. This bookmark may serve as a label for the indexing described above in connection with FIGS. 3-6, or may include such a label.

  While exemplary embodiments have been shown and described, extensive modifications, changes and substitutions have been discussed in the above disclosure and some examples, some aspects of the present disclosure are in response to other aspects. It may be employed without use. Further, as described above, the components and extensions described above in connection with the multi-modality processing system may be implemented in hardware, software, or a combination of both. The processing system may be designed to operate on any particular architecture. For example, the system may be implemented in a single computer, local area network, client server network, wide area network, the Internet, portable and other portable and wireless devices and networks. It will be appreciated that such variations may be performed in the foregoing without departing from the scope of the present disclosure. Accordingly, it is appropriate that the claims of this disclosure be construed as broad and consistent with the scope of the specification.

DESCRIPTION OF SYMBOLS 100 Medical system 101 Multi-modality processing system 102 Catheter laboratory 104 Management room 106 Patient 108 Catheter 110 Catheter 116 Electrocardiograph 112 IVUS
114 OCT PIM
117 Angiography System 118 Bedside Controller 120 Main Controller 122 Boom Display 125 Network 130 Network Console 200 Processing Framework 202 System Controller 204 Workflow Controller Component 206 Event Logging Component 208 Resource Arbiter Component 210 Message delivery component 212 Security component 214 MMCM workflow component 216 Database management component 220 IVUS acquisition component 222 IVUS workflow component 224 Modality “N” acquisition component 226 Modality “N” workflow component 228 Modality “N” PIM
230 Co-registration Interface Component 232 Co-registration Workflow Component 234 Data Collection Tool 240 UI Framework Service 242 UI Framework Service 244 System UI Extension 246 IVUS UI Extension 248 IVUS UI Extension 250 Modality N "UI extension 252 Modality" N "UI extension 254 Network 256 Message delivery component 260 Remote access component 262 Remote access workflow component 264 Legacy control component 266 Legacy control workflow component 268 Legacy console 300 User interface Component 300
302 Label Builder Module 304 Controller 306 Label Builder Engine 308 Medical Data Interface 310 Label Set Database 312 Procedure Database 314 User Input Device 316 User Display Device 702 Measurement Zoom Module 704 Controller 706 Measurement Zoom Engine 708 Transaction Record 710 Modality Feedback Interface 712 Medical Data Interface 714 User Input Device 716 First User Display Device 718 Second User Display Device 1202 Gesture Recognition Module 1204 Gesture Recognition Engine 1206 Gesture Database 1208 Module Interface 1210 controller 1212 User input device 1214 User display device 1902 Measurement navigator module 1904 Navigator engine 1906 Medical data interface 1908 Transaction recording 1910 Controller 1912 User input device 1914 User display device

  While exemplary embodiments have been shown and described, extensive modifications, changes and substitutions have been discussed in the above disclosure and some examples, some aspects of the present disclosure are in response to other aspects. It may be employed without use. Further, as described above, the components and extensions described above in connection with the multi-modality processing system may be implemented in hardware, software, or a combination of both. The processing system may be designed to operate on any particular architecture. For example, the system may be implemented in a single computer, local area network, client server network, wide area network, the Internet, portable and other portable and wireless devices and networks. It will be appreciated that such variations may be performed in the foregoing without departing from the scope of the present disclosure. Accordingly, it is appropriate that the claims of this disclosure be construed as broad and consistent with the scope of the specification.
  The present invention includes the following inventions in one aspect.
(Invention 1)
  A method for emphasizing medical data in a medical processing system, comprising:
  Receiving a reference set of medical data by the medical processing system;
  Receiving a region identifier corresponding to the region to be highlighted;
  Receiving a highlight selection,
  Identifying a target set of data based on the received region identifier and the received enhancement selection;
  Performing, by the medical processing system, enhancement corresponding to the enhancement selection for the target set of data; and
  Displaying the highlighted goal set to the user
  A method comprising:
(Invention 2)
  Displaying the reference set of medical data on a first user display;
  Further comprising
  Displaying the enhanced target set includes displaying the enhanced target set of data on a second user display, the second user display including the first user display and the second user display. Is different,
  The method according to invention 1.
(Invention 3)
  The reference set of medical data corresponds to a first modality;
  The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
  The method according to invention 1.
(Invention 4)
  The step of performing the enhancement includes at least one of zooming, increasing resolution, decreasing resolution, resampling, distance measurement, area measurement, volume measurement, and measurement of the rate of change of the target set of data. The method according to invention 1.
(Invention 5)
  The step of performing the enhancement comprises:
  Determining a change in a processing parameter based on at least one of the region identifier and the enhancement selection; and
  Applying the change to the processing parameter;
  A method according to invention 1, comprising:
(Invention 6)
  The method of claim 1, wherein the receiving the region identifier comprises receiving a region identifier input from a user input device.
(Invention 7)
  Receiving the region identifier comprises receiving the region identifier from a workflow component of the medical processing system;
  The region identifier is configured to focus on points of interest in the reference set of medical data;
  The method according to invention 1.
(Invention 8)
  The method of claim 1, wherein receiving the highlight selection comprises receiving a highlight selection input from a user input device.
(Invention 9)
  A method of measuring data in a medical processing system,
  Receiving a reference set of medical data by the medical processing system;
  Receiving a position marker associated with the reference set of medical data;
  Receiving a measurement selection;
  Identifying a target set of data based on the received reference set of medical data, the position marker and the measurement selection;
  Performing, by the medical processing system, a measurement operation corresponding to the measurement selection for the target set of data to determine a measurement value; and
  Displaying the target set of medical data and the measured values to a user;
  A method comprising:
(Invention 10)
  The reference set of medical data corresponds to a first modality;
  The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
  The method according to invention 9.
(Invention 11)
  The method according to claim 9, wherein the performing the measurement operation includes at least one of distance measurement, area measurement, volume measurement, change rate measurement, and pinpoint measurement.
(Invention 12)
  The method of claim 11, wherein the performing the measurement further comprises cross-reference data across a plurality of modalities.
(Invention 13)
  10. The method of claim 9, wherein the performing the measurement includes receiving a transform coefficient and using the transform coefficient to determine the measurement value.
(Invention 14)
  The method of claim 9, wherein the receiving the position marker receives the position marker from a user input device.
(Invention 15)
  Receiving the position marker comprises:
  Receiving user input from a user input device;
  Receiving a target point from a module of the medical processing system;
  Determining the position marker based on the user input and the target point;
  A method according to invention 9, comprising:
(Invention 16)
  An apparatus comprising a non-transitory computer readable medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
  Displaying a reference set of medical data on a first user display;
  Receiving user input for highlighting a portion of the reference set of medical data;
  Determining a target set of data based on the received user input;
  Highlighting a portion of the goal set based on the received user input;
  Display the target set of the highlighted data on a second user display
  For the device.
(Invention 17)
  The reference set of medical data corresponds to a first modality;
  The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
  The device according to invention 16.
(Invention 18)
  The apparatus of claim 16, wherein the medium further stores instructions for storing a highlighted partial transaction record of the target set of data.
(Invention 19)
  The apparatus of claim 18, wherein the medium further stores instructions for using the transaction record to restore a previous state of the computer processor.
(Invention 20)
  The medium is
  Capture a subsequent data set based on the received user input;
  Include the subsequent data set in the target set of data
  An apparatus according to invention 16, further storing instructions for:
(Invention 21)
  The apparatus of claim 16, wherein the medium further stores instructions for selecting data from the target set of data.
(Invention 22)
  The apparatus of claim 21, wherein the instructions for selecting data from the target set of data are configured to select data outside a region of interest corresponding to the received user input.
(Invention 23)
  The apparatus of claim 16, wherein the medium further stores instructions for converting the user input from a region identifier format to a data format.
(Invention 24)
  24. The apparatus of claim 23, wherein the instructions for converting the user input include instructions for performing at least one of scale conversion, coordinate conversion, and shape conversion.
(Invention 25)
  A method for navigating medical data in a medical processing system, comprising:
  Receiving a reference set of medical data by the medical processing system, the medical data corresponding to a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT; , Stage,
  Receiving a navigation command by the medical processing system;
  Identifying the enhancement to be performed based on the navigation command;
  Identifying a subset of the reference set of medical data based on the navigation command;
  Performing the enhancement of the subset by the medical processing system; and
  Displaying the highlighted subset
  A method comprising:
(Invention 26)
  26. The method of invention 25, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality.
(Invention 27)
  Receiving the navigation command comprises:
  Receiving a user input sequence by a gesture recognition module of the medical processing system;
  Translating the user input sequence into the navigation command;
  A method according to invention 25, comprising:
(Invention 28)
  28. The method of invention 27, wherein the user input sequence indicates a first touch-based input simultaneous with a second touch-based input.
(Invention 29)
  29. The invention of claim 28, wherein the navigation command specifies a first boundary for a subset corresponding to the first touch-based input and a second boundary for a subset corresponding to the second touch-based input. the method of.
(Invention 30)
  Displaying a reference set of medical data on a display device; and
  Updating the display of the reference set of medical data on the display device based on a navigation command
  The method of claim 1, further comprising:
(Invention 31)
  32. The method of invention 30, wherein the updating the displaying comprises displaying subset boundaries.
(Invention 32)
  The method of claim 30, wherein the updating the displaying comprises displaying an icon representing a bookmark location.
(Invention 33)
  26. The method of invention 25, further comprising storing the highlighted subset of transaction records.
(Invention 34)
  The transaction record includes a bookmark configured to restore a previous state and apply one of a previous data set, region selection, enhancement, and parameter to a subsequent set of medical data. The method described in 1.
(Invention 35)
  26. The method of invention 25, wherein the performing the enhancement comprises performing a uniaxial zoom on the subset.
(Invention 36)
  An apparatus comprising a non-transitory computer readable storage medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
  Receiving a reference set of medical data corresponding to a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT;
  Receive navigation commands,
  Performing an enhancement on the subset of the reference set specified by the navigation command;
  Providing the highlighted subset for display
  For the device.
(Invention 37)
  The apparatus of claim 36, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality.
(Invention 38)
  The instructions for the reception of the navigation command are:
  Receiving a user input sequence indicative of a first touch-based input simultaneously with a second touch-based input;
  Translating the user input sequence into the navigation command, the navigation command for a first boundary for the subset corresponding to the first touch-based input and for a subset corresponding to the second touch-based input; Specifies the second boundary
  An apparatus according to invention 36, comprising instructions for.
(Invention 39)
  The apparatus of claim 36, wherein the instructions further comprise instructions for providing an update for display of the reference set based on the navigation command.
(Invention 40)
  40. The apparatus of invention 39, wherein the update of the display includes a representation of a boundary of the subset.
(Invention 41)
  40. The apparatus of aspect 39, wherein the update of the display includes an icon representing a bookmark location.
(Invention 42)
  The instructions further include storing a transaction record of the execution of the highlight, wherein the transaction record restores a previous state to a subsequent set of media data, a previous data set, a region selection, The apparatus of aspect 36, comprising a bookmark configured to apply one or more of emphasis and parameters.
(Invention 43)
  The apparatus of claim 36, wherein the instructions for performing the enhancement include instructions for performing a single axis zoom on the subset.
(Invention 44)
  A medical data interface configured to receive a reference set of medical data, wherein the medical data is a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT. Medical data interface corresponding to
  Receive navigation commands from the controller,
  Identify the emphasis to be performed based on the navigation command;
  Identifying a subset of the reference set of medical data to be enhanced based on the navigation command;
  Performing the identified enhancement on the identified subset;
  Providing the highlighted subset for display
With a navigator engine configured to
  Medical treatment system with.
(Invention 45)
  45. The system of invention 44, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality.
(Invention 46)
  Receiving a user input sequence indicative of a first touch-based input simultaneously with a second touch-based input;
  The user input to a navigation command specifying an area for the subset having a first boundary corresponding to the first touch-based input and a second boundary corresponding to the second touch-based input Translate sequences
  45. The system of aspect 44, further comprising a gesture recognition module configured as follows.
(Invention 47)
  45. The system of invention 44, wherein the navigator engine is further configured to provide an update for display of a reference set of the medical data based on the navigation command.
(Invention 48)
  48. The system of invention 47, wherein the update includes a presentation of the subset boundaries.
(Invention 49)
  48. The system of invention 47, wherein the update includes an icon representing a bookmark location.
(Invention 50)
  45. The system of aspect 44, further comprising an electronic database, wherein the navigator engine is further configured to store highlight transaction records identified in the electronic database.
(Invention 51)
  The transaction record includes a bookmark configured to recover a previous state and apply one or more of the previous data set, region selection, emphasis, and parameters to a subsequent set of media data. The system according to claim 50.
(Invention 52)
  45. The system of invention 44, wherein the navigator engine is further configured to perform a uniaxial zoom on the subset.
(Invention 53)
  A method of interpreting user input in a medical processing system comprising:
Receiving a status indicator corresponding to an operating mode of the medical processing system, the operating mode including a value representing a modality selected from the group of modalities consisting of IVUS, OCT, pressure, and flow. ,
  Generating a list of a plurality of active commands based on the received status indicator;
  Receiving a user input sequence from one or more user input devices;
  Correlating the user input sequence to a command in the list of active commands by the medical processing system; and
  Using the command to control operation of a component of the medical processing system;
  A method comprising:
(Invention 54)
  54. The method of invention 53, wherein the list of active commands includes a plurality of commands related to medical measurements.
(Invention 55)
  55. The method of invention 54, wherein the medical measurement comprises one or more measurements selected from the group consisting of length, area and volume.
(Invention 56)
  55. The method of invention 54, wherein the medical measurement comprises one or more measurements selected from the group consisting of a reserve reserve ratio and a coronary reserve reserve.
(Invention 57)
  54. The method of invention 53, wherein the list of multiple active commands includes a subset of commands common to multiple modalities.
(Invention 58)
  54. The method of invention 53, further comprising displaying a visual representation of the command on a user display device prior to the use of the command that controls the operation of the component of the medical processing system.
(Invention 59)
  59. The method of claim 58, further comprising receiving confirmation of the displayed command from the user input device prior to the use of the command that controls the operation of the component of the medical processing system.
(Invention 60)
  54. The method of claim 53, wherein the generating the list of active commands includes querying an electronic database based on the received status indicator.
(Invention 61)
  54. The method of aspect 53, further comprising determining parameters of the command based on elements of the user input sequence.
(Invention 62)
  A method for controlling a medical processing system, comprising:
  Displaying medical data on a user display device, the medical data corresponding to an active modality of the medical processing system, wherein the active modality is selected from a plurality of modalities of the medical processing system; ,
  Determining a list of active commands based on the active modality;
  Receiving a user input sequence from one or more user display devices by the medical processing system, wherein the user input sequence is provided by a user in response to the displayed medical data;
  Adapting the user input sequence to a command selected from the list of active commands; and
  Using the selected command to control operation of a component of the medical processing system;
  A method comprising:
(Invention 63)
  The method of claim 62, wherein the list of active commands includes a subset of commands common to at least two modalities of the plurality of modalities.
(Invention 64)
  The invention 62 further comprises the step of displaying a visual representation of the selected command on the user display device prior to the using the selected command that controls operation of a component of the medical processing system. The method described.
(Invention 65)
  The method of claim 62, wherein said determining the list of active commands includes querying an electronic database.
(Invention 66)
  63. A method according to invention 62, wherein the selected command is a measurement command.
(Invention 67)
  68. The method of invention 66, wherein the measurement command corresponds to one or more measurements selected from the group consisting of length, area and volume.
(Invention 68)
  68. The method of invention 66, wherein the measurement command corresponds to one or more measurements selected from the group consisting of a blood flow reserve ratio and a coronary blood flow reserve.
(Invention 69)
  An apparatus comprising a non-transitory computer readable storage medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
  Determining an operating mode of the medical processing system, the operating mode corresponding to a modality of the medical processing system;
  Generating a list of active commands based on the mode of operation;
  Receiving a user input sequence from one or more user input devices;
  Correlating the user input sequence to commands in the list of active commands;
  Use commands to control the behavior of the medical processing system
For the device.
(Invention 70)
  68. The apparatus of invention 69, wherein the modality is selected from the group consisting of IVUS, OCT, pressure and flow.
(Invention 71)
  The apparatus of claim 69, wherein the list of active commands includes a subset of commands common to at least two modalities of the medical processing system.
(Invention 72)
  The apparatus of claim 69, wherein the instructions further include an instruction to display a visual representation of the command on a user display device prior to using the command to control the behavior of the medical processing system.
(Invention 73)
  70. The apparatus of claim 69, wherein the instructions for generating the list of active commands include instructions for querying an electronic database.
(Invention 74)
  70. The apparatus of claim 69, wherein the instructions further comprise instructions for specifying parameters of the command based on elements of the user input sequence.
(Invention 75)
  The apparatus according to claim 69, wherein the command is a measurement command.
(Invention 76)
  The apparatus of claim 75, wherein the measurement command corresponds to one or more measurements selected from the group consisting of length, area and volume.
(Invention 77)
  The apparatus of claim 75, wherein the measurement command corresponds to one or more measurements selected from the group consisting of a blood flow reserve ratio and a coronary blood flow reserve.
(Invention 78)
  A labeling method in a medical processing system, comprising:
  Receiving medical data labeled by the medical processing system;
  Presenting a first set of labels corresponding to the medical data on a display device;
  Receiving a selection of a first label from the first set of labels from an interface device;
  Presenting a second set of labels corresponding to the medical data on the display device, the second set of labels being based on the first label;
  Receiving a selection of a second label from the second set of labels from the interface device; and
  Labeling the medical data based on the first label and the second label
  A method comprising:
(Invention 79)
  Receiving from the interface device a command to suspend the labeling method until a resume command is received; and
  Receiving from the interface device a command to resume the labeling method;
  The method of invention 78, further comprising:
(Invention 80)
  79. The method of invention 78, wherein the second set of labels is further based on a modality corresponding to the medical data.
(Invention 81)
  79. The method of invention 78, wherein the second set of labels is further based on at least one of an operable course of procedure, patient identification, patient demographics, and patient medical history.
(Invention 82)
  79. The method of invention 78, wherein the target set of labels includes pre-selecting values from the second set of labels prior to the receiving the selection of the second label.
(Invention 83)
  83. The method of invention 82, wherein the preselected value is determined further based on at least one of system defaults, user preferences, operable course of procedure, patient identification, patient demographics and patient medical history. .
(Invention 84)
  83. The method of invention 82, wherein the selection of the second label overrides the preselected value.
(Invention 85)
  An indexing method in a multi-modality medical processing system, comprising:
  Receiving medical data corresponding to a modality by the multi-modality medical processing system;
  Populating a set of labels based on the modality;
  Presenting the set of labels to a user;
  Receiving user input based on the presented set of labels;
  Assigning a label to the medical data based on the received user input; and
  Storing the labeled medical data such that the labeled medical data is indexed based on the assigned label.
  A method comprising:
(Invention 86)
  86. The method of invention 85, wherein the set of labels is further populated based on additional medical data received by the multi-modality medical processing system.
(Invention 87)
  The modality is a first modality;
  The additional medical data corresponds to a second modality;
  The second modality is different from the first modality
  90. A method according to invention 86.
(Invention 88)
  86. The method of invention 85, wherein the set of labels is further populated based on at least one of an operational course of procedure, patient identification, patient demographics, and patient medical history.
(Invention 89)
  86. The method of invention 85, wherein the user input specifies a selected value contained within the set of labels.
(Invention 90)
  86. The method of invention 85, wherein the user input specifies selected values that are not included in the set of labels.
(Invention 91)
  86. The method of invention 85, wherein the value of the set of labels is preselected prior to the receiving step of the user input.
(Invention 92)
  92. The method of invention 91, wherein the received user input invalidates the preselected value from the set of labels.
(Invention 93)
  A method for labeling data in a data processing system comprising:
  Receiving medical data labeled by the data processing system;
  Retrieving procedural information from the first database;
  Retrieving a set of labels for the medical data from a second database, wherein the set of labels is based on the procedural information;
  Presenting the set of labels to a user on a user display;
  Receiving user input based on the presented set of labels from a user interface; and
  Assigning a final label to the medical data based on the received user input
  A method comprising:
(Invention 94)
  94. The method of invention 93, wherein the set of labels is further based on a previous label applied to a previous set of medical data.
(Invention 95)
  94. The method of invention 93, wherein the set of labels is further based on previous labels selected for the medical data.
(Invention 96)
  94. The method of invention 93, wherein the user input includes selected values included in the set of labels.
(Invention 97)
  94. The method of invention 93, wherein the user input includes selected values that are not included in the set of labels.
(Invention 98)
  94. The method of aspect 93, wherein presenting the set of labels comprises pre-selecting a value from the set of labels prior to the receiving of the user input.
(Invention 99)
  99. The method of invention 98, wherein the received user input invalidates the preselected value from the set of labels.

Claims (99)

A method for emphasizing medical data in a medical processing system, comprising:
Receiving a reference set of medical data by the medical processing system;
Receiving a region identifier corresponding to the region to be highlighted;
Receiving a highlight selection,
Identifying a target set of data based on the received region identifier and the received enhancement selection;
Performing, with the medical processing system, enhancement corresponding to the enhancement selection for the target set of data, and displaying the enhanced target set to a user. Displaying the reference set of medical data on a first user display;
Displaying the enhanced target set includes displaying the enhanced target set of data on a second user display, the second user display including the first user display and the second user display. Is different,
The method of claim 1. The reference set of medical data corresponds to a first modality;
The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
The method of claim 1.   The step of performing the enhancement includes at least one of zooming, increasing resolution, decreasing resolution, resampling, distance measurement, area measurement, volume measurement, and measurement of the rate of change of the target set of data. The method of claim 1. The step of performing the enhancement comprises:
The method of claim 1, comprising: determining a process parameter change based on at least one of the region identifier and the enhancement selection; and applying the change to the process parameter.   The method of claim 1, wherein the receiving the region identifier comprises receiving a region identifier input from a user input device. Receiving the region identifier comprises receiving the region identifier from a workflow component of the medical processing system;
The region identifier is configured to focus on points of interest in the reference set of medical data;
The method of claim 1.   The method of claim 1, wherein receiving the highlight selection comprises receiving a highlight selection input from a user input device. A method of measuring data in a medical processing system,
Receiving a reference set of medical data by the medical processing system;
Receiving a position marker associated with the reference set of medical data;
Receiving a measurement selection;
Identifying a target set of data based on the received reference set of medical data, the position marker and the measurement selection;
Performing a measurement operation corresponding to the measurement selection on the target set of data to determine a measurement value by the medical processing system; and displaying the target set of medical data and the measurement value to a user A method comprising steps. The reference set of medical data corresponds to a first modality;
The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
The method of claim 9.   The method of claim 9, wherein the performing the measurement operation includes at least one of a distance measurement, an area measurement, a volume measurement, a change rate measurement, and a pinpoint measurement.   The method of claim 11, wherein the performing the measurement further includes cross-reference data across a plurality of modalities.   The method of claim 9, wherein the performing the measurement includes receiving a transform coefficient and using the transform coefficient to determine the measurement.   The method of claim 9, wherein the receiving the position marker receives the position marker from a user input device. Receiving the position marker comprises:
Receiving user input from a user input device;
Receiving a target point from a module of the medical processing system;
10. The method of claim 9, comprising determining the position marker based on the user input and the point of interest. An apparatus comprising a non-transitory computer readable medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
Displaying a reference set of medical data on a first user display;
Receiving user input for highlighting a portion of the reference set of medical data;
Determining a target set of data based on the received user input;
Highlighting a portion of the goal set based on the received user input;
A device for displaying the highlighted target set of data on a second user display. The reference set of medical data corresponds to a first modality;
The target set of data includes data corresponding to a second modality, the second modality being different from the first modality;
The apparatus of claim 16.   The apparatus of claim 16, wherein the medium further stores instructions for storing a highlighted partial transaction record of the target set of data.   The apparatus of claim 18, wherein the medium further stores instructions for using the transaction record to restore a previous state of the computer processor. The medium is
Capture a subsequent data set based on the received user input;
The apparatus of claim 16, further storing instructions for including the subsequent data set in the target set of data.   The apparatus of claim 16, wherein the medium further stores instructions for selecting data from the target set of data.   The apparatus of claim 21, wherein the instructions for selecting data from the target set of data are configured to select data outside a region of interest corresponding to the received user input.   The apparatus of claim 16, wherein the medium further stores instructions for converting the user input from a region identifier format to a data format.   24. The apparatus of claim 23, wherein the instructions for converting the user input include instructions for performing at least one of scale conversion, coordinate conversion, and shape conversion. A method for navigating medical data in a medical processing system, comprising:
Receiving a reference set of medical data by the medical processing system, the medical data corresponding to a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT; , Stage,
Receiving a navigation command by the medical processing system;
Identifying the enhancement to be performed based on the navigation command;
Identifying a subset of the reference set of medical data based on the navigation command;
Performing the enhancement of the subset by the medical processing system; and displaying the enhanced subset.   26. The method of claim 25, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality. Receiving the navigation command comprises:
Receiving a user input sequence by a gesture recognition module of the medical processing system;
26. The method of claim 25, comprising translating the user input sequence into the navigation command.   28. The method of claim 27, wherein the user input sequence indicates a first touch-based input simultaneous with a second touch-based input.   29. The navigation command according to claim 28, wherein the navigation command specifies a first boundary for a subset corresponding to the first touch-based input and a second boundary for a subset corresponding to the second touch-based input. The method described. The method of claim 1, further comprising: displaying a reference set of medical data on a display device; and updating the display of the reference set of medical data on the display device based on a navigation command. .   31. The method of claim 30, wherein the updating the displaying comprises displaying subset boundaries.   32. The method of claim 30, wherein the updating the displaying comprises displaying an icon representing a bookmark location.   26. The method of claim 25, further comprising storing the highlighted subset of transaction records.   The transaction record includes a bookmark configured to restore a previous state and apply one of a previous data set, region selection, enhancement, and parameter to a subsequent set of medical data. 34. The method according to 33.   26. The method of claim 25, wherein the performing the enhancement comprises performing a uniaxial zoom on the subset. An apparatus comprising a non-transitory computer readable storage medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
Receiving a reference set of medical data corresponding to a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT;
Receive navigation commands,
Performing an enhancement on the subset of the reference set specified by the navigation command;
An apparatus for providing said highlighted subset for display.   38. The apparatus of claim 36, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality. The instructions for the reception of the navigation command are:
Receiving a user input sequence indicative of a first touch-based input simultaneously with a second touch-based input;
Translating the user input sequence into the navigation command, the navigation command for a first boundary for the subset corresponding to the first touch-based input and for a subset corresponding to the second touch-based input; 37. The apparatus of claim 36, comprising instructions for designating a second boundary.   37. The apparatus of claim 36, wherein the instructions further comprise instructions for providing an update for display of the reference set based on the navigation command.   40. The apparatus of claim 39, wherein the update of the display includes a representation of a boundary of the subset.   40. The apparatus of claim 39, wherein the update of the display includes an icon representing a bookmark location.   The instructions further include storing a transaction record of the execution of the highlight, wherein the transaction record restores a previous state to a subsequent set of media data, a previous data set, a region selection, 38. The apparatus of claim 36, comprising a bookmark configured to apply one or more of emphasis and parameters.   37. The apparatus of claim 36, wherein the instructions for performing enhancement include instructions for performing a single axis zoom on the subset. A medical data interface configured to receive a reference set of medical data, wherein the medical data is a first modality selected from the group of modalities consisting of FFR, iFR, pressure, flow, IVUS and OCT. Medical data interface corresponding to
Receive navigation commands from the controller,
Identify the emphasis to be performed based on the navigation command;
Identifying a subset of the reference set of medical data to be enhanced based on the navigation command;
Performing the identified enhancement on the identified subset;
A medical processing system comprising: a navigator engine configured to provide the highlighted subset for display.   45. The system of claim 44, wherein the navigation command specifies an additional set of medical data corresponding to a second modality, wherein the second modality is different from the first modality. Receiving a user input sequence indicative of a first touch-based input simultaneously with a second touch-based input;
The user input to a navigation command specifying an area for the subset having a first boundary corresponding to the first touch-based input and a second boundary corresponding to the second touch-based input 45. The system of claim 44, further comprising a gesture recognition module configured to translate the sequence.   45. The system of claim 44, wherein the navigator engine is further configured to provide an update for display of a reference set of the medical data based on the navigation command.   48. The system of claim 47, wherein the update includes a presentation of the subset boundaries.   48. The system of claim 47, wherein the update includes an icon representing a bookmark location.   45. The system of claim 44, further comprising an electronic database, wherein the navigator engine is further configured to store highlight transaction records identified in the electronic database.   The transaction record includes a bookmark configured to recover a previous state and apply one or more of the previous data set, region selection, emphasis, and parameters to a subsequent set of media data. 51. The system of claim 50.   45. The system of claim 44, wherein the navigator engine is further configured to perform uniaxial zoom on the subset. A method of interpreting user input in a medical processing system comprising:
Receiving a status indicator corresponding to an operating mode of the medical processing system, the operating mode including a value representing a modality selected from the group of modalities consisting of IVUS, OCT, pressure, and flow. ,
Generating a list of a plurality of active commands based on the received status indicator;
Receiving a user input sequence from one or more user input devices;
Correlating the user input sequence with a command in the list of active commands by the medical processing system; and using the command to control operation of a component of the medical processing system. Method.   54. The method of claim 53, wherein the list of active commands includes a plurality of commands related to medical measurements.   55. The method of claim 54, wherein the medical measurement comprises one or more measurements selected from the group consisting of length, area and volume.   55. The method of claim 54, wherein the medical measurement comprises one or more measurements selected from the group consisting of a reserve blood flow ratio and a coronary flow reserve.   54. The method of claim 53, wherein the list of active commands includes a subset of commands common to a plurality of modalities.   54. The method of claim 53, further comprising displaying a visual representation of the command on a user display device prior to the use of the command that controls the operation of the component of the medical processing system.   59. The method of claim 58, further comprising receiving confirmation of the displayed command from the user input device prior to the use of the command that controls the operation of the component of the medical processing system.   54. The method of claim 53, wherein the generating the list of active commands comprises querying an electronic database based on the received status indicator.   54. The method of claim 53, further comprising determining parameters of the command based on elements of the user input sequence. A method for controlling a medical processing system, comprising:
Displaying medical data on a user display device, the medical data corresponding to an active modality of the medical processing system, wherein the active modality is selected from a plurality of modalities of the medical processing system; ,
Determining a list of active commands based on the active modality;
Receiving a user input sequence from one or more user display devices by the medical processing system, wherein the user input sequence is provided by a user in response to the displayed medical data;
Adapting the user input sequence to a command selected from the list of active commands, and using the selected command to control operation of a component of the medical processing system.   64. The method of claim 62, wherein the list of active commands includes a subset of commands common to at least two modalities of the plurality of modalities.   63. The method further comprising displaying a visual representation of the selected command on the user display device prior to the using the selected command that controls operation of a component of the medical processing system. The method described in 1.   64. The method of claim 62, wherein the step of determining the list of active commands includes querying an electronic database.   64. The method of claim 62, wherein the selected command is a measurement command.   68. The method of claim 66, wherein the measurement command corresponds to one or more measurements selected from the group consisting of length, area and volume.   68. The method of claim 66, wherein the measurement command corresponds to one or more measurements selected from the group consisting of a blood flow reserve ratio and a coronary blood flow reserve. An apparatus comprising a non-transitory computer readable storage medium storing a plurality of instructions for execution by at least one computer processor, the instructions comprising:
Determining an operating mode of the medical processing system, the operating mode corresponding to a modality of the medical processing system;
Generating a list of active commands based on the mode of operation;
Receiving a user input sequence from one or more user input devices;
Correlating the user input sequence to commands in the list of active commands;
An apparatus for using commands to control the behavior of the medical processing system.   70. The apparatus of claim 69, wherein the modality is selected from the group consisting of IVUS, OCT, pressure and flow.   70. The apparatus of claim 69, wherein the list of active commands includes a subset of commands common to at least two modalities of the medical processing system.   70. The apparatus of claim 69, wherein the instructions further comprise instructions for displaying a visual representation of the commands on a user display device prior to using the commands that control the behavior of the medical processing system.   70. The apparatus of claim 69, wherein the instructions for generating the list of active commands include instructions for querying an electronic database.   70. The apparatus of claim 69, wherein the instructions further comprise instructions for specifying parameters of the command based on elements of the user input sequence.   70. The apparatus of claim 69, wherein the command is a measurement command.   The apparatus of claim 75, wherein the measurement command corresponds to one or more measurements selected from the group consisting of length, area and volume.   76. The apparatus of claim 75, wherein the measurement command corresponds to one or more measurements selected from the group consisting of a blood flow reserve ratio and a coronary flow reserve. A labeling method in a medical processing system, comprising:
Receiving medical data labeled by the medical processing system;
Presenting a first set of labels corresponding to the medical data on a display device;
Receiving a selection of a first label from the first set of labels from an interface device;
Presenting a second set of labels corresponding to the medical data on the display device, the second set of labels being based on the first label;
Receiving a selection of a second label from the second set of labels from the interface device; and labeling the medical data based on the first label and the second label. Method. 79. The method of claim 78, further comprising: receiving from the interface device a command to suspend the labeling method until a resume command is received; and receiving a command from the interface device to resume the labeling method. The method described.   79. The method of claim 78, wherein the second set of labels is further based on a modality corresponding to the medical data.   79. The method of claim 78, wherein the second set of labels is further based on at least one of an operable course of procedures, patient identification, patient demographics, and patient medical history.   79. The method of claim 78, wherein the target set of labels includes pre-selecting values from the second set of labels prior to the receiving the selection of the second label.   83. The preselected value is determined further based on at least one of system defaults, user preferences, operable courses of procedure, patient identification, patient demographics and patient medical history. Method.   83. The method of claim 82, wherein the selection of the second label overrides the preselected value. An indexing method in a multi-modality medical processing system, comprising:
Receiving medical data corresponding to a modality by the multi-modality medical processing system;
Populating a set of labels based on the modality;
Presenting the set of labels to a user;
Receiving user input based on the presented set of labels;
Assigning a label to the medical data based on the received user input; and labeling the medical data so that the labeled medical data is indexed based on the assigned label. A method comprising the step of memorizing.   86. The method of claim 85, wherein the set of labels is further populated based on additional medical data received by the multi-modality medical processing system. The modality is a first modality;
The additional medical data corresponds to a second modality;
88. The method of claim 86, wherein the second modality is different from the first modality.   86. The method of claim 85, wherein the set of labels is further populated based on at least one of an operable course of procedure, patient identification, patient demographics, and patient medical history.   86. The method of claim 85, wherein the user input specifies a selected value contained within the set of labels.   86. The method of claim 85, wherein the user input specifies a selected value that is not included in the set of labels.   86. The method of claim 85, wherein the value of the set of labels is preselected prior to the receiving step of the user input.   92. The method of claim 91, wherein the received user input invalidates the preselected value from the set of labels. A method for labeling data in a data processing system comprising:
Receiving medical data labeled by the data processing system;
Retrieving procedural information from the first database;
Retrieving a set of labels for the medical data from a second database, wherein the set of labels is based on the procedural information;
Presenting the set of labels to a user on a user display;
Receiving a user input based on the presented set of labels from a user interface; and assigning a final label to the medical data based on the received user input.   94. The method of claim 93, wherein the set of labels is further based on previous labels applied to a previous set of medical data.   94. The method of claim 93, wherein the set of labels is further based on previous labels selected for the medical data.   94. The method of claim 93, wherein the user input includes a selected value included within the set of labels.   94. The method of claim 93, wherein the user input includes selected values that are not included in the set of labels.   94. The method of claim 93, wherein presenting the set of labels comprises pre-selecting a value from the set of labels prior to the receiving of the user input.   99. The method of claim 98, wherein the received user input invalidates the preselected value from the set of labels.