WO2025051987A1 - Surgical data insight system - Google Patents
Surgical data insight system Download PDFInfo
- Publication number
- WO2025051987A1 WO2025051987A1 PCT/EP2024/075041 EP2024075041W WO2025051987A1 WO 2025051987 A1 WO2025051987 A1 WO 2025051987A1 EP 2024075041 W EP2024075041 W EP 2024075041W WO 2025051987 A1 WO2025051987 A1 WO 2025051987A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- surgical
- case
- data
- viewer
- computer
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 94
- 238000004458 analytical method Methods 0.000 claims abstract description 37
- 238000001356 surgical procedure Methods 0.000 claims description 39
- 238000004590 computer program Methods 0.000 claims description 11
- 206010019909 Hernia Diseases 0.000 claims description 5
- 230000000740 bleeding effect Effects 0.000 claims description 3
- 230000007547 defect Effects 0.000 claims description 3
- 238000003860 storage Methods 0.000 description 33
- 238000013480 data collection Methods 0.000 description 16
- 238000013523 data management Methods 0.000 description 15
- 238000012545 processing Methods 0.000 description 15
- 238000010586 diagram Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 11
- 238000004891 communication Methods 0.000 description 8
- 230000008569 process Effects 0.000 description 8
- 230000009471 action Effects 0.000 description 7
- 238000012805 post-processing Methods 0.000 description 7
- 210000003484 anatomy Anatomy 0.000 description 6
- 238000012552 review Methods 0.000 description 5
- 230000005540 biological transmission Effects 0.000 description 4
- 238000013500 data storage Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 239000000203 mixture Substances 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 4
- 238000003491 array Methods 0.000 description 3
- 238000013473 artificial intelligence Methods 0.000 description 3
- 230000008901 benefit Effects 0.000 description 3
- 238000002224 dissection Methods 0.000 description 3
- 238000012549 training Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 230000008878 coupling Effects 0.000 description 2
- 238000010168 coupling process Methods 0.000 description 2
- 238000005859 coupling reaction Methods 0.000 description 2
- 238000013461 design Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 239000012636 effector Substances 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000001902 propagating effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000007704 transition Effects 0.000 description 2
- RYGMFSIKBFXOCR-UHFFFAOYSA-N Copper Chemical compound [Cu] RYGMFSIKBFXOCR-UHFFFAOYSA-N 0.000 description 1
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 229910052802 copper Inorganic materials 0.000 description 1
- 239000010949 copper Substances 0.000 description 1
- 238000012517 data analytics Methods 0.000 description 1
- 238000013075 data extraction Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000023597 hemostasis Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 230000000670 limiting effect Effects 0.000 description 1
- 230000003211 malignant effect Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 230000005055 memory storage Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 238000013439 planning Methods 0.000 description 1
- 230000036316 preload Effects 0.000 description 1
- 230000009467 reduction Effects 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 238000010079 rubber tapping Methods 0.000 description 1
- 238000007789 sealing Methods 0.000 description 1
- 238000004088 simulation Methods 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000000153 supplemental effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
- G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
- G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
Definitions
- the present disclosure relates in general to computing technology and relates more particularly to computing technology for a surgical data insight system.
- a computer-implemented method includes prepopulating surgical data in a surgical data insight viewer, the surgical data associated with one or more surgical cases accessible to a user based on permissions of the user and providing the user with access through the surgical data insight viewer to the surgical data in combination with surgical device data recorded for the one or more surgical cases.
- the method also includes displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data and providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
- a system includes a memory system and one or more processors coupled to the memory system and configured to execute a plurality of instructions to perform a plurality of operations.
- the operations include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases and displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data.
- the operations also include providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
- a computer program product includes a memory device having computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform a plurality of operations.
- the operations include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases, displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data, and providing a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
- FIG. 1 depicts a computer-assisted surgery (CAS) system according to one or more aspects
- FIG. 2 depicts a surgical procedure system according to one or more aspects
- FIG. 3 A depicts a portion of a user interface providing case summary data according to one or more aspects
- FIG. 3B depicts another portion of a user interface providing case summary data according to one or more aspects
- FIG. 3C depicts a further portion of a user interface providing case summary data according to one or more aspects
- FIG. 4A depicts a portion of a user interface providing case details according to one or more aspects
- FIG. 4B depicts another portion of a user interface providing case details according to one or more aspects
- FIG. 4C depicts a further portion of a user interface providing case details according to one or more aspects
- FIG. 6 depicts a user interface of case details according to one or more aspects
- FIG. 7 depicts a user interface of a video playback and workflow list viewer according to one or more aspects
- FIG. 8 depicts a user interface of a video playback and workflow timeline viewer according to one or more aspects
- FIG. 9 depicts a user interface of a video playback and expanded workflow timeline viewer according to one or more aspects
- FIG. 10 depicts a user interface of phase analysis according to one or more aspects
- FIG. 11 A depicts a portion of a user interface of multiple procedure insights according to one or more aspects
- FIG. 1 IB depicts a portion of a user interface of multiple procedure insights according to one or more aspects
- FIG. 12 depicts a user interface of a procedure volume view according to one or more aspects
- FIG. 13 A depicts a portion of a user interface of multiple case procedure analysis according to one or more aspects
- FIG. 13B depicts a portion of a user interface of multiple case procedure analysis of instrument usage according to one or more aspects
- FIG. 13C depicts a portion of a user interface of multiple case procedure analysis with performance and outlier case data according to one or more aspects
- FIG. 14 depicts a flowchart of a method according to one or more aspects.
- FIG. 15 depicts a block diagram of a computer system according to one or more aspects.
- Exemplary aspects of the technical solutions described herein include systems and methods for surgical data insights.
- Technical challenges of surgical data viewing can include collecting data from multiple sources, such as video data, instrument data, patient data, surgical team data, and historical data and presenting the information in a meaningful way.
- data may be collected with a different time base and format that can be challenging to align in time to coordinate the associated information.
- the associated data can be partitioned into case records with index information generated to allow for various performance statistics to be computed efficiently across case types, surgical groups, specific users, geographic locations, patient groups, and other such groupings. Having the associated information available and ready for viewing as close in time to surgical procedure completion can be challenging.
- Controlling data collection, tag generation, and other aspects during a surgical procedure can allow the resulting data sets to be processed more rapidly to coordinate and display the associated information without manual intervention.
- User interfaces of a surgical data insight viewer can be designed for ease of navigation through large volumes of surgical data by providing a combination of summary data, detailed data, and various analytics for performance comparison.
- Technical solutions are described herein to address such technical challenges. Particularly, technical solutions herein provide surgical data insight generation and display.
- the CAS system 100 includes at least a computing system 102, a video recording system 104, and a surgical instrumentation system 106.
- an actor 112 can be medical personnel that uses the CAS system 100 to perform a surgical procedure on a patient 110. Medical personnel can be a surgeon, assistant, nurse, administrator, or any other actor that interacts with the CAS system 100 in a surgical environment. The surgical procedure can be any type of surgery. In other examples, actor 112 can be a technician, an administrator, an engineer, or any other such personnel that interacts with the CAS system 100.
- actor 112 can record data from the CAS system 100, configure/update one or more attributes of the CAS system 100, review past performance of the CAS system 100, repair the CAS system 100, and/or the like including combinations and/or multiples thereof.
- a surgical procedure can include multiple phases, and each phase can include one or more surgical actions.
- a “surgical action” can include an incision, a compression, a stapling, a clipping, a suturing, a cauterization, a sealing, or any other such actions performed to complete a phase in the surgical procedure.
- a “phase” represents a surgical event that is composed of a series of steps (e.g., closure).
- a “step” refers to the completion of a named surgical objective (e.g., hemostasis).
- certain surgical instruments 108 e.g., forceps
- a particular anatomical structure of the patient may be the target of the surgical action(s).
- the video recording system 104 includes one or more cameras 105, such as operating room cameras, endoscopic cameras, and/or the like including combinations and/or multiples thereof.
- the cameras 105 capture video data of the surgical procedure being performed.
- the video recording system 104 includes one or more video capture devices that can include cameras 105 placed in the surgical room to capture events surrounding (i.e., outside) the patient being operated upon.
- the video recording system 104 further includes cameras 105 that are passed inside (e.g., endoscopic cameras) the patient 110 to capture endoscopic data.
- the endoscopic data provides video and images of the surgical procedure.
- the computing system 102 includes one or more memory devices, one or more processors, a user interface device, among other components. All or a portion of the computing system 102 shown in FIG. 1 can be implemented for example, by all or a portion of computer system 1500 of FIG. 15. Computing system 102 can execute one or more computer-executable instructions. The execution of the instructions facilitates the computing system 102 to perform one or more methods, including those described herein.
- the computing system 102 can communicate with other computing systems via a wired and/or a wireless network.
- a data collection system 150 can be employed to store the surgical data, including the video(s) captured during the surgical procedures.
- the data collection system 150 includes one or more storage devices 152.
- the data collection system 150 can be a local storage system, a cloud-based storage system, or a combination thereof. Further, the data collection system 150 can use any type of cloud-based storage architecture, for example, public cloud, private cloud, hybrid cloud, and/or the like including combinations and/or multiples thereof.
- the data collection system can use a distributed storage, i.e., the storage devices 152 are located at different geographic locations.
- the storage devices 152 can include any type of electronic data storage media used for recording machine-readable data, such as semiconductor-based, magnetic-based, optical-based storage media, and/or the like including combinations and/or multiples thereof.
- the data storage media can include flash-based solid-state drives (SSDs), magnetic-based hard disk drives, magnetic tape, optical discs, and/or the like including combinations and/or multiples thereof.
- the data collection system 150 can be part of the video recording system 104, or vice-versa.
- the data collection system 150, the video recording system 104, and the computing system 102 can communicate with each other via a communication network, which can be wired, wireless, or a combination thereof.
- the communication between the systems can include the transfer of data (e.g., video data, instrumentation data, and/or the like including combinations and/or multiples thereof), data manipulation commands (e.g., browse, copy, paste, move, delete, create, compress, and/or the like including combinations and/or multiples thereof), data manipulation results, and/or the like including combinations and/or multiples thereof.
- the computing system 102 can manipulate the data already stored/being stored in the data collection system 150. Alternatively, or in addition, the computing system 102 can manipulate the data already stored/being stored in the data collection system 150 based on information from the surgical instrumentation system 106.
- the video captured by the video recording system 104 is stored on the data collection system 150.
- the computing system 102 curates parts of the video data being stored on the data collection system 150.
- the computing system 102 filters the video captured by the video recording system 104 before it is stored on the data collection system 150.
- the computing system 102 filters the video captured by the video recording system 104 after it is stored on the data collection system 150.
- Instrument data e.g., robotic logs, electrosurgical instrument logs, etc.
- a surgical data management system 160 can provide access to portions of data captured in the data collection system 150, as well as data and records stored in other systems.
- the surgical data management system 160 can establish user access permissions to patient and surgical data.
- the surgical data management system 160 can also control access through an interface based on the user access permissions. Access to the surgical data management system 160 can be provided through one or more applications or secure web pages.
- the surgical data management system 160 can be a stand-alone application, module, and/or an extension of another system. Additional aspects of the surgical data management system 160 can include accessing artificial intelligence (Al)-powered surgical video and analytics. Further aspects of the surgical data management system 160 can include accessing simulation materials that can assist surgeons to prepare, practice, and teach surgical procedures. Further aspects of the surgical data management system 160 can include integrating aspects of equipment in an operating room, surgery planning, rating surgeon performance, and other such features.
- the surgical data management system 160 can also provide access to technical specifications and information relating to the use of surgical instruments, for example.
- FIG. 2 a surgical procedure system 200 is generally shown according to one or more aspects.
- the example of FIG. 2 depicts a surgical procedure support system 202 that can include or may be coupled to the CAS system 100 of FIG. 1.
- the surgical procedure support system 202 can acquire image or video data using one or more cameras 204.
- the surgical procedure support system 202 can also interface with one or more sensors 206 and/or one or more effectors 208.
- the sensors 206 may be associated with surgical support equipment and/or patient monitoring.
- the effectors 208 can be robotic components or other equipment controllable through the surgical procedure support system 202.
- the surgical procedure support system 202 can also interact with one or more user interfaces 210, such as various input and/or output devices.
- the surgical procedure support system 202 can store, access, and/or update surgical data 214 associated with a training dataset and/or live data as a surgical procedure is being performed on patient 110 of FIG. 1.
- the surgical procedure support system 202 can store, access, and/or update surgical objectives 216 to assist in training and guidance for one or more surgical procedures.
- User configurations 218 can track and store user preferences.
- the surgical procedure support system 202 can also communicate with other systems through a network 230.
- the surgical procedure support system 202 can communicate with a surgical data insight viewer 240 and a surgical data postprocessing system 250 through the network 230.
- Other types of devices such as a computing device 234 (e.g., a mobile phone, laptop, personal computer, or tablet computer), can communicate directly with the surgical procedure support system 202 or through the network 230.
- user interfaces 210 may be connected to or integrated with the surgical procedure support system 202 by a wired connection while the computing device 234 connects to the surgical procedure support system 202 via a wireless connection.
- the computing device 234 can execute or link to another computer system that executes the surgical data management system 160 of FIG.
- the surgical data post-processing system 250 can receive surgical data and associated data generated by the surgical procedure support system 202 and may be separately stored and secured through other data storage. Access to specific data or portions of data through the surgical data post-processing system 250 may be limited by associated permissions.
- the surgical data post-processing system 250 may include features such as video viewing, video sharing, data analytics, and selective data extraction.
- the surgical data insight viewer 240 can provide viewing access to various data sources, such as surgical data 214, data collected in the one or more storage devices 152 of FIG. 1, and post-processed data generated by the surgical data post-processing system 250.
- surgical instrument data, video data, and artificial intelligence generated data can be accessed for viewing through the surgical data insight viewer 240.
- the surgical data post-processing system 250 may generate surgical performance metrics and comparison data across data sets collected at multiple locations, making analytics data available to the surgical data insight viewer 240.
- the surgical data insight viewer 240 can preload case data for more than one case of a user, for example, when the user performs a login or otherwise activates the surgical data insight viewer 240.
- the surgical data insight viewer 240 and/or surgical data post-processing system 250 can be components of the surgical data management system 160 of FIG. 1.
- One or more computing device 264 can execute the surgical data management system 160 of FIG. 1 to access various data sources through a network 260.
- the network 230 may be within a facility or multiple facilities maintained within a private network.
- the network 260 may be a wider area network, such as the internet. Accordingly, the networks 230 and 260 may have access to different files and data sets along with shared access to select files and data sets. In some aspects, networks 230 and 260 can be combined.
- the surgical data management system 160 of FIG. 1 can provide multiple user interfaces and functions through the surgical data insight viewer 240, such as those described with respect to FIGS. 3A-13.
- Various user interfaces can be designed for specific types of devices, such as handheld mobile devices, for efficient navigation and display of content. Variations in the features and flow of transitions between user interfaces and content accessible through user interfaces can exist, for example, with respect to interactive interfaces for case summary data, case details, case tags, video viewers with supplemental information, phase analysis, insights, procedure volume, procedure analysis, performance metrics, instrument usage, outlier case identification, and other such aspects.
- FIGS. 3 A, 3B, and 3C depict portions of a user interface providing case summary data according to one or more aspects.
- FIG. 3 A depicts an example of a home screen 300, which can summarize aspects of a selected case.
- a user may scroll to user interface portions 320, 340 to view other aspects, such as a performance comparison for a same type of procedure over a period of time, case volume information, and case selection. Further, the user may select navigation features to see greater detail about the selected case, procedure analysis information, case volume information, further insights, a learning portal, profile information, and/or other content.
- a surgeon may access the home screen 300 upon completing a surgical procedure to review the case and make notes.
- the case can be preloaded for viewing through the surgical data insight viewer 240, for instance, along with the last five cases for immediate selection.
- Data e.g., video, robotic logs, electrical instrument logs, etc.
- Summary information can be indexed/linked and available for immediate access through the surgical data insight viewer 240.
- Summary information such as date and time, along with metrics such as active console time, case duration and difference from personal average can appear on the home screen 300 as a case summary card.
- the home screen 300 can also provide access to setup/configure user preferences for displaying content and generating output for various metrics and reporting tools.
- User interface portion 340 can display case volume information, such as a total number of cases that have associated data along with graphical and/or numeric distributions of the case volume, such as a total number of laparoscopic cases, robotic cases, cases with trainees, and other such categories.
- Case volume information such as a total number of cases that have associated data along with graphical and/or numeric distributions of the case volume, such as a total number of laparoscopic cases, robotic cases, cases with trainees, and other such categories.
- High-level case summaries can be viewable and selectable through user interface portion 340, for instance, to display associate surgical videos and metrics for particular surgeries.
- Information such as case name, a representative image, date, duration, and case tags can also be displayed.
- Case tags can include procedure categories (e.g., complex, training, etc.), patient categories (e.g., body mass range), outcome categories (e.g., benign, malignant, etc.), and/or other information that may assist with grouping or searching for cases.
- the video with a workflow can also be accessed from user interface portion 400 to display, for instance, a user interface 700 of a video playback and workflow list viewer of FIG. 7, a user interface 800 of a video playback and workflow timeline viewer of FIG. 8, and/or a user interface 900 of a video playback and expanded workflow timeline viewer of FIG. 9.
- Features, such as timelines of the user interface 900 can be displayed or hidden in response to user selection to allow users to further customize individual user experience.
- a video thumbnail or preview image may be displayed on user interface portion 400 prior to displaying a video.
- a preview image may be selectable to trigger starting of video playback along with another button that may also trigger video playback.
- Summary metrics can be displayed on user interface portions 400, 420, 440 to help identify how the selected case differs from others and whether it may be worthwhile reviewing.
- Summary metrics can be derived from multiple data sources. For instance, console time can be derived from robotics logs, camera out time can be derived from video, and defect size can be derived from video using artificial intelligence algorithms.
- a surgical phase breakdown can be provided to understand where time was spent during the procedure and can be clicked on to navigate to the appropriate part of the video, for instance, through a user interface 1000 of phase analysis of FIG. 10.
- Personal comparison metrics can also be displayed on user interface portions 420 and 440 to understand surgical efficiency compared to a user’s average. Data can be displayed both graphically and in text/numbers.
- Targeted links can result in faster searching and locating of content of interest than manually attempting to find specific features within a video.
- one or more of the user interface portions 400, 420, 440 can include a video preview and link to navigate directly to the associated video.
- the user interface 500 of a case tag selector of FIG. 5 can be used to quickly add information to a case using pre-populated tags that are common across procedures and specific to the procedure in the selected case.
- tags can include complexity tags (e.g., standard or complex), emergent tags (e.g., urgent or elective), patient category tags (e.g., various body mass index ranges), teaching tags (e.g., teaching or not teaching), procedure specific tags (e.g., hernia types of direct, indirect, femoral, or orbturator), location tags (e.g., left or right).
- complexity tags e.g., standard or complex
- emergent tags e.g., urgent or elective
- patient category tags e.g., various body mass index ranges
- teaching tags e.g., teaching or not teaching
- procedure specific tags e.g., hernia types of direct, indirect, femoral, or orbturator
- location tags e.g., left or right.
- Custom tags can also be created and added.
- a video of a case can be viewed, for instance, through the user interface 700 of the video playback and workflow list viewer of FIG. 7, the user interface 800 of the video playback and workflow timeline viewer of FIG. 8, and/or the user interface 900 of the video playback and expanded workflow timeline viewer of FIG. 9. Further, a video can be navigated to from the home screen 300 of FIG. 3A for recent cases and/or a case detail page. To save time reviewing the whole video and have a more in-depth understanding of the case, workflow information can be presented below the video.
- a workflow can summarize a sequence of events, e.g., start of phase, anatomy in view, camera out of view, instrument in use, point of interest, and surgeon’s annotations, for example.
- the events can be clicked on, and the video proceeds to the relevant timestamp.
- the timeline shows portions of the video where instruments, anatomy, surgical phases, and/or other such aspects are observed.
- the timeline can illustrate transitions, combinations of features, and where such aspects occur in the video.
- the video proceeds (e.g., skips) to the relevant timestamp.
- monopolar curved shears may be used in a surgical phase which is not common practice, and such an occurrence can be identified.
- Turning/rotating a mobile device can give a larger timeline in a landscape orientation for ease of use as illustrated in the example of FIG. 9.
- the video view can also be augmented with information that is difficult or impossible to see from the video alone. For example, when electrosurgical instruments are powered up there is not always visible vapor, but from electrical logs, such information can be determined and displayed with the video.
- timelines are hidden, the video can be maximized for viewing and may include overlay information and controls, such as fast forward, pause/play, and rewind.
- the corresponding portion of the workflow list viewer may be highlighted.
- the corresponding workflow portion can change in color, intensity, or other variation to highlight which portion of the workflow is being displayed during video playback. For example, from time 3:21 to 12:27 of video playback, “Preperitoneal Space Dissection” may be highlighted and upon video playback advancing to time 12:28, “Hernia Sac Reduction” may be highlighted with the highlighting removed from “Preperitoneal Space Dissection”.
- workflow portions documented in the workflow list viewer may be active inputs to jump to desired portions of video playback.
- comments can be added to the video or portions of the workflow.
- Some content can be selectable to display or hide during video playback. For example, selecting a “show timelines” input on user interface 700 can result in expanding to display timeline information as depicted in user interface 800. Notably, expansion of a portion of the user interface 800 may result in shifting or hiding other content that had been visible on user interface 700. For example, making timeline visible may shift the workflow list viewer portion off of the visible portion of a user display but may still be viewable by scrolling.
- phase analysis information can be viewed in user interface 1000.
- the user interface 1000 may be reached, for example, by a selection from user interface portion 420.
- Phase analysis can provide more information relevant to the phases and highlight personalized insights.
- Each phase can be shown with metrics and color to highlight outliers (e.g., longer/shorter durations).
- the user’s average durations can be displayed to contrast against the current case duration and aid the user in understanding why that phase has been highlighted. Clicking on a phase can take a user to an occurrence in the video to review.
- a current phase being displayed during video playback can result in highlight an associated phase in the user interface 1000 if video playback is paused or active when transitioning to user interface 1000.
- Other types of highlighting can include identifying a phase having the greatest variation from average duration, which can include changing a color, font, or background of the corresponding time on the user interface 1000 (e.g., 32 minute duration versus 19 minute average).
- Navigation from user interface portion 340 of FIG. 3C can lead to insights displayed on portions 1100 and 1120 of a user interface of multiple procedure insights of FIGS. 11 A and 1 IB.
- the metrics along the top of user interface portion 1100 can include high-level information associated with case volume.
- Various controls can also be included, such as searching, zooming, filtering, alert settings, and other such controls and notifications.
- a view procedure volume selection can lead to a user interface 1200 of a procedure volume view of FIG. 12 that can provide more details on volume and insights for a specific procedure.
- Case distribution can be illustrated graphically and numerically, such as number of case type and percentage of total cases for each case type.
- FIGS. 13A, 13B, 13C depict portions 1300, 1320, 1340 of a user interface of multiple case procedure analysis.
- a user can navigate here from the user interface portions 320, 440, 1100, and/or 1200, for example.
- the user interface portions 1300, 1320, 1340 can present an analysis of cases of a specific procedure type.
- the data presented can include summaries of timings, instrument usage, and/or performance metrics, for example.
- the data can be compared to various averages and benchmarks.
- Outlier cases can be identified and displayed for easy access for review. Data from multiple sources can be used for wide scale comparisons across multiple locations (e.g., across a country).
- Instrument set usage for a country may be depicted as groups, such as combinations of instruments most frequently used together where instrument groups can have partial overlap, such as the same shear type used with different forceps types.
- FIG. 14 a flowchart of a method 1400 for surgical data insight generation and display is generally shown in accordance with one or more aspects. All or a portion of method 1400 can be implemented, for example, by all or a portion of CAS system 100 of FIG. 1, the system 200 of FIG. 2, and/or computer system 1500 of FIG. 15, for instance through execution of the surgical data management system 160.
- surgical data can be prepopulated in a surgical data insight viewer 240, where the surgical data (e.g., surgical data 214) can be associated with one or more surgical cases accessible to a user based on permissions of the user.
- the user can be provided with access through the surgical data insight viewer 240 to the surgical data in combination with surgical device data (e.g., associated with surgical instruments 108) recorded for the one or more surgical cases.
- the surgical data insight viewer 240 can display one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data.
- a workflow summary of the selected case can be provided that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer 240.
- displaying, through the surgical data insight viewer 240 can include a case summary including a case type, a case duration, an active time, and a time comparison to an average case time.
- displaying, through the surgical data insight viewer 240 can include a chart illustrating past case durations over a selectable period of time.
- displaying, through the surgical data insight viewer 240 can include case volume data for a plurality of case types.
- the surgical data insight viewer 240 can provide a user interface of a case tag selector to allow the user to select customized tags for summarizing case details.
- the surgical data insight viewer 240 can provide a user interface of phase analysis.
- the phase analysis can visually indicate an outlier condition detected in one or more surgical procedure phases.
- the surgical data insight viewer 240 can provide a user interface of insights including one or more comparisons of time a surgeon takes to complete a case and a progression as a number of cases increases over time.
- the surgical data insight viewer 240 can provide a user interface of a procedure volume view.
- the surgical data insight viewer 240 can provide a user interface of the procedure analysis including one or more of: summaries of timings, instrument usage, and performance metrics.
- the procedure analysis can include one or more of: averages and benchmarks relative to a geographic region.
- the procedure analysis can include a summary of outlier cases selected based on a difference between observed performance and the benchmarks.
- the averages and benchmarks for a hernia case type can be defined for one or more of the performance metrics including: a time to fix mesh, a defect size, bleeding, adhesiolysis, body mass index, and malignity.
- the processing shown in FIG. 14 is not intended to indicate that the operations are to be executed in any particular order or that all of the operations shown in FIG. 14 are to be included in every case. Additionally, the processing shown in FIG. 14 can include any suitable number of additional operations.
- a system can include a memory system and one or more processors coupled to the memory system and configured to execute a plurality of instructions to perform a plurality of operations.
- the operations can include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases.
- the operations can also include displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data.
- the operations can further include providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
- a computer program product can include a memory device having computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform a plurality of operations.
- the operations can include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases.
- the operations can also include displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data.
- the operations can further include providing a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
- the computer system 1500 can be an electronic computer framework comprising and/or employing any number and combination of computing devices and networks utilizing various communication technologies, as described herein.
- the computer system 1500 can be easily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others.
- the computer system 1500 may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone.
- computer system 1500 may be a cloud computing node.
- Computer system 1500 may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer system.
- program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types.
- Computer system 1500 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network.
- program modules may be located in both local and remote computer system storage media, including memory storage devices.
- the computer system 1500 has one or more central processing units (CPU(s)) 1501a, 1501b, 1501c, etc. (collectively or generically referred to as processor(s) 1501).
- the processors 1501 can be a single-core processor, multi-core processor, computing cluster, or any number of other configurations.
- the processors 1501 can be any type of circuitry capable of executing instructions.
- the processors 1501, also referred to as processing circuits are coupled via a system bus 1502 to a system memory 1503 and various other components.
- the system memory 1503 can include one or more memory devices, such as read-only memory (ROM) 1504 and a random-access memory (RAM) 1505.
- ROM read-only memory
- RAM random-access memory
- the ROM 1504 is coupled to the system bus 1502 and may include a basic input/output system (BIOS), which controls certain basic functions of the computer system 1500.
- BIOS basic input/output system
- the RAM is read-write memory coupled to the system bus 1502 for use by the processors 1501.
- the system memory 1503 provides temporary memory space for operations of said instructions during operation.
- the system memory 1503 can include random access memory (RAM), read-only memory, flash memory, or any other suitable memory systems.
- the computer system 1500 comprises an input/output (I/O) adapter 1506 and a communications adapter 1507 coupled to the system bus 1502.
- the I/O adapter 1506 may be a small computer system interface (SCSI) adapter that communicates with a hard disk 1508 and/or any other similar component.
- SCSI small computer system interface
- the I/O adapter 1506 and the hard disk 1508 are collectively referred to herein as a mass storage 1510.
- Software 1511 for execution on the computer system 1500 may be stored in the mass storage 1510.
- the mass storage 1510 is an example of a tangible storage medium readable by the processors 1501, where the software 1511 is stored as instructions for execution by the processors 1501 to cause the computer system 1500 to operate, such as is described hereinbelow with respect to the various Figures. Examples of computer program product and the execution of such instruction is discussed herein in more detail.
- the communications adapter 1507 interconnects the system bus 1502 with a network 1512, which may be an outside network, enabling the computer system 1500 to communicate with other such systems.
- a portion of the system memory 1503 and the mass storage 1510 collectively store an operating system, which may be any appropriate operating system to coordinate the functions of the various components shown in FIG. 15.
- Additional input/output devices are shown as connected to the system bus 1502 via a display adapter 1515 and an interface adapter 1516 and.
- the adapters 1506, 1507, 1515, and 1516 may be connected to one or more I/O buses that are connected to the system bus 1502 via an intermediate bus bridge (not shown).
- a display 1519 e.g., a screen or a display monitor
- a display adapter 1515 which may include a graphics controller to improve the performance of graphics-intensive applications and a video controller.
- a keyboard, a mouse, a touchscreen, one or more buttons, a speaker, etc. can be interconnected to the system bus 1502 via the interface adapter 1516, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit.
- Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI).
- PCI Peripheral Component Interconnect
- the computer system 1500 includes processing capability in the form of the processors 1501, and storage capability including the system memory 1503 and the mass storage 1510, input means such as the buttons, touchscreen, and output capability including the speaker 823 and the display 1519.
- the communications adapter 1507 can transmit data using any suitable interface or protocol, such as the internet small computer system interface, among others.
- the network 1512 may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others.
- An external computing device may connect to the computer system 1500 through the network 1512.
- an external computing device may be an external web server or a cloud computing node.
- FIG. 15 is not intended to indicate that the computer system 1500 is to include all of the components shown in FIG. 15. Rather, the computer system 1500 can include any appropriate fewer or additional components not illustrated in FIG. 15 (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Further, the aspects described herein with respect to computer system 1500 may be implemented with any appropriate logic, wherein the logic, as referred to herein, can include any suitable hardware (e.g., a processor, an embedded controller, or an application-specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware, in various aspects. Various aspects can be combined to include two or more of the aspects described herein.
- aspects disclosed herein may be a system, a method, and/or a computer program product at any possible technical detail level of integration
- the computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to carry out various aspects.
- the computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
- the computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
- a non- exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
- RAM random access memory
- ROM read-only memory
- EPROM or Flash memory erasable programmable read-only memory
- SRAM static random access memory
- CD-ROM compact disc read-only memory
- DVD digital versatile disk
- memory stick a floppy disk
- mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
- a computer-readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
- Computer-readable program instructions described herein can be downloaded to respective computing/processing devices from a computer-readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network, and/or a wireless network.
- the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers.
- a network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer- readable program instructions for storage in a computer-readable storage medium within the respective computing/processing device.
- Computer-readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source-code or object code written in any combination of one or more programming languages, including an object-oriented programming language, such as Smalltalk, C++, high-level languages such as Python, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages.
- the computer-readable program instructions may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer, or entirely on the remote computer or server.
- the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
- electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer-readable program instruction by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
- These computer-readable program instructions may be provided to a processor of a computer system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
- These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
- the computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer-implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
- each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s).
- the functions noted in the blocks may occur out of the order noted in the Figures.
- two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
- a coupling of entities can refer to either a direct or an indirect coupling, and a positional relationship between entities can be a direct or indirect positional relationship.
- a positional relationship between entities can be a direct or indirect positional relationship.
- the various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein.
- exemplary is used herein to mean “serving as an example, instance or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs.
- the terms “at least one” and “one or more” may be understood to include any integer number greater than or equal to one, i.e., one, two, three, four, etc.
- the terms “a plurality” may be understood to include any integer number greater than or equal to two, i.e., two, three, four, five, etc.
- connection may include both an indirect “connection” and a direct “connection.”
- the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit.
- Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium, such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).
- processors such as one or more digital signal processors (DSPs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.
- DSPs digital signal processors
- GPUs graphics processing units
- ASICs application-specific integrated circuits
- FPGAs field programmable logic arrays
- processors may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Public Health (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Epidemiology (AREA)
- Biomedical Technology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Radiology & Medical Imaging (AREA)
- Business, Economics & Management (AREA)
- General Business, Economics & Management (AREA)
- Medical Treatment And Welfare Office Work (AREA)
Abstract
Examples described herein provide a surgical data insight system that provides a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases. The surgical data insight viewer can display one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data. The surgical data insight viewer can also provide a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
Description
SURGICAL DATA INSIGHT SYSTEM
BACKGROUND
[0001] The present disclosure relates in general to computing technology and relates more particularly to computing technology for a surgical data insight system.
[0002] Many devices and systems can generate and record data separately during surgical procedures. Interpreting the captured data and determining how such data can be useful is a complex process. Data captured as log data or raw recordings may not be readily interpretable by parties interested in analyzing such data.
SUMMARY
[0003] According to an aspect, a computer-implemented method is provided. The method includes prepopulating surgical data in a surgical data insight viewer, the surgical data associated with one or more surgical cases accessible to a user based on permissions of the user and providing the user with access through the surgical data insight viewer to the surgical data in combination with surgical device data recorded for the one or more surgical cases. The method also includes displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data and providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
[0004] According to another aspect, a system includes a memory system and one or more processors coupled to the memory system and configured to execute a plurality of instructions to perform a plurality of operations. The operations include providing a user with access through a surgical data insight viewer to surgical data in combination with
surgical device data recorded for one or more surgical cases and displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data. The operations also include providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
[0005] According to a further aspect, a computer program product includes a memory device having computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform a plurality of operations. The operations include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases, displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data, and providing a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
[0006] The above features and advantages, and other features and advantages, of the disclosure are readily apparent from the following detailed description when taken in connection with the accompanying drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The specifics of the exclusive rights described herein are particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features and advantages of the aspects of the disclosure are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:
[0008] FIG. 1 depicts a computer-assisted surgery (CAS) system according to one or more aspects;
[0009] FIG. 2 depicts a surgical procedure system according to one or more aspects;
[0010] FIG. 3 A depicts a portion of a user interface providing case summary data according to one or more aspects;
[0011] FIG. 3B depicts another portion of a user interface providing case summary data according to one or more aspects;
[0012] FIG. 3C depicts a further portion of a user interface providing case summary data according to one or more aspects;
[0013] FIG. 4A depicts a portion of a user interface providing case details according to one or more aspects;
[0014] FIG. 4B depicts another portion of a user interface providing case details according to one or more aspects;
[0015] FIG. 4C depicts a further portion of a user interface providing case details according to one or more aspects;
[0016] FIG. 5 depicts a user interface of a case tag selector according to one or more aspects;
[0017] FIG. 6 depicts a user interface of case details according to one or more aspects;
[0018] FIG. 7 depicts a user interface of a video playback and workflow list viewer according to one or more aspects;
[0019] FIG. 8 depicts a user interface of a video playback and workflow timeline viewer according to one or more aspects;
[0020] FIG. 9 depicts a user interface of a video playback and expanded workflow timeline viewer according to one or more aspects;
[0021] FIG. 10 depicts a user interface of phase analysis according to one or more aspects;
[0022] FIG. 11 A depicts a portion of a user interface of multiple procedure insights according to one or more aspects;
[0023] FIG. 1 IB depicts a portion of a user interface of multiple procedure insights according to one or more aspects;
[0024] FIG. 12 depicts a user interface of a procedure volume view according to one or more aspects;
[0025] FIG. 13 A depicts a portion of a user interface of multiple case procedure analysis according to one or more aspects;
[0026] FIG. 13B depicts a portion of a user interface of multiple case procedure analysis of instrument usage according to one or more aspects;
[0027] FIG. 13C depicts a portion of a user interface of multiple case procedure analysis with performance and outlier case data according to one or more aspects;
[0028] FIG. 14 depicts a flowchart of a method according to one or more aspects; and
[0029] FIG. 15 depicts a block diagram of a computer system according to one or more aspects.
[0030] The diagrams depicted herein are illustrative. There can be many variations to the diagrams and/or the operations described herein without departing from the spirit of the described aspects. For instance, the actions can be performed in a differing order, or actions can be added, deleted, or modified. Also, the term “coupled” and variations thereof describe having a communications path between two elements and do not imply a
direct connection between the elements with no intervening elements/connections between them. All of these variations are considered a part of the specification.
DETAILED DESCRIPTION
[0031] Exemplary aspects of the technical solutions described herein include systems and methods for surgical data insights. Technical challenges of surgical data viewing can include collecting data from multiple sources, such as video data, instrument data, patient data, surgical team data, and historical data and presenting the information in a meaningful way. For example, data may be collected with a different time base and format that can be challenging to align in time to coordinate the associated information. The associated data can be partitioned into case records with index information generated to allow for various performance statistics to be computed efficiently across case types, surgical groups, specific users, geographic locations, patient groups, and other such groupings. Having the associated information available and ready for viewing as close in time to surgical procedure completion can be challenging. Controlling data collection, tag generation, and other aspects during a surgical procedure can allow the resulting data sets to be processed more rapidly to coordinate and display the associated information without manual intervention. User interfaces of a surgical data insight viewer can be designed for ease of navigation through large volumes of surgical data by providing a combination of summary data, detailed data, and various analytics for performance comparison. Technical solutions are described herein to address such technical challenges. Particularly, technical solutions herein provide surgical data insight generation and display.
[0032] Turning now to FIG. 1, an example computer-assisted system (CAS) system 100 is generally shown in accordance with one or more aspects. The CAS system 100 includes at least a computing system 102, a video recording system 104, and a surgical instrumentation system 106. As illustrated in FIG. 1, an actor 112 can be medical personnel that uses the CAS system 100 to perform a surgical procedure on a patient 110. Medical personnel can be a surgeon, assistant, nurse, administrator, or any other actor
that interacts with the CAS system 100 in a surgical environment. The surgical procedure can be any type of surgery. In other examples, actor 112 can be a technician, an administrator, an engineer, or any other such personnel that interacts with the CAS system 100. For example, actor 112 can record data from the CAS system 100, configure/update one or more attributes of the CAS system 100, review past performance of the CAS system 100, repair the CAS system 100, and/or the like including combinations and/or multiples thereof.
[0033] A surgical procedure can include multiple phases, and each phase can include one or more surgical actions. A “surgical action” can include an incision, a compression, a stapling, a clipping, a suturing, a cauterization, a sealing, or any other such actions performed to complete a phase in the surgical procedure. A “phase” represents a surgical event that is composed of a series of steps (e.g., closure). A “step” refers to the completion of a named surgical objective (e.g., hemostasis). During each step, certain surgical instruments 108 (e.g., forceps) are used to achieve a specific objective by performing one or more surgical actions. In addition, a particular anatomical structure of the patient may be the target of the surgical action(s).
[0034] The video recording system 104 includes one or more cameras 105, such as operating room cameras, endoscopic cameras, and/or the like including combinations and/or multiples thereof. The cameras 105 capture video data of the surgical procedure being performed. The video recording system 104 includes one or more video capture devices that can include cameras 105 placed in the surgical room to capture events surrounding (i.e., outside) the patient being operated upon. The video recording system 104 further includes cameras 105 that are passed inside (e.g., endoscopic cameras) the patient 110 to capture endoscopic data. The endoscopic data provides video and images of the surgical procedure.
[0035] The computing system 102 includes one or more memory devices, one or more processors, a user interface device, among other components. All or a portion of the computing system 102 shown in FIG. 1 can be implemented for example, by all or a
portion of computer system 1500 of FIG. 15. Computing system 102 can execute one or more computer-executable instructions. The execution of the instructions facilitates the computing system 102 to perform one or more methods, including those described herein. The computing system 102 can communicate with other computing systems via a wired and/or a wireless network.
[0036] A data collection system 150 can be employed to store the surgical data, including the video(s) captured during the surgical procedures. The data collection system 150 includes one or more storage devices 152. The data collection system 150 can be a local storage system, a cloud-based storage system, or a combination thereof. Further, the data collection system 150 can use any type of cloud-based storage architecture, for example, public cloud, private cloud, hybrid cloud, and/or the like including combinations and/or multiples thereof. In some examples, the data collection system can use a distributed storage, i.e., the storage devices 152 are located at different geographic locations. The storage devices 152 can include any type of electronic data storage media used for recording machine-readable data, such as semiconductor-based, magnetic-based, optical-based storage media, and/or the like including combinations and/or multiples thereof. For example, the data storage media can include flash-based solid-state drives (SSDs), magnetic-based hard disk drives, magnetic tape, optical discs, and/or the like including combinations and/or multiples thereof.
[0037] In one or more examples, the data collection system 150 can be part of the video recording system 104, or vice-versa. In some examples, the data collection system 150, the video recording system 104, and the computing system 102, can communicate with each other via a communication network, which can be wired, wireless, or a combination thereof. The communication between the systems can include the transfer of data (e.g., video data, instrumentation data, and/or the like including combinations and/or multiples thereof), data manipulation commands (e.g., browse, copy, paste, move, delete, create, compress, and/or the like including combinations and/or multiples thereof), data manipulation results, and/or the like including combinations and/or multiples
thereof. In one or more examples, the computing system 102 can manipulate the data already stored/being stored in the data collection system 150. Alternatively, or in addition, the computing system 102 can manipulate the data already stored/being stored in the data collection system 150 based on information from the surgical instrumentation system 106.
[0038] In one or more examples, the video captured by the video recording system 104 is stored on the data collection system 150. In some examples, the computing system 102 curates parts of the video data being stored on the data collection system 150. In some examples, the computing system 102 filters the video captured by the video recording system 104 before it is stored on the data collection system 150. Alternatively, or in addition, the computing system 102 filters the video captured by the video recording system 104 after it is stored on the data collection system 150. Instrument data (e.g., robotic logs, electrosurgical instrument logs, etc.) can also be stored in the data collection system 150.
[0039] A surgical data management system 160 can provide access to portions of data captured in the data collection system 150, as well as data and records stored in other systems. The surgical data management system 160 can establish user access permissions to patient and surgical data. The surgical data management system 160 can also control access through an interface based on the user access permissions. Access to the surgical data management system 160 can be provided through one or more applications or secure web pages. The surgical data management system 160 can be a stand-alone application, module, and/or an extension of another system. Additional aspects of the surgical data management system 160 can include accessing artificial intelligence (Al)-powered surgical video and analytics. Further aspects of the surgical data management system 160 can include accessing simulation materials that can assist surgeons to prepare, practice, and teach surgical procedures. Further aspects of the surgical data management system 160 can include integrating aspects of equipment in an operating room, surgery planning, rating surgeon performance, and other such features.
The surgical data management system 160 can also provide access to technical specifications and information relating to the use of surgical instruments, for example.
[0040] Turning now to FIG. 2, a surgical procedure system 200 is generally shown according to one or more aspects. The example of FIG. 2 depicts a surgical procedure support system 202 that can include or may be coupled to the CAS system 100 of FIG. 1. The surgical procedure support system 202 can acquire image or video data using one or more cameras 204. The surgical procedure support system 202 can also interface with one or more sensors 206 and/or one or more effectors 208. The sensors 206 may be associated with surgical support equipment and/or patient monitoring. The effectors 208 can be robotic components or other equipment controllable through the surgical procedure support system 202. The surgical procedure support system 202 can also interact with one or more user interfaces 210, such as various input and/or output devices. The surgical procedure support system 202 can store, access, and/or update surgical data 214 associated with a training dataset and/or live data as a surgical procedure is being performed on patient 110 of FIG. 1. The surgical procedure support system 202 can store, access, and/or update surgical objectives 216 to assist in training and guidance for one or more surgical procedures. User configurations 218 can track and store user preferences.
[0041] The surgical procedure support system 202 can also communicate with other systems through a network 230. For example, the surgical procedure support system 202 can communicate with a surgical data insight viewer 240 and a surgical data postprocessing system 250 through the network 230. Other types of devices, such as a computing device 234 (e.g., a mobile phone, laptop, personal computer, or tablet computer), can communicate directly with the surgical procedure support system 202 or through the network 230. As one example, user interfaces 210 may be connected to or integrated with the surgical procedure support system 202 by a wired connection while the computing device 234 connects to the surgical procedure support system 202 via a wireless connection. In some aspects, the computing device 234 can execute or link to
another computer system that executes the surgical data management system 160 of FIG.
1 to access various data sources through the network 230.
[0042] The surgical data post-processing system 250 can receive surgical data and associated data generated by the surgical procedure support system 202 and may be separately stored and secured through other data storage. Access to specific data or portions of data through the surgical data post-processing system 250 may be limited by associated permissions. The surgical data post-processing system 250 may include features such as video viewing, video sharing, data analytics, and selective data extraction.
[0043] The surgical data insight viewer 240 can provide viewing access to various data sources, such as surgical data 214, data collected in the one or more storage devices 152 of FIG. 1, and post-processed data generated by the surgical data post-processing system 250. For example, surgical instrument data, video data, and artificial intelligence generated data can be accessed for viewing through the surgical data insight viewer 240. The surgical data post-processing system 250 may generate surgical performance metrics and comparison data across data sets collected at multiple locations, making analytics data available to the surgical data insight viewer 240. The surgical data insight viewer 240 can preload case data for more than one case of a user, for example, when the user performs a login or otherwise activates the surgical data insight viewer 240. In some aspects, the surgical data insight viewer 240 and/or surgical data post-processing system 250 can be components of the surgical data management system 160 of FIG. 1.
[0044] One or more computing device 264 (e.g., a mobile phone, laptop, personal computer, or tablet computer), can execute the surgical data management system 160 of FIG. 1 to access various data sources through a network 260. The network 230 may be within a facility or multiple facilities maintained within a private network. The network 260 may be a wider area network, such as the internet. Accordingly, the networks 230 and 260 may have access to different files and data sets along with shared access to select files and data sets. In some aspects, networks 230 and 260 can be combined.
[0045] According to aspects, the surgical data management system 160 of FIG. 1 can provide multiple user interfaces and functions through the surgical data insight viewer 240, such as those described with respect to FIGS. 3A-13. Various user interfaces can be designed for specific types of devices, such as handheld mobile devices, for efficient navigation and display of content. Variations in the features and flow of transitions between user interfaces and content accessible through user interfaces can exist, for example, with respect to interactive interfaces for case summary data, case details, case tags, video viewers with supplemental information, phase analysis, insights, procedure volume, procedure analysis, performance metrics, instrument usage, outlier case identification, and other such aspects.
[0046] FIGS. 3 A, 3B, and 3C depict portions of a user interface providing case summary data according to one or more aspects. FIG. 3 A depicts an example of a home screen 300, which can summarize aspects of a selected case. A user may scroll to user interface portions 320, 340 to view other aspects, such as a performance comparison for a same type of procedure over a period of time, case volume information, and case selection. Further, the user may select navigation features to see greater detail about the selected case, procedure analysis information, case volume information, further insights, a learning portal, profile information, and/or other content.
[0047] As an example, a surgeon may access the home screen 300 upon completing a surgical procedure to review the case and make notes. The case can be preloaded for viewing through the surgical data insight viewer 240, for instance, along with the last five cases for immediate selection. Data (e.g., video, robotic logs, electrical instrument logs, etc.) can be indexed/linked and available for immediate access through the surgical data insight viewer 240. Summary information, such as date and time, along with metrics such as active console time, case duration and difference from personal average can appear on the home screen 300 as a case summary card. The home screen 300 can also provide access to setup/configure user preferences for displaying content and generating output for various metrics and reporting tools.
[0048] User interface portion 320 can display graphical content, such as a chart or graph associated with a selected surgical procedure. The time scale can be adjusted on the user interface portion 320 in response to a user timescale selection. Further procedure analytics or associated insights can be selected through a use input, such as a tapping, clicking, or swiping gesture.
[0049] User interface portion 340 can display case volume information, such as a total number of cases that have associated data along with graphical and/or numeric distributions of the case volume, such as a total number of laparoscopic cases, robotic cases, cases with trainees, and other such categories. High-level case summaries can be viewable and selectable through user interface portion 340, for instance, to display associate surgical videos and metrics for particular surgeries. Information, such as case name, a representative image, date, duration, and case tags can also be displayed. Case tags can include procedure categories (e.g., complex, training, etc.), patient categories (e.g., body mass range), outcome categories (e.g., benign, malignant, etc.), and/or other information that may assist with grouping or searching for cases.
[0050] After selecting a recent case on the home screen 300 (or elsewhere through the surgical data insight viewer 240), user interface portions 400, 420, 440 of FIGS. 4A, 4B, 4C can be viewed through the surgical data insight viewer 240. The user interface portions 400, 420, 440 can summarize a surgical procedure at-a-glance (perhaps with some scrolling). Watching a whole video to understand what happened during surgery can be time consuming. The view depicted in FIGS. 4A-4C can make the review process faster. Case tags can be added by the user to help identify cases at a later date or add extra information, for instance, through a user interface 500 of a case tag selector of FIG. 5. The video with a workflow can also be accessed from user interface portion 400 to display, for instance, a user interface 700 of a video playback and workflow list viewer of FIG. 7, a user interface 800 of a video playback and workflow timeline viewer of FIG. 8, and/or a user interface 900 of a video playback and expanded workflow timeline viewer of FIG. 9. Features, such as timelines of the user interface 900, can be displayed or
hidden in response to user selection to allow users to further customize individual user experience. In some aspects, a video thumbnail or preview image may be displayed on user interface portion 400 prior to displaying a video. For instance, a preview image may be selectable to trigger starting of video playback along with another button that may also trigger video playback.
[0051] Summary metrics can be displayed on user interface portions 400, 420, 440 to help identify how the selected case differs from others and whether it may be worthwhile reviewing. Summary metrics can be derived from multiple data sources. For instance, console time can be derived from robotics logs, camera out time can be derived from video, and defect size can be derived from video using artificial intelligence algorithms. A surgical phase breakdown can be provided to understand where time was spent during the procedure and can be clicked on to navigate to the appropriate part of the video, for instance, through a user interface 1000 of phase analysis of FIG. 10. Personal comparison metrics can also be displayed on user interface portions 420 and 440 to understand surgical efficiency compared to a user’s average. Data can be displayed both graphically and in text/numbers. Instruments used can also be detailed along with metrics about instrument usage in user interface portion 440. Numbers of units used, amount of time elapsed during a procedure, and energy deposited are some example metrics that can be tracked. The metrics can be useful in understanding efficiency and experience, for example. Time of different or otherwise notable anatomy visible during the procedure can also be displayed. Time may be expressed in total time, e.g., number of minutes and/or percentage relative to total time duration of the procedure. Clicking upon an instrument summary or an anatomy summary can open the associated video and skip to the first occurrence in the video. This usability feature allows a surgeon to go directly to reviewing the handling/dissection of anatomy critical to completion of the procedure. Targeted links can result in faster searching and locating of content of interest than manually attempting to find specific features within a video. In some aspects, one or more of the user interface portions 400, 420, 440 can include a video preview and link to navigate directly to the associated video.
[0052] In some aspects, the user interface 500 of a case tag selector of FIG. 5 can be used to quickly add information to a case using pre-populated tags that are common across procedures and specific to the procedure in the selected case. Examples of tags can include complexity tags (e.g., standard or complex), emergent tags (e.g., urgent or elective), patient category tags (e.g., various body mass index ranges), teaching tags (e.g., teaching or not teaching), procedure specific tags (e.g., hernia types of direct, indirect, femoral, or orbturator), location tags (e.g., left or right). Custom tags can also be created and added. Upon exiting the user interface 500, the tags can be visible on the home screen 300 of FIG. 3A and in a user interface 600 of case details of FIG. 6, which illustrates a modification to the user interface portion 400 of FIG. 4 after case tags have been added.
[0053] Upon selecting a watch video workflow button from user interface portion 400 or 600, a video of a case can be viewed, for instance, through the user interface 700 of the video playback and workflow list viewer of FIG. 7, the user interface 800 of the video playback and workflow timeline viewer of FIG. 8, and/or the user interface 900 of the video playback and expanded workflow timeline viewer of FIG. 9. Further, a video can be navigated to from the home screen 300 of FIG. 3A for recent cases and/or a case detail page. To save time reviewing the whole video and have a more in-depth understanding of the case, workflow information can be presented below the video. A workflow can summarize a sequence of events, e.g., start of phase, anatomy in view, camera out of view, instrument in use, point of interest, and surgeon’s annotations, for example. The events can be clicked on, and the video proceeds to the relevant timestamp. The timeline shows portions of the video where instruments, anatomy, surgical phases, and/or other such aspects are observed. The timeline can illustrate transitions, combinations of features, and where such aspects occur in the video. By clicking on the timeline, the video proceeds (e.g., skips) to the relevant timestamp. As an example, monopolar curved shears may be used in a surgical phase which is not common practice, and such an occurrence can be identified. Turning/rotating a mobile device (e.g., computing device 234, 264 with surgical data insight viewer 240) can give a larger timeline in a landscape
orientation for ease of use as illustrated in the example of FIG. 9. The video view can also be augmented with information that is difficult or impossible to see from the video alone. For example, when electrosurgical instruments are powered up there is not always visible vapor, but from electrical logs, such information can be determined and displayed with the video. When timelines are hidden, the video can be maximized for viewing and may include overlay information and controls, such as fast forward, pause/play, and rewind.
[0054] Further, to assist in user understanding, as a video is played, for instance, through user interface 700, the corresponding portion of the workflow list viewer may be highlighted. For example, upon video playback reaching a workflow portion documented in the workflow list viewer portion of the user interface 700, the corresponding workflow portion can change in color, intensity, or other variation to highlight which portion of the workflow is being displayed during video playback. For example, from time 3:21 to 12:27 of video playback, “Preperitoneal Space Dissection” may be highlighted and upon video playback advancing to time 12:28, “Hernia Sac Reduction” may be highlighted with the highlighting removed from “Preperitoneal Space Dissection”. In some aspect, workflow portions documented in the workflow list viewer may be active inputs to jump to desired portions of video playback. In some aspects, comments can be added to the video or portions of the workflow.
[0055] Some content can be selectable to display or hide during video playback. For example, selecting a “show timelines” input on user interface 700 can result in expanding to display timeline information as depicted in user interface 800. Notably, expansion of a portion of the user interface 800 may result in shifting or hiding other content that had been visible on user interface 700. For example, making timeline visible may shift the workflow list viewer portion off of the visible portion of a user display but may still be viewable by scrolling.
[0056] In the example of FIG. 10, phase analysis information can be viewed in user interface 1000. The user interface 1000 may be reached, for example, by a selection from
user interface portion 420. Phase analysis can provide more information relevant to the phases and highlight personalized insights. Each phase can be shown with metrics and color to highlight outliers (e.g., longer/shorter durations). The user’s average durations can be displayed to contrast against the current case duration and aid the user in understanding why that phase has been highlighted. Clicking on a phase can take a user to an occurrence in the video to review. In some aspects, a current phase being displayed during video playback can result in highlight an associated phase in the user interface 1000 if video playback is paused or active when transitioning to user interface 1000. Other types of highlighting can include identifying a phase having the greatest variation from average duration, which can include changing a color, font, or background of the corresponding time on the user interface 1000 (e.g., 32 minute duration versus 19 minute average).
[0057] Navigation from user interface portion 340 of FIG. 3C can lead to insights displayed on portions 1100 and 1120 of a user interface of multiple procedure insights of FIGS. 11 A and 1 IB. The metrics along the top of user interface portion 1100 can include high-level information associated with case volume. Various controls can also be included, such as searching, zooming, filtering, alert settings, and other such controls and notifications. A view procedure volume selection can lead to a user interface 1200 of a procedure volume view of FIG. 12 that can provide more details on volume and insights for a specific procedure. Case distribution can be illustrated graphically and numerically, such as number of case type and percentage of total cases for each case type. Some insights may not be comparable between different procedure types, and thus, other procedure types that the surgeon has performed can be presented as separate cards in user interface portions 1100, 1120. The cards can depict high-level comparisons of the time the surgeon takes to complete a case and how that progresses as the number of cases increase. There can also be a comparison to benchmarks derived from the surgical data management system 160 of FIG. 1, such as countrywide average, global average, top 10, etc. Selecting a card can take the user to insights for a specific procedure type. Cards
can include summary information, such as a number of cases used for computing user averages.
[0058] FIGS. 13A, 13B, 13C depict portions 1300, 1320, 1340 of a user interface of multiple case procedure analysis. A user can navigate here from the user interface portions 320, 440, 1100, and/or 1200, for example. The user interface portions 1300, 1320, 1340 can present an analysis of cases of a specific procedure type. The data presented can include summaries of timings, instrument usage, and/or performance metrics, for example. The data can be compared to various averages and benchmarks. Outlier cases can be identified and displayed for easy access for review. Data from multiple sources can be used for wide scale comparisons across multiple locations (e.g., across a country). Information, such as active console time can be displayed graphically for cases over time to compare individual user performance to other averages (e.g., department average, countrywide average, global average, top 10% average, top 25% average, etc.). Performance numbers can be further sorted or filtered at various levels of granularity, such as comparisons of different types (e.g., direct, indirect, femoral, obturator, etc. as examples for hernia procedures), different locations (e.g., left or right), bleeding, patient range (e.g., patient body mass index ranges), malignity, and other such aspects. In some aspects, instrument set usage for a country may be depicted as groups, such as combinations of instruments most frequently used together where instrument groups can have partial overlap, such as the same shear type used with different forceps types.
[0059] Turning now to FIG. 14, a flowchart of a method 1400 for surgical data insight generation and display is generally shown in accordance with one or more aspects. All or a portion of method 1400 can be implemented, for example, by all or a portion of CAS system 100 of FIG. 1, the system 200 of FIG. 2, and/or computer system 1500 of FIG. 15, for instance through execution of the surgical data management system 160.
[0060] At block 1402, surgical data can be prepopulated in a surgical data insight viewer 240, where the surgical data (e.g., surgical data 214) can be associated with one or
more surgical cases accessible to a user based on permissions of the user. At block 1404, the user can be provided with access through the surgical data insight viewer 240 to the surgical data in combination with surgical device data (e.g., associated with surgical instruments 108) recorded for the one or more surgical cases. At block 1406, the surgical data insight viewer 240 can display one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data. At block 1408, a workflow summary of the selected case can be provided that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer 240.
[0061] According to some aspects, a practitioner set of permissions can be provided to access patient and surgical data based on determining the role of the user.
[0062] According to an aspect, displaying, through the surgical data insight viewer 240, can include a case summary including a case type, a case duration, an active time, and a time comparison to an average case time.
[0063] According to an aspect, displaying, through the surgical data insight viewer 240, can include a chart illustrating past case durations over a selectable period of time.
[0064] According to an aspect, displaying, through the surgical data insight viewer 240, can include case volume data for a plurality of case types.
[0065] According to an aspect, the surgical data insight viewer 240 can provide a user interface of a case tag selector to allow the user to select customized tags for summarizing case details.
[0066] According to an aspect, the surgical data insight viewer 240 can provide a user interface of phase analysis. The phase analysis can visually indicate an outlier condition detected in one or more surgical procedure phases.
[0067] According to an aspect, the surgical data insight viewer 240 can provide a user interface of insights including one or more comparisons of time a surgeon takes to complete a case and a progression as a number of cases increases over time.
[0068] According to an aspect, the surgical data insight viewer 240 can provide a user interface of a procedure volume view.
[0069] According to an aspect, the surgical data insight viewer 240 can provide a user interface of the procedure analysis including one or more of: summaries of timings, instrument usage, and performance metrics. The procedure analysis can include one or more of: averages and benchmarks relative to a geographic region. The procedure analysis can include a summary of outlier cases selected based on a difference between observed performance and the benchmarks. The averages and benchmarks for a hernia case type can be defined for one or more of the performance metrics including: a time to fix mesh, a defect size, bleeding, adhesiolysis, body mass index, and malignity.
[0070] The processing shown in FIG. 14 is not intended to indicate that the operations are to be executed in any particular order or that all of the operations shown in FIG. 14 are to be included in every case. Additionally, the processing shown in FIG. 14 can include any suitable number of additional operations.
[0071] According to another aspect, a system can include a memory system and one or more processors coupled to the memory system and configured to execute a plurality of instructions to perform a plurality of operations. The operations can include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases. The operations can also include displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data. The operations can further include providing a workflow summary of the selected case that visually displays a plurality of timelines
associated with a video playback of the selected case through the surgical data insight viewer.
[0072] According to a further aspect, a computer program product can include a memory device having computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform a plurality of operations. The operations can include providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases. The operations can also include displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data. The operations can further include providing a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
[0073] Turning now to FIG. 15, a computer system 1500 is generally shown in accordance with an aspect. The computer system 1500 can be an electronic computer framework comprising and/or employing any number and combination of computing devices and networks utilizing various communication technologies, as described herein. The computer system 1500 can be easily scalable, extensible, and modular, with the ability to change to different services or reconfigure some features independently of others. The computer system 1500 may be, for example, a server, desktop computer, laptop computer, tablet computer, or smartphone. In some examples, computer system 1500 may be a cloud computing node. Computer system 1500 may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer system. Generally, program modules may include routines, programs, objects, components, logic, data structures, and so on that perform particular tasks or implement particular abstract data types. Computer system 1500 may be practiced in distributed cloud computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a
distributed cloud computing environment, program modules may be located in both local and remote computer system storage media, including memory storage devices.
[0074] As shown in FIG. 15, the computer system 1500 has one or more central processing units (CPU(s)) 1501a, 1501b, 1501c, etc. (collectively or generically referred to as processor(s) 1501). The processors 1501 can be a single-core processor, multi-core processor, computing cluster, or any number of other configurations. The processors 1501 can be any type of circuitry capable of executing instructions. The processors 1501, also referred to as processing circuits, are coupled via a system bus 1502 to a system memory 1503 and various other components. The system memory 1503 can include one or more memory devices, such as read-only memory (ROM) 1504 and a random-access memory (RAM) 1505. The ROM 1504 is coupled to the system bus 1502 and may include a basic input/output system (BIOS), which controls certain basic functions of the computer system 1500. The RAM is read-write memory coupled to the system bus 1502 for use by the processors 1501. The system memory 1503 provides temporary memory space for operations of said instructions during operation. The system memory 1503 can include random access memory (RAM), read-only memory, flash memory, or any other suitable memory systems.
[0075] The computer system 1500 comprises an input/output (I/O) adapter 1506 and a communications adapter 1507 coupled to the system bus 1502. The I/O adapter 1506 may be a small computer system interface (SCSI) adapter that communicates with a hard disk 1508 and/or any other similar component. The I/O adapter 1506 and the hard disk 1508 are collectively referred to herein as a mass storage 1510.
[0076] Software 1511 for execution on the computer system 1500 may be stored in the mass storage 1510. The mass storage 1510 is an example of a tangible storage medium readable by the processors 1501, where the software 1511 is stored as instructions for execution by the processors 1501 to cause the computer system 1500 to operate, such as is described hereinbelow with respect to the various Figures. Examples of computer program product and the execution of such instruction is discussed herein in more detail.
The communications adapter 1507 interconnects the system bus 1502 with a network 1512, which may be an outside network, enabling the computer system 1500 to communicate with other such systems. In one aspect, a portion of the system memory 1503 and the mass storage 1510 collectively store an operating system, which may be any appropriate operating system to coordinate the functions of the various components shown in FIG. 15.
[0077] Additional input/output devices are shown as connected to the system bus 1502 via a display adapter 1515 and an interface adapter 1516 and. In one aspect, the adapters 1506, 1507, 1515, and 1516 may be connected to one or more I/O buses that are connected to the system bus 1502 via an intermediate bus bridge (not shown). A display 1519 (e.g., a screen or a display monitor) is connected to the system bus 1502 by a display adapter 1515, which may include a graphics controller to improve the performance of graphics-intensive applications and a video controller. A keyboard, a mouse, a touchscreen, one or more buttons, a speaker, etc., can be interconnected to the system bus 1502 via the interface adapter 1516, which may include, for example, a Super I/O chip integrating multiple device adapters into a single integrated circuit. Suitable I/O buses for connecting peripheral devices such as hard disk controllers, network adapters, and graphics adapters typically include common protocols, such as the Peripheral Component Interconnect (PCI). Thus, as configured in FIG. 15, the computer system 1500 includes processing capability in the form of the processors 1501, and storage capability including the system memory 1503 and the mass storage 1510, input means such as the buttons, touchscreen, and output capability including the speaker 823 and the display 1519.
[0078] In some aspects, the communications adapter 1507 can transmit data using any suitable interface or protocol, such as the internet small computer system interface, among others. The network 1512 may be a cellular network, a radio network, a wide area network (WAN), a local area network (LAN), or the Internet, among others. An external computing device may connect to the computer system 1500 through the network 1512.
In some examples, an external computing device may be an external web server or a cloud computing node.
[0079] It is to be understood that the block diagram of FIG. 15 is not intended to indicate that the computer system 1500 is to include all of the components shown in FIG. 15. Rather, the computer system 1500 can include any appropriate fewer or additional components not illustrated in FIG. 15 (e.g., additional memory components, embedded controllers, modules, additional network interfaces, etc.). Further, the aspects described herein with respect to computer system 1500 may be implemented with any appropriate logic, wherein the logic, as referred to herein, can include any suitable hardware (e.g., a processor, an embedded controller, or an application-specific integrated circuit, among others), software (e.g., an application, among others), firmware, or any suitable combination of hardware, software, and firmware, in various aspects. Various aspects can be combined to include two or more of the aspects described herein.
[0080] Aspects disclosed herein may be a system, a method, and/or a computer program product at any possible technical detail level of integration. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to carry out various aspects.
[0081] The computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non- exhaustive list of more specific examples of the computer-readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory
stick, a floppy disk, a mechanically encoded device, such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer-readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
[0082] Computer-readable program instructions described herein can be downloaded to respective computing/processing devices from a computer-readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network, and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer- readable program instructions for storage in a computer-readable storage medium within the respective computing/processing device.
[0083] Computer-readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state-setting data, configuration data for integrated circuitry, or either source-code or object code written in any combination of one or more programming languages, including an object-oriented programming language, such as Smalltalk, C++, high-level languages such as Python, or the like, and procedural programming languages, such as the “C” programming language or similar programming languages. The computer-readable program instructions may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer, or entirely on the remote computer or server.
In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some aspects, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer-readable program instruction by utilizing state information of the computer-readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
[0084] Aspects are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to aspects of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer-readable program instructions.
[0085] These computer-readable program instructions may be provided to a processor of a computer system, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer-readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
[0086] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a
series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer-implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
[0087] The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various aspects. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
[0088] The descriptions of the various aspects have been presented for purposes of illustration but are not intended to be exhaustive or limited to the aspects disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described aspects. The terminology used herein was chosen to best explain the principles of the aspects, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the aspects described herein.
[0089] Various aspects are described herein with reference to the related drawings.
Alternative aspects can be devised without departing from the scope of this disclosure.
Various connections and positional relationships (e.g., over, below, adjacent, etc.) are set forth between elements in the following description and in the drawings. These connections and/or positional relationships, unless specified otherwise, can be direct or indirect, and the present disclosure is not intended to be limiting in this respect.
Accordingly, a coupling of entities can refer to either a direct or an indirect coupling, and a positional relationship between entities can be a direct or indirect positional relationship. Moreover, the various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein.
[0090] The following definitions and abbreviations are to be used for the interpretation of the claims and the specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains,” or “containing,” or any other variation thereof are intended to cover a non-exclusive inclusion. For example, a composition, a mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but can include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.
[0091] Additionally, the term “exemplary” is used herein to mean “serving as an example, instance or illustration.” Any aspect or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects or designs. The terms “at least one” and “one or more” may be understood to include any integer number greater than or equal to one, i.e., one, two, three, four, etc. The terms “a plurality” may be understood to include any integer number greater than or equal to two, i.e., two, three, four, five, etc. The term “connection” may include both an indirect “connection” and a direct “connection.”
[0092] The terms “about,” “substantially,” “approximately,” and variations thereof are intended to include the degree of error associated with measurement of the particular
quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ± 8% or 5%, or 2% of a given value.
[0093] For the sake of brevity, conventional techniques related to making and using aspects may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and/or process details.
[0094] It should be understood that various aspects disclosed herein may be combined in different combinations than the combinations specifically presented in the description and accompanying drawings. It should also be understood that, depending on the example, certain acts or events of any of the processes or methods described herein may be performed in a different sequence, may be added, merged, or left out altogether (e.g., all described acts or events may not be necessary to carry out the techniques). In addition, while certain aspects of this disclosure are described as being performed by a single module or unit for purposes of clarity, it should be understood that the techniques of this disclosure may be performed by a combination of units or modules associated with, for example, a medical device.
[0095] In one or more examples, the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit. Computer-readable media may include non-transitory computer-readable media, which corresponds to a tangible medium, such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).
[0096] Instructions may be executed by one or more processors, such as one or more digital signal processors (DSPs), graphics processing units (GPUs), microprocessors, application-specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor” as used herein may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.
Claims
1. A computer-implemented method comprising: prepopulating surgical data in a surgical data insight viewer, the surgical data associated with one or more surgical cases accessible to a user based on permissions of the user; providing the user with access through the surgical data insight viewer to the surgical data in combination with surgical device data recorded for the one or more surgical cases; displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data; and providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
2. The computer-implemented method of claim 1, wherein displaying, through the surgical data insight viewer, comprises a case summary including a case type, a case duration, an active time, and a time comparison to an average case time.
3. The computer-implemented method of claim 1 or claim 2, wherein displaying, through the surgical data insight viewer, comprises a chart illustrating past case durations over a selectable period of time.
4. The computer-implemented method of any preceding claim, wherein displaying, through the surgical data insight viewer, comprises case volume data for a plurality of
case types.
5. The computer-implemented method of any preceding claim, wherein the surgical data insight viewer provides a user interface of a case tag selector to allow the user to select customized tags for summarizing case details.
6. The computer-implemented method of any preceding claim, wherein the surgical data insight viewer provides a user interface of phase analysis.
7. The computer-implemented method of claim 6, wherein the phase analysis visually indicates an outlier condition detected in one or more surgical procedure phases.
8. The computer-implemented method of any preceding claim, wherein the surgical data insight viewer provides a user interface of insights comprising one or more comparisons of time a surgeon takes to complete a case and a progression as a number of cases increases over time.
9. The computer-implemented method of any preceding claim, wherein the surgical data insight viewer provides a user interface of a procedure volume view.
10. The computer-implemented method of any preceding claim, wherein the surgical data insight viewer provides a user interface of the procedure analysis comprising one or more of: summaries of timings, instrument usage, and performance metrics.
11. The computer-implemented method of claim 10, wherein the procedure analysis comprises one or more of: averages and benchmarks relative to a geographic region.
12. The computer-implemented method of claim 11, wherein the procedure analysis comprises a summary of outlier cases selected based on a difference between observed performance and the benchmarks.
13. The computer-implemented method of claim 11 or claim 12, wherein the averages and benchmarks for a hernia case type are defined for one or more of the performance
metrics comprising: a time to fix mesh, a defect size, bleeding, adhesiolysis, body mass index, and malignity.
14. A system comprising: a memory system; and one or more processors coupled to the memory system and configured to execute a plurality of instructions to perform a plurality of operations comprising: providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases; displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, case comparison timing data for a plurality of cases of two or more types, and surgical instrument usage data; and providing a workflow summary of the selected case that visually displays a plurality of timelines associated with a video playback of the selected case through the surgical data insight viewer.
15. The system of claim 14, wherein displaying, through the surgical data insight viewer, comprises a chart illustrating past case durations over a selectable period of time and case volume data for a plurality of case types.
16. The system of claim 14 or claim 15, wherein the surgical data insight viewer provides a user interface of a case tag selector to allow the user to select customized tags for summarizing case details and a user interface of phase analysis that visually indicates an outlier condition detected in one or more surgical procedure phases.
17. The system of claims 14 to 16, wherein the surgical data insight viewer provides a
user interface of insights comprising one or more comparisons of time a surgeon takes to complete a case and a progression as a number of cases increases over time, and the surgical data insight viewer provides a user interface of the procedure analysis comprising one or more of: summaries of timings, instrument usage, and performance metrics.
18. A computer program product comprising a memory device having computer executable instructions stored thereon, which when executed by one or more processors cause the one or more processors to perform a plurality of operations comprising: providing a user with access through a surgical data insight viewer to surgical data in combination with surgical device data recorded for one or more surgical cases; displaying, through the surgical data insight viewer, one or more of: case details of a selected case of the one or more cases, a procedure analysis for a plurality of cases of a same type, and surgical instrument usage data; and providing a summary of the selected case associated with a video playback of the selected case through the surgical data insight viewer.
19. The computer program product of claim 18, wherein the surgical data insight viewer provides a user interface of the procedure analysis comprising one or more of: summaries of timings, instrument usage, and performance metrics, and the procedure analysis comprises one or more of: averages and benchmarks relative to a geographic region.
20. The computer program product of claim 19, wherein the procedure analysis comprises a summary of outlier cases selected based on a difference between observed performance and the benchmarks.
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US202363536971P | 2023-09-07 | 2023-09-07 | |
US63/536,971 | 2023-09-07 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2025051987A1 true WO2025051987A1 (en) | 2025-03-13 |
Family
ID=92711150
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/EP2024/075041 WO2025051987A1 (en) | 2023-09-07 | 2024-09-06 | Surgical data insight system |
Country Status (1)
Country | Link |
---|---|
WO (1) | WO2025051987A1 (en) |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180122506A1 (en) * | 2015-03-26 | 2018-05-03 | Surgical Safety Technologies Inc. | Operating room black-box device, system, method and computer readable medium for event and error prediction |
US20210307840A1 (en) * | 2020-04-05 | 2021-10-07 | Theator inc. | Image analysis for detecting deviations from a surgical plane |
WO2021252384A1 (en) * | 2020-06-08 | 2021-12-16 | Activ Surgical, Inc. | Systems and methods for processing medical data |
WO2022231993A1 (en) * | 2021-04-27 | 2022-11-03 | Intuitive Surgical Operations, Inc. | Graphical user interface for surgical performance assessment |
WO2023144356A1 (en) * | 2022-01-28 | 2023-08-03 | Covidien Lp | Provision of surgical guidance based on audiovisual data and instrument data |
-
2024
- 2024-09-06 WO PCT/EP2024/075041 patent/WO2025051987A1/en unknown
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180122506A1 (en) * | 2015-03-26 | 2018-05-03 | Surgical Safety Technologies Inc. | Operating room black-box device, system, method and computer readable medium for event and error prediction |
US20210307840A1 (en) * | 2020-04-05 | 2021-10-07 | Theator inc. | Image analysis for detecting deviations from a surgical plane |
WO2021252384A1 (en) * | 2020-06-08 | 2021-12-16 | Activ Surgical, Inc. | Systems and methods for processing medical data |
WO2022231993A1 (en) * | 2021-04-27 | 2022-11-03 | Intuitive Surgical Operations, Inc. | Graphical user interface for surgical performance assessment |
WO2023144356A1 (en) * | 2022-01-28 | 2023-08-03 | Covidien Lp | Provision of surgical guidance based on audiovisual data and instrument data |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20150046818A1 (en) | Software tools platform for medical environments | |
US20240395287A1 (en) | Low-latency video capture and overlay | |
US20240087699A1 (en) | Graphical user interface for surgical performance assessment | |
EP4356290A1 (en) | Detection of surgical states, motion profiles, and instruments | |
US20250148790A1 (en) | Position-aware temporal graph networks for surgical phase recognition on laparoscopic videos | |
Eryigit et al. | Association of video completed by audio in laparoscopic cholecystectomy with improvements in operative reporting | |
EP4430834A1 (en) | Feature contingent surgical video compression | |
EP4323972A1 (en) | Identifying variation in surgical approaches | |
WO2025051987A1 (en) | Surgical data insight system | |
US20250014717A1 (en) | Removing redundant data from catalogue of surgical video | |
CN120188199A (en) | Spatiotemporal network for video semantic segmentation in surgical video | |
US20240428956A1 (en) | Query similar cases based on video information | |
WO2023144356A1 (en) | Provision of surgical guidance based on audiovisual data and instrument data | |
WO2025036996A1 (en) | Context-aware surgical summary for streaming | |
WO2025036995A1 (en) | Annotation overlay through streaming interface | |
WO2024110547A1 (en) | Video analysis dashboard for case review | |
WO2024213571A1 (en) | Surgeon swap control | |
WO2024094838A1 (en) | Operating room dashboard | |
WO2024223462A1 (en) | User interface for participant selection during surgical streaming | |
WO2025021978A1 (en) | Procedure metrics editor and procedure metric database | |
US20240037949A1 (en) | Surgical workflow visualization as deviations to a standard | |
WO2025036994A1 (en) | Context-contingent streaming notification | |
EP4258274A1 (en) | De-identifying data obtained from microphones | |
WO2025036993A1 (en) | Dynamic view selector for multiple operating room observation | |
EP4430833A1 (en) | Compression of catalogue of surgical video |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 24768303 Country of ref document: EP Kind code of ref document: A1 |