EP1103028A2

EP1103028A2 - Surgical recording and reporting system

Info

Publication number: EP1103028A2
Application number: EP99935009A
Authority: EP
Inventors: Nitzan Sneh
Original assignee: Contec Medical Ltd
Current assignee: Contec Medical Ltd
Priority date: 1998-08-04
Filing date: 1999-07-27
Publication date: 2001-05-30
Also published as: WO2000008585A3; IL140472A0; WO2000008585A2

Abstract

A series of digital video snapshot pictures taken by a surgeon (12) during endoscopic surgery is correlated with a series of comments made by the surgeon during surgery. The pictures and comments are displayed on a monitor (20) in an associated correlated format after surgery to allow the surgeon to select those images and comments deemed most appropriate for inclusion in a surgical report. The selected images and comments are downloaded into a word processing software package which combines the images and audio comments with other information such as patient data and surgery data MsTEP 7). A final report is produced with preformatted electronically produced templates displaying text and images and allowing for the audio replay of the surgeon's comments (Step 8).

Description

Surgical Recording and Reporting System

Background of the Invention Field of the Invention

The present invention relates in general to an audio-visual recording system for documenting surgical and diagnostic procedures and in particular to such a system which

integrates and collates video camera images with verbal dictation recorded during surgery and which provides for the subsequent selection and entry of such audio-visual data into a

Preformatted report. Description of Prior Developments

The preparation of medical reports, and particularly those documenting surgical procedures, can be a time consuming and burdensome task. As a surgeon's time can be more effectively spent practicing surgery than drafting reports, a more efficient method and system is needed to prepare such reports. A particular need exists for such a method and system which can record both audio and video data documenting a surgical procedure and facilitate the integration

of such data into a preformatted report.

Summary of the Invention

The present invention has been developed to fulfill the needs noted above and therefore has as an object the provision of a method and system for recording, integrating and reporting

audio and video data documenting a surgical procedure.

Another object of the invention is to minimize the time, effort and duplication involved in

recording and editing both video data and audio data provided by a surgeon during a surgical procedure for subsequent preparation of a surgical and diagnostic report.

Still another object of the invention is the provision of a method and system which

integrates specific patient intake data obtained prior to surgery with selected video data taken during the surgery and with selected audio data provided by a surgeon during surgery and which presents such data in a preformatted audio-visual textual report.

Yet another object of the invention is the provision of a method and system which allows

a surgeon to select one or more video images of a surgical procedure, correlate such image or images with corresponding audio descriptions of such image or images and incorporate the correlated audio and visual information into a pre-formatted written report.

These and other objects are met by the present invention which is directed to a method and system which records audio and visual data during surgery. The method and system are

particularly effective when used during endoscopic surgery and endoscopic procedures. The invention provides electronic management and archiving of surgery records while providing online consultation with remote sites via existing communication networks. The preparation of reports on a user friendly operating system increases the efficiency and effectiveness of health professionals and facilitates faster report processing and payment from health insurance

companies.

The present invention serves as a source of detailed and accurate surgical information and documentation by, during surgery, capturing dictation together with full-color surgical images relating to the dictation. The system, as constructed in accordance with the invention, can

enlarge the images to highlight specific areas of interest, store or print color images on paper, transparencies or slides, and transfer the images and data over computer and communications networks in real-time, thereby facilitating on-line analysis and consultations with remote sites.

The system of the present invention stores all collected data digitally. This enables easy access for future use. Images, texts and audio recordings can be easily recalled for a range of applications, including evaluation of patients, and preparation of post-operative reports, lectures and presentations on related subjects. Such electronic management of medical data enables easy selection of information according to any defined criteria. The invention also can provide a powerful marketing tool for every facility where it is

employed, indicating investment in modern systems. The photographic data incorporated in the reports generated by the system is particularly beneficial when validating and indicating medical

needs and procedures to health insurance companies, thereby facilitating quicker reimbursements

and providing increased liability protection.

The present invention further enables the easy creation of reports by selecting data from a predefined, customized library of texts and templates thereby saving precious time and reducing human errors. By operating on a Pentium PC processor using Windows NT software, the present invention is fast and reliable, fully compatible and integrating with any Windows-based software applications. The system can also input data from various image databases, including MRI, CT, ultrasound and X-ray.

While capturing the audio and visual data during endoscopic surgery or endoscopic procedures on a personal computer under Windows NT software, the system associates a recording of the surgeon's dictation with the relevant pictures selected by the surgeon, thereby liberating the surgeon from subsequently typing the explanations and descriptions of the pictures

and procedure and thereby saving the surgeon^'s time. Direct integration with a widely available word processor (Word) enables viewing and editing of the report at home, in the office and in any place where Word software is installed. Editing with popular word processor program software (Word) that many people are familiar with saves on the time needed to learn the system.

Changes in the report are in real time. Any changes done in the report are automatically saved in the system and the user can see the updated version immediately. A file/insert feature enables the integration into the report of any valuable information, both digital and not digital (such as X-Ray, MRI, ultrasound, patient history etc.) to create a complete patient record. A spelling feature is integrated in the system and a medical vocabulary may be included. Image zoom enlargement is provided for optimal review of video image details. Export of these pictures such as by digital saving of the pictures on a computer disk or diskette or other media in a bitmap format enables the creation of slides, transparencies and articles in addition to

picture processing for diagnosis and further allows for networking of the visual information for

consulting and training. The system advantageously includes a PC (personal computer) using standard software (Access database, Word, NT) allowing the customization of the system to the specific needs of any user in a very fast and affordable manner. Backward compatibility

provided.

The system is easy to use for any computer novice yet enables those who have some understanding of computers to optimize the efficiency and effectiveness of the system. Permanent, easy, and reliable archiving and archiving statistics are made available in the form of a digital multidimensional database (audio, visual and text). The information can be retrieved according to any predefined criteria such as a date, patient name, surgeon name etc.

Integration of continuous speech recognition from dictation can create a report more quickly than ever before possible. The use of key words inserts predefined paragraphs into the report. The integration of voice commands for various functions, such as capture of the pictures and recording dictations further increases the speed and efficiency of report creation. The system can be adapted to accept and read smart and magnetic cards and automatically fill in the patient details from the data on the cards. Data safety can be achieved with passwords and data

encryption.

System components include, in one example of the invention:

Pentium PC 300, 4GB, Windows NT, Word 7, video and display cards

Monitor (Trinitron)

Printer (HP 720)

U.P.S. (Uninterrupted power supply for data safety) Foot Switch and Remote Control Microphone

Speakers Keyboard Mouse

The work flow which defines the method of the invention includes several basic stages. In the case of new surgery:

1. Patient data is entered or transmitted into the system by an administrative assistant when a patient is admitted to a hospital or clinic.

2. Surgery data is entered into the system by an assistant before the surgery/ diagnostic procedure.

3. During surgery, digital pictures of the surgery and dictations corresponding to the pictures are recorded on audio and video cards provided in the system. The system provides simultaneous access and retrieving to the surgeon's monitor of previous examinations, images and reports. A video setup controls the video parameters in real time.

4. When the surgery is over, a new report is created and edited. From all the recorded pictures and dictations during the surgery, the surgeon chooses only the desirable pictures with their associated recorded descriptions.

The system software captures the time associated with a particular picture when the picture is captured and recorded. It also associates the time when the audio is captured and

recorded. The video is stored in a standard video storage medium available as standard computer cards installable in most PC's. Likewise, the audio is stored in a standard audio package provided in the computer in the form of an audio card. After the operation, the surgeon calls up the video images on the computer. The video images are now displayed, by way of example, twelve on a screen, each with a time associated

with it. The system can also call up the audio recordings corresponding to one or more of the

video pictures. Each dictation is displayed on the side of the screen as a listing of all of the audios with their recording time. By looking at the recording times of the audio and visual recordings, the surgeon can easily coordinate which audio goes with which video. The surgeon

can also select which of these he wants to include in his report. After making the selection or selections and associating the audio recordings with the video recordings, the correlated audiovisual recordings can automatically be transferred to the surgeon's standard work processing package, such as Microsoft Word, or the like.

As noted above, a surgery report is typically preformatted in the system's work processing package with patient biographical information provided as part of such a report. When the surgeon reenters into the word processing package, the surgeon can integrate his selected video/audio recordings directly into the preformatted report. One of the benefits is that reporting is done in a standard word processing package. This facilitates use of the system. Writing a separate software program to achieve this interaction makes it complex and doctors may not want to learn such separate systems. However, by utilizing a standard work processing package, it facilitates use by the doctor. One of the significant features of the invention is the transfer of the video and audio files into the word processing package so that a surgeon can formulate a report in the work processing package.

For future use, any report can be viewed and edited. The retrieval and selection may be according to any predefined criteria such as patient name, surgeon name etc. The reports can be deleted as well, according to privilege. It is possible to define reports as "read only" in order to prevent unauthorized changes and loss of clinical date. The aforementioned objects, features and advantages of the invention will, in part, be pointed out with particularity, and will, in part, become obvious from the following more

detailed description of the invention, taken in conjunction with the accompanying drawings, which form an integral part thereof.

Brief Description of the Drawings

In the drawings:

Fig. 1 is a flow chart depicting a sequence of process steps for carrying out a recording and reporting method in accordance with the invention;

Fig. 2 is a schematic view of a surgical recording system constructed in accordance with

the invention;

Fig. 3 is a view of a computer monitor screen showing a patient data template produced in accordance with the invention and depicting a typical set of patient input data;

Fig. 4 is a view of a computer monitor screen similar to Fig. 3, showing an additional surgery data template produced with the system of the present invention;

Fig. 5 is a view of a computer monitor screen depicting raw audio and visual data for

producing a medical report in accordance with the invention; and

Fig. 6 is a view of a computer monitor screen of an edited medical report selected and

created from the data provided from Fig. 3, 4 and 5, including a textual description.

Attached Appendix is a printout of the source code of the software of this invention.

In the various figures of the drawings, like reference characters designate like parts. Detailed Description of the Preferred Embodiment

The invention will now be described in conjunction with the drawings, beginning with

Fig. 1 which depicts a sequence of steps for carrying out the invention. The first step includes entering certain patient information into a word processor using a common, commercially available word processing package such as Microsoft Word. This software is used with the system of Fig. 2 to generate one or more data entry templates such as the patient data template and patient comments template depicted on the monitor screen shown in Fig. 3.

Data, such as the location and frequency of pain, time of day of pain and other data on

pain may be entered under the patient comments template of Fig. 3. Biographical and

demographical data may be entered on the patient data template. Additional surgery data may be provided on the surgery data template created by the system in accordance with the invention as shown in Fig. 4.

At this point, the system has been provided with sufficient background information for

recording and reporting purposes to enable a surgeon to begin inputting audio and visual data during surgery. This completes the first step of the flow chart of Fig. 1.

The second step of the flow chart of Fig. 1 is carried out as the patient is operated on, as schematically depicted in Fig. 2. In this particular example, a patient 10 is undergoing an endoscopic procedure wherein a surgeon 12 is operating on the patient with the aid of an endoscope 14. Endoscope 14 includes a camera which transmits a video signal 15 and image of the site of the surgery, such as the patient's gall bladder, to an audio video recording and storage device 16 typically installed on a standard PC such as computer 30.

A standard commercially available video memory board or "card" 18 can be used for video storage of signal 15. The video signal 15 is also transmitted to a surgeon's monitor 20 to enable the surgeon to view the surgery site. An optional or alternate video camera 22 may be used to provide an alternate or supplemental video signal 24 to monitor 20 and to card 18 to display and record the surgery or procedure.

Although video signal 15, or alternatively video signal 24, are constantly displayed on monitor 20 (as well as any number of remote viewing monitors), they are not constantly recorded

on video card 18. Rather, only certain portions of the surgical procedure are captured and recorded by the video card 18. These recorded poπions are selected by the surgeon 12 activating a switch such as foot switch 26 which takes a video snapshot of the surgery at that instant in time

when the switch is activated. Any suitable switching mechanism may be used instead of a foot

switch. For example, a hand switch such as the left button 19 on mouse 21 or a voice activated switch 23 in microphone 25 may be used to trigger the storage device 16 to record an instantaneous picture of the surgery on video card 18.

As the surgery progresses, the surgeon can record as many snapshot pictures as desired for later review. Each time a snapshot is recorded, a time clock 28, typically provided in computer 30, associates a time with each video snapshot. This time label or association is provided by the system software program and is depicted in Fig. 1 as step 3.

The video card 18 is in this example is a commercially available package which comes with its own imaging library package. It includes all the features for acquisition, data manipulation, graphics and display control. The package has been designed for fast application prototyping and ease of use. It has a completely transparent management system and entails virtual rather than physical data objet manipulation, allowing for platform independent applications. This package handles physical objects (system devices, digitizers, display

controllers and data buffers) as virtual objects. These virtual objects must be allocated before they can be manipulated and must be released when they are no longer required.

When a capture screen event is identified by the triggering of switch 19, 23 or 26, two pictures of the screen are captured, a small one and a big one, and saved on card 18. The big picture enables one to zoom the picture and maintain a high quality of picture resolution. The

big picture is typically exported for transparencies, slides and training purposes where high quality and size are essential. For the surgeon's report, the smaller size picture is sufficient and avoids wasting the system storage. Two different buffers are defined in order to hold the different image data so it can be displayed, manipulated and grabbed. The screen is frozen, the image is grabbed and saved in the corresponding buffer using the following functions: 1)

digitizer.halt, 2) digitizer.grab, 3) buffer.save filename.

If there is no audio recording at the same time that the image is recorded, the system will

provide an audio feedback that the image was indeed captured. The audio confirmation of the capture is very useful to the surgeon who is not looking on the screen of monitor 20. It is transparent to the system whether the mouse was clicked, a remote control switch activated or the foot switch activated. A video set-up button enables one to control the video parameters such as hue, saturation contrast and brightness during the surgery. The picture recording can be updated

in real time. PAL or NTSC source can be as well redefined during the surgery with no need to restart the system.

The system software has been developed with VB which is an "event driven" language meaning that whenever an event is identified by the system something happens. When a "mouse down" event happens the system recognizes that somebody clicked on the mouse. If (button=l)

the system knows that the left mouse button was pushed and the screen (the image) is recorded, otherwise if (button=0) the audio is recorded. Audio recording is accomplished with microphone 23 which may be voice activated or activated by pressing the right mouse button 17. When an audio record is identified the dictation is recorded. This corresponds to step 4 in Fig. 1.

The audio dictation is stored in a standard commercially available audio card 34 provided

in storage device 16 connected to and being part of computer 30. A dictate button will change into end dictate and an audio confirmation is provided so the surgeon will know that the dictation is recorded. A dictation timer is typically set on 28 seconds but it can be set up to 48 seconds which is a Windows limitation. When the timer limit is reached, the dictation is stopped automatically. If the surgeon wants to continue the dictation, an additional push on the right

mouse button will start recording again. By clicking on the right button again, the surgeon can stop the dictation before the timer is activated. After the surgery, a full report of the surgery needs to be created. The surgery report to be created is selected according to the patient name, the surgeon name, date, etc. The "pre-

report" form for the selected surgery will pop up when selected by the surgeon. All the small

pictures recorded for the surgery will be displayed as shown in Fig. 5. When clicking on the right button of the mouse next to any picture, a pop-up menu as shown in Fig. 5 will pop up with the following options: 1) send to report, 2) remove from report, 3) zoom the picture, 4) export the picture, 5) cancel. The surgeon then selects one or more pictures and their associated audio descriptions. This corresponds to step 6 in Fig. 1.

By default, all the pictures are sent to the report and when selected have a blue frame to indicate they were chosen for the report. When removed; from the report, a picture will no longer have the blue frame, and of course it will not be in the report. When zooming a specific picture, the bigger image will be displayed. Exporting will bring the Save option of Windows, so the selected picture can be saved on the hard drive or on a diskette which can be later transferred to any remote site or transformed into slides, transparencies, etc. There is virtually no limitation on

the number of pictures to be stored.

If more than 12 pictures were captured for the surgery, the right and left arrow buttons will scroll the pictures to control their display on the screen. As the capture time for every picture and every dictation is stored in the system and displayed on the preformatted surgical report template or form, the doctor can easily integrate one with the other.

To manipulate the audio recording, a toolbar like that of a tape recorder is used. The user can stop, rewind forward and play any selected dictation. Next to each dictation there is a

checkbox indicating whether the dictation will go to the report or not. When clicking on the "report button" in Fig. 5, the word processing integration occurs as indicated by step 7 in Fig. 1. The system creates a Word object, loads the predefined template, handles patient and surgery data (the information typed before the surgery is automatically entered to the report), inserts

11

^' audio voice, inserts free text/comments, inserts the preselected pictures according to the selected

layout (vertical or horizontal) and saves surgery documents. At this point a final report is created, examples of which are shown in Fig. 6 and 7. The creation of the final report

corresponds to step 8 in Fig. 1.

There has been disclosed heretofore the best embodiment of the invention presently contemplated. However, it is to be understood that various changes and modifications may be

made thereto without departing from the spirit of the invention.

Claims

What is claimed is:

1. A system for creating audio-visual surgical and diagnostic reports, comprising: a video camera for recording surgical images; a video storage device for receiving and storing said surgical images produced by said camera; a first switch associated with said video storage device for selecting certain ones of said

surgical images for storage in said video storage device; a microphone for recording surgical comments; an audio storage device for receiving and storing said surgical comments transmitted via said microphone;

a second switch associated with said audio storage device for entering and storing said surgical comments in said audio storage device; a timer associated with said video storage device and with said audio storage device for assigning a time to each of said certain ones of said surgical images and to each of said surgical comments; and a computer operatively associated with said video storage device and said audio storage device and having first software for combining said surgical comments and said certain ones of said surgical images on a preformatted report template produced by said first software.

2. The system of claim 1, further comprising an endoscope and wherein said video camera

is provided on said endoscope.

3. The system of claim 1, wherein said video storage device comprises a video card

connected to said computer.

4. The system of claim 1, wherein said audio storage device comprises an audio card connected to said computer.

5. The system of claim 1 , wherein said first switch comprises a foot-operated switch.

6. The system of claim 1. further comprising a mouse switch connected to said computer

and wherein said first switch is provided on said mouse switch.

7. The system of claim 1, further comprising a mouse switch connected to said computer and wherein said second switch is provided on said mouse switch.

8. The system of claim 1, further comprising second software for producing a patient data

template.

9. The system of claim 1 , further comprising third software for producing a patient comment template.

10. The system of claim 1, further comprising fourth software for producing a surgery data template.

11. The system of claim 1 , wherein said report template comprises at least one of said certain ones of said surgical images and at least one of said surgical comments describing said at

least one of said surgical images.

12. A digitally recorded and reproduced audio-visual surgical report, comprising: at least one digital picture of a surgical procedure; and at least one digital audio recording of a surgeons' comments relating to said at least one digital picture.

13. The report of claim 1, wherein said report is stored on an electronically readable medium.

14. The report of claim 13, wherein said medium comprises a computer disk.

15. The report of claim 12, further comprising a textual patient data report portion.

16. The report of claim 12, further comprising a textual surgery date report portion.

17. The report of claim 12, further comprising a textual surgical procedure report portion.

18. A method of preparing a digital report of a surgical procedure, comprising: entering patient data into a word processor using a word processing package; recording and storing a digital video image of a portion of said procedure on a video

computer card; associating and recording a time with said video image;

recording and storing a digital audio report of said video image on an audio computer

card; associating and recording a time with said audio report; and combining said audio report and said video image on a preformatted digitally produced

template.

19. The method of claim 18, further comprising: sequentially recording and storing a plurality of video images on said video computer card; sequentially recording and storing a plurality of audio reports on said audio computer

card; selecting at least one of said video images and selecting at least one of said audio reports and transferring said selected video images and selected audio reports to a word processing software package; and combining said selected reports and selected video images on said preformatted digitally

produced template.

20. The method of claim 19, further comprising entering patient data on said template.