WO2012071016A1

WO2012071016A1 - A method for creating multimedia-based radiology reports

Info

Publication number: WO2012071016A1
Application number: PCT/SG2011/000417
Authority: WO
Inventors: Thirunavuukarusuu Arumugam; Guoliang Yang
Original assignee: Agency for Science Technology and Research Singapore
Current assignee: Agency for Science Technology and Research Singapore
Priority date: 2010-11-26
Filing date: 2011-11-25
Publication date: 2012-05-31
Anticipated expiration: 2013-05-26

Abstract

A method is disclosed for creating at a workstation a report of a radiological image, the method comprising the steps of displaying the radiological image on a screen of the workstation, receiving a series of inputs from a user signaling a series of interactions with the displayed radiological image, displaying graphically on the screen the result of the interactions, generating the report, the report incorporating at least part of the radiological image together with data encoding a video (with or without voice narration) indicating the series of interactions, and storing the report in a database for subsequent retrieval. Also disclosed is a method for collaboratively creating a report with another workstation according to the disclosed method.

Description

A Method for Creating Multimedia-Based Radiology Reports

Field of the Invention

The invention relates to a method for creating a report of a radiological image.

Background of the Invention

When creating a report of a radiological image, a radiologist typically dictates his observations and interpretations into a recording device while assessing the image. These dictations are then transcribed into text either manually or using a speech recognition software. The end result is thus a text report narrating the radiologist's findings.

This text report however is static and provides no information beyond textual information. Information which cannot be conveniently expressed using words is not reflected in the text report and this information may be useful for further interpretation and analysis of the radiologist's findings. The absence of the physical presence of the radiologist thus hampers the communication of information from the radiologist to clinicians. Thus, the radiologist has to meet with the referring clinicians in order to discuss about the radiological image and convey information which is not expressed in the text report. This however results in a longer reaction time to a patient's condition and thus a longer turn-around time per patient. Consequently, the cost of radiological report and/or medical care is increased. It is noted that while online video conference solutions such as MSN Messenger and Skype exist, they are not suitable for the sharing of medical images because their traffic is compressed such that image quality suffers, thus rendering them unsuitable for diagnostic purposes.

Summary of the Invention

The present invention aims to provide a new and useful method for creating a report of a radiological image, and a workstation for carrying out the method.

In general terms, the invention proposes a workstation receiving a series of inputs from a user signaling a series of interactions with a radiological image displayed on a screen. A report is then generated at the workstation for storage in a database, the report incorporating at least part of the radiological image together with data encoding a video indicating the series of interactions.

Specifically, a first expression of the invention is a method for creating at a workstation a report of a radiological image, the method comprising the steps of displaying the radiological image on a screen of the workstation;

receiving a series of inputs from a user signaling a series of interactions with the displayed radiological image;

displaying graphically on the screen the result of the interactions; generating the report, the report incorporating at least part of the radiological image together with data encoding a video indicating the series of interactions; and

storing the report in a database for subsequent retrieval.

Such a method produces reports which are better able to convey information between different radiologists and/or different clinicians. These reports when stored in the database serve as a better record of information. Additionally, the stored reports may be used for presenting information during conferences and training sessions.

Preferably, the method further comprises the step of recording a voice or text narration which is associated with the series of interactions, the voice or text narration being included in the report. Such a method allows the report produced to be multi-media, thus further enhancing the report's ability to convey information.

Preferably, the video is compressed using lossless encoding. The use of the term "lossless" is understood to mean that the compressed video has the same image quality as the original image. This ensures that image detail is not lost when compressing the video, thus allowing for further observations or interpretations to be made upon the compressed video.

Preferably, the series of interactions is selected from the group consisting of a scrolling of the displayed radiological image, a drawing of a region of interest (ROI) on the displayed radiological image, an overlay of a registered atlas on the displayed radiological image, and one or more annotations of the displayed radiological image. A variety of interactions is available in order to allow for information to be better captured in the video.

Preferably, the series of interactions is encoded in the video in a vectorized format. This advantageously results in videos which take up less space.

Preferably, the method further comprises the steps of sending the series of inputs from the workstation to another workstation, and displaying graphically on a screen of the other workstation the result of the interactions. Such a method allows a clinician/radiologist to convey information about the

radiological image to other clinicians in a "live" manner. Preferably, the method further comprises the steps of receiving at the workstation from a further workstation a further series of inputs signaling a further series of interactions with the radiological image, and displaying graphically on the screen of the workstation the result of the received further series of interactions; and wherein the report further incorporates data encoding a video indicating the further series of interactions. Such a method allows multiple clinicians to collaborate on producing the report.

Preferably, the database is connected to the workstation by the Internet. This allows the database to be accessed from multiple geographically distributed sites. Preferably, the workstation is a mobile device. This allows the radiologist or clinicians to work while on the move.

A second expression of the invention is a workstation for carrying out the method of the first expression, the workstation comprising

a screen configured to display the radiological image;

an input device configured to receive from the user the series of inputs signaling the series of interactions with the displayed radiological image;

a processor having software configured to display graphically on the screen the result of the interactions and generate the report; and

a transmitter configured to store the report in the database.

A third expression of the invention is a method for collaborative analysis of a radiological image across at least two workstations, the method comprising the steps of at each workstation,

(i.) displaying the radiological image on a respective screen of each workstation;

(ii.) receiving from each user of each workstation a series of inputs signaling a series of interactions with the displayed radiological image;

(in.) streaming the series of inputs from each workstation to the other workstation(s) for real-time display on the respective screens of the other workstations; and

(iv.) displaying graphically on the screen of each workstation the result of the interactions. Such a method allows a multi-way synchronous real time communication of information, thus facilitating discussions and greatly increasing the productivity of the radiologist(s) and clinician(s). A fourth expression of the invention is a method for creating a report comprising the steps of performing collaborative analysis of a radiological image across at least two workstations using the method of the third expression; and

at one of the at least two workstations,

(v.) generating the report, the report incorporating at least part of the radiological image together with data encoding a video indicating the sequences of interactions from the at least two workstations; and

(vi.) storing the report in a database for subsequent retrieval.

Such a method allows a report of the radiological image to be created with the consensus and input of multiple parties from different disciplines.

Certain embodiments of the present invention may have the advantages of:

- boosting openness and discussions between clinicians;

- allowing for faster decision making;

- leading to medical decisions which are more robust;

- allowing a clinician to get instant help from a radiologist who reported on a radiological image;

- ensuring greater case consistency;

- producing a content-rich radiology report; and - permitting the discussion of a radiological image between remotely located consultants and/or researchers.

Brief Description of the Figures

By way of example only, an embodiment will be described with reference to the accompanying drawings, in which:

Figure 1 is a schematic drawing of a system for radiological reporting and/or analysis according to an example embodiment;

Figure 2 is a flow-chart of a method for creating a report of a radiological image at a workstation of the system of Figure 1 ;

Figure 3 is a screenshot of a text report variation of the report of Figure 2 showing embedded links to two videos;

Figure 4 is a screenshot of a Radiology Information System (RIS) variation of the report of Figure 2 showing a patient case with an embedded link to a video;

Figure 5 is a screenshot of a user interface displaying the radiological image of Figure 2; and

Figure 6 is a flow-chart of a variation of the method of Figure 2 which allows the report of Figure 2 to be created with collaboration of another workstation of the system of Figure 1.

Detailed Description of the Preferred Embodiment Figure 1 shows a system 100 for radiological reporting and/or analysis according to an example embodiment. The system 100 comprises a Picture Archiving and Communication System (PACS) 180, a reports database 170, an acquisition terminal 110, and workstations 130 and 150. The acquisition terminal 110 and workstations 130 and 150 are connected to the image repository 180 and reports database 170 by way of respective data networks 120, 140 and 160. The data networks 120, 140 and 160 are for example the Internet, or a local network, and it is envisaged that the mode of connection for the data networks 120, 140 and 160 may either be wired or wireless. Further, the data networks 120, 140 and 160 may be different from each other.

The PACS 180 includes one or more radiological images 185 stored in it. These images 185 for example are X-ray images or CT, MRI and/or PET tomographic images. The acquisition terminal 110 acquires the images from an imaging device (not shown) such as a digital camera, an X-ray scanner, or a

tomographic scanner. The acquired images are then uploaded onto the PACS 180 to form the radiological images 185. These images 185 after being uploaded are retrievable by the workstations 130 and/or 150.

The acquisition terminal 110 may further comprise an input device 114 e.g. a mouse, a transmitter/receiver (not shown) for receiving and transmitting data, processor (not shown) configured to execute software, a screen 112 for displaying images and/or graphics, a pair of headphones 118, speakers 117 and/or a microphone 116. Each of the workstations 130 and 150 comprises an input device (134 and 154 respectively) for receiving inputs from a user, a transmitter/receiver (not shown) for receiving, transmitting data, a processor (not shown) configured to execute software and a screen (132 and 152 respectively) for displaying images and/or graphics. Optionally, each of the workstations 130 and 150 include respective headphones 138 and 158, respective speakers 137 and 157, and/or respective microphones 136 and 156. The workstations 130 and 150 are configured to execute software to retrieve at least one of the images 185 from the PACS 180. The respective users of the workstations 130 and 150 then analyzes and/or creates a report of the retrieved image(s). This may be done either individually by the user at each workstation 130 or 150 or may be done collaboratively between all the workstations 130 and 150. Methods for creating a report of the retrieved image at one of the workstations 130 and 150 are disclosed later with the aid of Figures 2 and 6. It is noted that although the workstations 130 and 150 may have webcams installed, webcams are not required for the methods of Figures 2 and 6.

After creating a report of the retrieved image either individually or

collaboratively, the report is then stored in the reports database 170 for subsequent retrieval by for example the workstations 130 and/or 150. The reports database 170 is a secure database which preserves the privacy of the reports stored within. Optionally, although the reports database 170 is illustrated in Figure 1 to be a separate device from the PACS 180, it is envisaged that the reports database 170 and the PACS 180 may co-exist on a single device and may even be provided by a common software. In the latter case, the reports database 170 may be integrated into the PACS 180. Further, it is envisaged that the acquisition terminal 110 and/or workstations 130 and 150 may be a desktop computer, an embedded device or a mobile computer.

Turning to Figure 2, Figure 2 shows a method 200 for creating a report of a retrieved image at a workstation 130.

In step 210, a radiological image is displayed on a screen 132 of the

workstation 130. Software in the workstation 130 is configured to retrieve the radiological image from the PACS 180 by way of the data network 140. The radiological image is read from the PACS 180 using the Digital Imaging and Communications in Medicine (DICOM) standard. The result of step 210 is illustrated in Figure 5 which shows a user interface of the software displaying a radiological image. In step 220, the workstation 130 receives a series of inputs from a user by way of the input device 134. This series of inputs signal the user's corresponding series of interactions with the displayed image. The interactions include actions performed directly on the image e.g. a scrolling, zooming or editing of the displayed image, or actions overlaid on the image e.g. a drawing of a region of interest (ROI), a text or graphical annotation, or the overlay of a registered atlas on the displayed image. The interactions also include the performing of window leveling i.e. the changing of the brightness and contrast of the image. In step 230, the results of the interaction is displayed graphically on the screen. This involves rendering the results of the interaction onto a graphical layer which is displayed on top of the original radiological image. The original radiological image does not need to be changed and the results of the interaction is rendered onto the graphical layer for overlay onto the original radiological image.

In step 240, a report of the image is generated. This report incorporates data encoding a video indicating the series of interactions and also includes at least part of the radiological image which is displayed on the screen 132 in step 210. In other words, the video which is generated for the report contains the graphical results of the user's interactions with the displayed image.

The video stores the graphical layer (showing the results of the interactions) separately from the original radiological image. By doing so, the radiological image is stored in its original image quality and the graphical layer is stored in a vectorized format. This represents an advantage over performing a direct capture of the contents of the screen 132 because in the case where a direct capture is performed, the radiological image that is captured from the screen 132 would have been down-sampled. In step 250, the report is stored in the reports database 170. The video of the report is compressed before the report is stored. The video is compressed using the method disclosed in PCT publication WO2007/046777, "Encoding, storing and decoding data for teaching radiology diagnosis", the contents of which are incorporated herein by reference. This method encodes the original image losslessly. Optionally however, it is envisaged that the video may be

compressed using any other lossless method which is known in the art. It is envisaged that in an embodiment, the user may provide a voice narration or type a text narration while performing the series of interactions of step 220. In this case, the workstation 130 additionally records the voice or text narration in step 220 and in step 240, the narration is included in the report. In the case where the narration is a voice narration, the video may include an

accompanying audio track carrying the voice narration i.e. it may be said that the voice narration is "recorded" in the video. A voice narration may be recorded at the workstation 130 using the microphone 136. Thus, the video is multi-media is nature and the video may be generated using the method disclosed in US Patent 7,047,235, "A method and apparatus for creating medical teaching files from image archives", the contents of which are incorporated herein by reference.

It is further envisaged that in another embodiment, the video is incorporated into the report either by embedding a link to the video within a text report, or by associating a link to the video with a patient case of a Radiology Information System (RIS).

The former case is illustrated in Figure 3 which shows a display of a text report with embedded links to two videos. In this case, the report includes a text narration 310 providing an analysis of the radiological image and the videos are accessible via embedded links 322, 324 within the report. The embedded links 322, 324 are created automatically within the report when the report is generated in step 240. The report and the associated videos are all stored in the reports database 170 in step 250. The embedded links 322, 324 take the form of pictorial representations of the respective videos and when a link is clicked, the corresponding video is streamed from the reports database 170 and played in a player. Should the video contain an audio narration, the narration is played back via the speakers 137 or the headphones 138.

In the latter case, the video is incorporated into the report by associating a link to the video with a patient case of a RIS. This is illustrated in Figure 4 which shows a display of a RIS with a patient case having an embedded link to a video. The RIS has a work-list 410 which lists a plurality of patient cases.

Each patient case forms a report including the patient's personal information and case history, and is associated with a video analysis of the one or more radiological images. When the report is generated in step 240, the patient case is generated along with the video and its associated link. Thereafter, when the RIS is displayed, the newly generated patient case is visible in the work-list 410. By clicking on the link associated with a patient case, the video is streamed from the reports database 170 and played in a player. Should the video contain an audio narration, the narration is played back via the speakers 137 or the headphones 138.

In yet another embodiment, the method 200 further comprises steps which allow the report to be created at the workstation 130 with the collaboration of another workstation 150. This method 600 is shown in Figure 6 where the steps 610, 620, 630, 640 and 650 respectively each include the steps 210, 220, 230, 240 and 250 as performed at the workstation 130.

By clicking a "discussion" button in the user interface of the workstation 130, the method 200 of creating a report is switched into the method 600 of

collaboratively creating the report. Figure 5 shows this "discussion" button as 510. When this is done, a notification is sent to all other parties involved in the discussion i.e. the other workstation 50. If a RIS is used at the workstation 150, the radiologist receives this notification as a new patient case in the work- list 410. The radiologist can then accept the discussion notification by clicking on the new patient case.

In step 610, the radiological image retrieved from the PACS 180 is displayed on the respective screens 132 and 152 of the workstation 130 and other workstation 150. In step 620, the workstation 130 receives a series of inputs from a user by way of the input device 134. This series of inputs signal the user's series of interactions with the image displayed on screen 132. The interactions which are possible are the same that of step 220.

In step 622, the workstation 130 from the other workstation 150 a further series of inputs. This further series of inputs is obtained from another user by way of the input device 154, thus signaling the other user's series of interactions with the image displayed on screen 152.

In step 624, the series of inputs is sent from the workstation 130 to the other workstation 150.

Thus, the steps 620, 622 and 624 collectively represent at each workstation, the receiving of inputs signaling each users' interactions with the displayed radiological image, and the sending of the received inputs to the other workstations. As the inputs at each workstation are sent to the other

workstations synchronously once the inputs are received from the respective input devices, it is thus said to be "streamed" to each other. Audio narration (if present) in this case is also streamed bi-directionally between the workstations 130 and 150. The audio narration may be captured at each of the workstations 130 and 150 via respective microphones 136 and 156. Audio narration may be played back at each of the workstations either on respective speakers 137 and 157 or respective headphones 138 and 158. In step 630, the results of both series of interactions (received from the input devices 134 and 154) are displayed graphically on respective screens 132 and 152 of the workstations 130 and 150. In other words, the screens 132 and 152 both show the combined results of both users' interactions with the display radiological image. The contents of the user interface of both screens 132 and 152 thus are identical.

In step 640, the workstation 130 generates a report of the radiological image. This report incorporates data encoding a video indicating both users' combined interactions with the radiological image. The video is stored in the same fashion as is done in step 240.

In step 650, report is stored in the reports database 170 in the same fashion as is done in step 250.

It is envisaged that the radiological image retrieved from the PACS 180 in step 610 may be a moving radiological image, that is for example a video of a radiological scan, or a video that is associated with a report stored in the reports database 170. In the case where it is the latter, the method 600 becomes a method for collaborative discussion of a radiological report. Additionally, it is understood that while the method 600 is described using only two workstations 130 and 150, the method 600 can be used with any number of workstations more than two.

Whilst example embodiments of the invention have been described in detail, many variations are possible within the scope of the invention as will be clear to a skilled reader.

Claims

1. A method for creating at a workstation a report of a radiological image, the method comprising the steps of

displaying the radiological image on a screen of the workstation;

displaying graphically on the screen the result of the interactions;

generating the report, the report incorporating at least part of the radiological image together with data encoding a video indicating the series of interactions; and

storing the report in a database for subsequent retrieval.

2. The method according to claim 1 wherein the method further comprises the step of

recording a voice or text narration which is associated with the series of interactions, the voice or text narration being included in the report.

3. The method according to any preceding claim wherein the video is compressed using lossless encoding.

4. The method according to any preceding claim wherein the series of interactions is selected from the group consisting of

a scrolling of the displayed radiological image; a drawing of a region of interest (ROI) on the displayed radiological image;

an overlay of a registered atlas on the displayed radiological image; and one or more annotations of the displayed radiological image.

5. The method according to any preceding claim wherein the series of interactions is encoded in the video in a vectorized format.

6. The method according to any preceding claim further comprising the steps of

sending the series of inputs from the workstation to another workstation; and

displaying graphically on a screen of the other workstation the result of the interactions.

7. The method according to any preceding claim further comprising the steps of

receiving at the workstation from a further workstation a further series of inputs signaling a further series of interactions with the radiological image; and displaying graphically on the screen of the workstation the result of the received further series of interactions; and

wherein the report further incorporates data encoding a video indicating the further series of interactions.

8. The method according to any preceding claim wherein the database is connected to the workstation by the Internet.

9. The method according to any preceding claim wherein the workstation is a mobile device.

10. A workstation for carrying out the method according to any of claims 1 to 9 comprising

a screen configured to display the radiological image;

a transmitter configured to store the report in the database.

11. A method for collaborative analysis of a radiological image across at least two workstations, the method comprising the steps of

at each workstation,

(ii.) receiving from each user of each workstation a series of inputs signaling a series of interactions with the displayed radiological image; (Hi.) streaming the series of inputs from each workstation to the other workstation(s) for real-time display on the respective screens of the other workstations; and

(iv.) displaying graphically on the screen of each workstation the result of the interactions.

12. A method for creating a report comprising the steps of

performing collaborative analysis of a radiological image across at least two workstations using the method of claim 11 ; and

at one of the at least two workstations,

(vi.) storing the report in a database for subsequent retrieval.