JPWO2021228541A5

JPWO2021228541A5 -

Info

Publication number: JPWO2021228541A5
Application number: JP2022568527A
Authority: JP
Publication date: 2024-05-09

Claims

1. A non-transitory computer readable medium storing instructions executable by at least one electronic processor for performing a method for providing remote expert assistance to a local operator of a medical imaging device during a medical imaging examination, the method comprising:
extracting image features from image frames displayed on a display device of a controller of the medical imaging device operable by the local operator during the medical imaging examination , the image features comprising one or more of a position on the display device of the controller of the medical imaging device, a text label, a type of information, a type of encoding, a type of formatting, a translation table or icon, and a shape or color;
converting the extracted image features into a representation of a current status of the medical imaging examination , the converting step comprising inputting the extracted image features into a generic imaging examination workflow model operable by the local operator to generate the representation, the workflow model being independent of a format of the image features displayed on the display device of the controller, the representation comprising one or more of: number of scans, remaining scan time, weight value of the patient to be scanned, time elapsed since start of the medical imaging examination, number of rescans, name of a scanning protocol, progress of a current medical imaging examination, heart rate of the patient to be scanned, and respiratory rate of the patient to be scanned;
and providing a user interface on a workstation operable by the remote expert that displays the representation.

The method,
inputting said representation into an imaging exam workflow model indicating a current state of said medical imaging exam;
and providing the imaging inspection workflow model on the workstation operable by the remote expert.

the extracted image features include data entered by the local operator into the controller and displayed on the display device of the controller, the method comprising:
3. The non-transitory computer readable medium of claim 2 , further comprising updating the imaging inspection workflow model provided on the workstation operable by the remote expert with the data entered by the local operator.

The method,
identifying a trigger event in the imaging exam workflow model at which action needs to be taken by the remote expert and/or the local operator;
and outputting, via the user interface operable by the remote expert, an alert indicative of the trigger event.

translating the extracted image features into a representation of a current status of the medical imaging examination,
identifying one or more of the extracted image features from the image frames as personally identifiable information of a patient to be scanned during the medical imaging examination;
generating a modified image frame from the image frame displayed on the display device of the controller by one of deleting the identified personally identifiable information feature from the image frame or replacing the personally identifiable information in the image frame with text, a symbol, or a color;
and displaying the modified image frames as a video feed presented on the user interface on the workstation operated by the remote expert.

extracting image features from image frames displayed on the display device of the controller operable by the local operator during the medical imaging examination,
identifying a dialog screen template corresponding to a dialog screen shown in an image frame, the corresponding dialog screen template identifying one or more screen regions and associating the one or more screen regions with a setting for the medical imaging examination;
6. The non-transitory computer readable medium of claim 1, further comprising: extracting information from the one image frame and associating the extracted information in the one or more screen regions with settings of the medical imaging examination using an association provided by the corresponding dialog screen template.

7. The non-transitory computer-readable medium of claim 1, wherein the extracted information includes one or more of: a location of the image feature on the display device, a text label of the image feature, a type of information of the image feature, a type of encoding of the image feature, a type of formatting of the image feature, a translation table or icon of the image feature, and a shape or color of the image feature.

extracting image features from image frames displayed on the display device of the controller of the medical imaging device operable by the local operator during the medical imaging examination,
performing optical character recognition on the image frames displayed on the display device of the controller operable by the local operator to extract text information;
extracting average color values over the image frames displayed on the display device of the controller operable by the local operator to extract color information;
and performing a pattern comparison operation on the image frames displayed on the display device of the controller operable by the local operator with images stored in a database.

The method further comprising:
extracting the image features from the image frames displayed on the display device of the controller using screen sharing software operating on the controller ; or
receiving, at the workstation operated by the remote expert, a video feed capturing the display device of the controller operated by the local operator;
displaying the video feed at the workstation operated by the remote expert;
extracting the image features from the received video feed;
9. The non-transitory computer readable medium of claim 1, further comprising:

The method,
A non-transitory computer-readable medium as described in any one of claims 1 to 9, further comprising a step of displaying the representation of the current status of the medical imaging examination at the workstation operated by the remote expert using a standard display format that is independent of the medical imaging device operated by the local operator during the medical imaging examination.

A non-transitory computer-readable medium as described in any one of claims 1 to 10, wherein the method is performed at a plurality of sites including medical imaging devices operated by a corresponding number of local operators, and the representation includes information from a site of the plurality of sites , in particular a list indicating the status of the medical imaging examination at a corresponding site .

1. An apparatus for providing assistance from a remote expert to a local operator during a medical imaging examination performed using a medical imaging device, the apparatus comprising:
a workstation operable by said remote expert;
extracting image features from image frames displayed on a display device of a controller of the medical imaging device operable by the local operator during the medical imaging examination , the image features comprising one or more of a position on the display device of the controller of the medical imaging device, a text label, a type of information, a type of encoding, a type of formatting, a translation table or icon, and a shape or color;
Transforming the extracted image features into a representation of a current status of the medical imaging examination by inputting the extracted image features into an imaging examination workflow model indicative of a current state of the medical imaging examination, the imaging examination workflow model having a standard format independent of a format of the image features displayed on the display device operable by the local operator to generate the representation, the representation including one or more of: number of scans, remaining scan time, weight value of a patient to be scanned, time elapsed since start of the medical imaging examination, number of rescans, name of a scanning protocol, progress of a current medical imaging examination, heart rate of the patient to be scanned, and respiratory rate of the patient to be scanned;
and providing a user interface on the workstation operable by the remote expert, the user interface displaying at least one of the representation and the imaging inspection workflow model.

The at least one electronic processor:
13. The apparatus of claim 12, further programmed to output an alert when the imaging exam workflow model reaches a procedure during the medical imaging exam at which action needs to be taken by either the remote expert or the local operator.

the at least one electronic processor:
identifying one or more of the extracted image features from the image frames as personally identifiable information of a patient to be scanned during the medical imaging examination;
generating a modified image frame from the image frame displayed on the display device of the controller by one of deleting the identified personally identifiable information feature from the image frame or replacing the personally identifiable information in the image frame with text, a symbol, or a color;
14. The apparatus of claim 12 or 13, further programmed to convert the extracted image features into a representation of a current status of the medical imaging examination by displaying the modified image frames as a video feed presented on the user interface on the workstation operated by the remote expert.

1. A method for providing assistance from a remote expert to a local operator during a medical imaging examination, the method comprising:
extracting image features from image frames displayed on a display device of a controller operable by the local operator during the medical imaging examination , the image features comprising one or more of a position on the display device of the controller of the medical imaging device, a text label, a type of information, a type of encoding, a type of formatting, a translation table or icon, and a shape or color;
identifying one or more of the extracted image features from the image frames as personally identifiable information of a patient to be scanned during the medical imaging examination;
generating modified image frames from the image frames displayed on the display device of the controller by one of removing the identified personally identifiable information features from the image frames or replacing the personally identifiable information in the image frames with text, a symbol, or a color; and converting the extracted image features into a representation indicative of a current status of the medical imaging examination.
inputting the representation indicating the current status of the medical imaging examination into an imaging examination workflow model indicating a current state of the medical imaging examination , the imaging examination workflow model being independent of a format of the image features displayed on the display device of the controller operable by the local operator to generate the representation, the representation including one or more of the following: number of scans, remaining scan time, weight value of a patient to be scanned, time elapsed since start of the medical imaging examination, number of rescans, name of a scanning protocol, progress of a current medical imaging examination, heart rate of the patient to be scanned, and respiratory rate of the patient to be scanned;
and providing, on a workstation operable by the remote expert, a user interface (UI) displaying the modified image frames as a video feed, an abstract representation, and the imaging inspection workflow model.