RU2800312C1 - Doctor's fatigue assessment system and method - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: system for implementing a method for estimating a doctor's fatigue, which comprises the steps of displaying a medical image on a display unit for viewing by a doctor, wherein the medical image contains an area to be diagnosed; dividing the area on the medical image to be diagnosed into segments using a neural network; tracking the doctor's gaze while viewing the displayed medical image, with tracking including identification of the location of the physician's gaze on the displayed image; measuring the coverage of each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data; and performing an estimate of the doctor’s fatigue based on segment coverage measurements.

EFFECT: increased automation, accuracy and speed of assessment, availability at the doctor's workplace, reduction of the influence of the fatigue factor on the accuracy of diagnostics of medical images.

6 cl

Description

The field of technology to which the invention belongs

The present invention relates to the field of measurements for diagnostic purposes, and more particularly to a system and method for assessing physician fatigue.

State of the art

The workload of radiologists has steadily increased in the last decade, mainly due to the increase in the number of medical images obtained for the purposes of diagnosis, treatment planning and post-treatment monitoring. There are studies showing that the quality of decisions made by doctors based on the results of medical imaging diagnostics is not ideal: up to 4-10% of diagnoses are erroneous. About 60-80% of the errors are perceptual errors, where anomalies are overlooked, while the rest are cognitive errors, where anomalies are seen but misinterpreted. Radiological errors are influenced by various factors, including a priori expectations, visual bias, experience, and fatigue. The present invention is directed to assessing physician fatigue in order to be able to eliminate or at least mitigate one of the factors mentioned.

Although general fatigue seems to be a subjective feeling, there are relatively objective methods for studying fatigue, which can be divided into 2 groups: questionnaires and instrumental methods. The first group of methods for studying fatigue uses a questioning (survey) of a doctor and the definition of fatigue based on the analysis of responses. However, filling out the answers to the questions takes time, can cause psychological discomfort for the doctor, and the accuracy of such methods is not high enough. The second group of methods involves complex laboratory measurements, such as measuring the concentration of oxygenated hemoglobin in the prefrontal cortex or fixing changes in the electroencephalogram. Although the accuracy of such measurements is quite high, they are not available at the workplaces of radiologists.

Accordingly, there is a need in the art for a system and method for assessing physician fatigue that is available in the physician's workplace and does not require the physician's time to perform the assessment.

The essence of the invention

In one aspect of the present invention, a physician fatigue rating system is provided, comprising:

a display unit configured to display a medical image for viewing by a physician, the medical image comprising an area to be diagnosed;

a segmentation unit, configured to divide an area in the medical image to be diagnosed into segments using a neural network, and transmit the segmentation data to the measurement unit;

a tracking unit configured to track a physician's gaze while viewing the displayed medical image and transmit gaze tracking data to the measurement unit, the tracking unit configured to fix the position of the physician's gaze on the displayed image;

a measurement unit, configured to measure the coverage of each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data, and transmit the measurement results to the assessment unit; And

an evaluation unit configured to evaluate the doctor's fatigue based on the measurement results.

In one of the embodiments, the tracking unit is additionally configured to fix the point in time corresponding to the location of the doctor's gaze on the displayed image, and

the measurement unit is further configured to measure the time taken to cover each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data.

In another aspect of the present invention, there is provided a method for assessing physician fatigue, comprising the steps of:

displaying on the display unit a medical image for viewing by a physician, the medical image containing an area to be diagnosed;

dividing the area on the medical image to be diagnosed into segments using a neural network;

tracking the physician's gaze while viewing the displayed medical image, the tracking comprising fixing the location of the physician's gaze on the displayed image;

measuring the coverage of each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data; And

performing an assessment of physician fatigue based on segment coverage measurements.

In one of the embodiments, tracking further comprises capturing a point in time corresponding to the location of the doctor's gaze on the displayed image,

wherein the method further comprises measuring the time taken to cover each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data,

moreover, the assessment of the doctor's fatigue is performed additionally based on the results of measurements of the segment coverage time.

Technical result

The present invention improves the efficiency of systems and methods for assessing physician fatigue. This provides:

- full automation of the evaluation;

- improving the accuracy of the assessment;

- increasing the speed of evaluation;

- reduced requirements for equipment, availability at the doctor's workplace;

- release of the doctor from the need to fill out questionnaires and take tests to assess fatigue;

- prevention of overwork of the doctor;

- reducing the influence of the fatigue factor on the accuracy of diagnosing medical images;

- improving the accuracy of diagnostics of medical images;

- the possibility of mathematically justified regulation of the doctor's workload in a mode close to real time;

- Prevention of unplanned delays in the processing of medical images.

These and other advantages of the present invention will become apparent upon reading the following detailed description with reference to the accompanying drawings.

Detailed description

The physician fatigue rating system comprises a display unit, a segmentation unit, a tracking unit, a measurement unit, and an evaluation unit.

The display unit may be a monitor or other suitable means for displaying a medical image for viewing by a physician. The medical image contains the area to be diagnosed. For example, the medical image is an X-ray image of the chest in frontal projection, and the area to be diagnosed is the lungs. It should be noted that this invention is directed to the diagnosis of two-dimensional images. In this case, the invention is not limited to the above example, and the image may also be a slice of an MRI or CT image, may contain a region of the heart, brain, hip joints, etc. to be diagnosed. The medical image for display and diagnostics is obtained from the system memory or from an external device via a data exchange interface.

The segmentation unit is implemented by a computing device, such as a computer, containing a processor and memory. The segmentation unit contains a neural network and is configured to segment the image, that is, to divide the area on the medical image to be diagnosed into segments using the neural network. For example, a neural network can be configured to search for lungs on a chest x-ray and segment the lungs into multiple segments according to lung lobes or according to the literature. The specific segments to be found in the image are not limited by the present invention and depend on the requirements of a specific application, type of diagnosis and type of image, and are determined by the anatomical features of the area (organ, cavity) to be diagnosed, or the prescriptions specified in the methodological literature on the diagnosis of this area. For example, on a section of an MRI image of the brain, different parts of the brain can be segmented, and so on. The segmentation data received by the segmentation unit is transmitted to the measurement unit. The specific neural network architecture for performing segmentation is also not limited to the present invention, and may include, for example, U-Net, YOLO, and so on.

The tracking unit is a device for tracking eye movements (eye tracker), which is installed in front of the doctor (for example, on a monitor). The tracking unit is configured to track the doctor's gaze while viewing the displayed medical image, and the tracking unit is configured to fix the location of the doctor's gaze on the displayed image. To do this, the tracking unit must know the exact dimensions and characteristics of the display area on the display unit (the screen on the monitor) and its exact location relative to the display unit in order to be able to determine the point on the screen at which the doctor’s gaze falls in the direction of the doctor’s gaze, and in accordance with this determine (fix) the coordinates of the pixels that the doctor is looking at. In addition, the tracking unit may additionally capture the point in time corresponding to the location of the doctor's gaze on the displayed image. Eye tracking data is transmitted to the measurement unit.

The measurement unit is implemented with a computing device such as a computer, including a processor and a memory. The measuring unit is configured to measure the coverage of each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data.

An image coverage map is generated from the eye tracking data. The gaze coordinate for each time point t ∈ T is defined as ( x , z) ∈ G(t), where x is the two-dimensional coordinate (in pixels) of the gaze in the target image I , and z is the distance (mm) between the monitor and the radiologist's eyes. The visual coverage of the image ψ ( y , t ) at time t for each pixel y is calculated as:

where ρ is the pixel density of the monitor, and θ=2° is the angle of view sufficient to detect anomalies, as defined in the medical imaging literature. The eye coverage of an image, Ψ, is calculated by accumulating ψ ( y , t ) over all image pixels:

In addition, the measurement unit may measure the time taken to cover each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data.

The measurement results are transmitted to the evaluation unit.

The estimator is implemented with a computing device, such as a computer, including a processor and memory. The evaluation unit is configured to evaluate the doctor's fatigue based on the measurement results. Experiments conducted by the inventors have shown a correlation between the onset of fatigue and the degree of coverage of the segments, as well as between the onset of fatigue and the time of coverage.

Physician fatigue score may vary by implementation. For example, the coverage measurement results for each segment can be compared to a coverage threshold for each segment. The results of each segment's coverage time measurement can be compared to a threshold of coverage time for each segment. In addition to thresholds, ranges or histograms can be specified. For each segment, the coverage multiplied by the coverage time can be calculated. More complex relationships can also be calculated. In accordance with this, it can be determined whether the doctor has studied the segment sufficiently, whether the doctor's attention concentration has fallen, etc. - that is, the degree of fatigue of the doctor is assessed. The degree of physician fatigue can be defined as a number (eg, 60%, 0.35), as a verbal range indicator (eg, high fatigue), and so on.

Thus, an accurate automatic assessment of physician fatigue is provided, which is available at the physician's workplace and does not take the physician's time.

Based on the determination that the doctor has reached a certain degree of fatigue, for example, a recommendation to reduce the load or the need to take a rest break can be issued, the number of images allocated to this doctor for the upcoming period of time can be reduced, etc. For example, if a doctor has already analyzed 60 images in the past 2 hours in a shift, and another 30 images have been allocated to him for the next hour, but the system has determined that the doctor’s fatigue has reached the first threshold, some of these 30 images are redistributed to other doctors or to the next hour, thereby reducing the load on the doctor in the next hour, preventing overwork, errors and unplanned delays in processing.

The method for assessing physician fatigue is to perform steps corresponding to the functions performed by the above blocks in the physician fatigue assessment system and will not be described separately.

Implementation example

The proposed system was implemented in practice. LG 10-bit diagnostic monitor with a resolution of 3840×2160 pixels and a pixel density of 7.21 pixels/mm, as well as a Tobii Eye Tracker 4C tracking unit and a computer based on an Intel(R) i7 processor with a Windows(R) operating system, a 4 GB graphics processor, 16 GB RAM, and a 500 GB SSD were used. The neural network was built on the U-Net architecture, pre-trained on the ImageNet dataset and further trained on the VinDr-CXR dataset for lung segmentation. The doctors viewed the images while sitting in a chair in front of the monitor, a tracking unit was installed on the monitor and tracked eye movements, recorded the coordinates and time of the gaze. Experiments have demonstrated compliance with the characteristics and benefits stated above.

Application

The systems and methods of the present invention can be used to assess physician fatigue, including in medical systems that allocate tasks to physicians.

Additional Implementation Features

Although this document may indicate that data is transmitted/sent or received/received by a person (e.g., a medical professional, doctor, expert), one skilled in the art should understand that such an indication is used solely for the purpose of simplifying the description, when in fact it is understood that data is transmitted/sent or received/received by the corresponding device used and/or controlled by this person.

One or more transmission units or devices (transmitters) described herein and one or more receiver units or devices (receivers) may be physically implemented in the same transceiver unit or device or in different units or devices.

A device or a transmission unit in this document, for the sake of simplicity of description, may be referred to as a device or unit having the functions of not only transmitting, but also receiving data, information and/or signals. Similarly, the receiving device or unit may also include data, information and/or signaling functions.

The various illustrative blocks and modules described in connection with the disclosure herein may be implemented or executed with a general purpose processor, digital signal processor (DSP), application specific integrated circuit (ASIC), field programmable gate array (FPGA) or other programmable logic device (PLD), discrete logic or transistor logic, discrete hardware components, or any combination of the foregoing designed to perform the functions described herein. A general purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices (eg, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors together with a DSP core, or any other similar configuration).

Some blocks individually or collectively may be, for example, a computer, and include a processor that is configured to call and execute computer programs from memory to perform method steps or block functions in accordance with embodiments of the present invention. According to embodiments, the device may further include a memory. The processor may call and execute computer programs from memory to execute the method. The memory may be a separate device independent of the processor or may be integrated with the processor. The memory may store code, instructions, commands, and/or data for execution on a set of one or more processors of the device described. Codes, instructions, commands may direct the processor to perform the steps of a method or device function.

The functions described herein may be implemented in hardware, software running on one or more processors, firmware, or any combination of the foregoing. The hardware and software implementing the functions may also be physically located in different locations, including such a distribution that parts of the functions are implemented in different physical locations, that is, distributed processing or distributed computing may be performed.

If necessary (for example, if the amount of data and / or calculations that need to be performed with respect to this data is large), multi-threaded data processing can be performed, which in a simple representation can be expressed in the fact that the entire set of data to be processed is divided into a set of subsets, and each processor core performs processing in relation to the data subset assigned to it.

The above memory may be volatile or non-volatile memory, or may include both volatile and non-volatile memory. One of ordinary skill in the art will also understand that when referring to memory and storage of data, programs, codes, instructions, commands, and the like, it is understood that there is a machine-readable (or computer-readable, processor-readable) storage medium. Computer-readable storage media includes both non-transitory computer storage media and communication media, including any transmission medium that facilitates movement of a computer program, or portion thereof, from one place to another. A non-transitory computer-readable storage medium can be any available medium that can be used to carry or store a desired piece of program code in the form of instructions or data structures, and that can be accessed by a computer, processor, or other general purpose or special purpose processing device.

By way of example, and not limitation, computer-readable media may include Read Only Memory (ROM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), Electronically Erasable Programmable Read Only Memory (EEPROM), Flash Memory, Random Access Memory (RAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SD). RAM), double data rate synchronous dynamic random access memory (DDR SDRAM), increased speed synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM) and direct access bus random access memory (DR RAM), register, cache memory, semiconductor memories, magnetic media such as internal hard drives and removable drives, magneto-optical media and optical media such as CD-ROMs and digital universal th disks (DVD), as well as any other data carriers known in the prior art.

The information and signals described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, and elementary signals that may be exemplified in the foregoing description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination of the foregoing as applicable to the present invention.

At least one of the steps in the method or blocks in the device may use an artificial intelligence (AI) model to perform the respective operations. The AI-related function may be executed via a processor and non-volatile and/or volatile memory.

The processor may include one or more processors. At the same time, one or more of the processors may be a general purpose processor such as a central processing unit (CPU), an application processor (AP) and the like, a graphics only processing unit such as a graphics processing unit (GPU), a visual processing unit (VPU), and/or a dedicated AI processor such as a neural processing unit (NPU).

One or more processors direct the processing of input data in accordance with a predetermined operating rule or artificial intelligence (AI) model stored in non-volatile memory and/or non-volatile memory. A predetermined operating rule or artificial intelligence model can be obtained through training. In doing so, the processor may perform a pre-processing operation on the data to convert it into a form suitable for use as input to the artificial intelligence model.

"Obtained by training" means that by applying a learning algorithm to a trainable artificial intelligence model using a set of training data, a predefined operation rule or AI model with a desired characteristic is created. The training may be performed on the device itself running the AI according to the embodiment and/or may be implemented via a separate server/system.

An artificial intelligence model may include multiple neural network layers. Each of the multiple layers of the neural network includes a plurality of weights (coefficients) and performs a work operation for a given layer by computing using the plurality of weights of a given layer with respect to the input or calculation result of a previous layer.

A learning algorithm is a method of training a predetermined target device (eg, a GPU or NPU based neural network) using a set of training data to call, enable, or control the target device to perform a determination or prediction. Examples of learning algorithms include, but are not limited to, supervised learning, unsupervised learning, partially supervised learning, or reinforcement learning.

It should be understood that although terms such as "first", "second", "third" and the like may be used herein to describe various blocks, modules, networks, elements, components, regions, layers and/or sections, these blocks, modules, networks, elements, components, regions, layers and/or sections should not be limited to these terms. These terms are only used to distinguish one block, module, network, element, component, region, layer, or section from another block, module, network, element, component, region, layer, or section. Thus, a first block, module, network, element, component, region, layer, or section may be referred to as a second block, module, network, element, component, region, layer, or section, without departing from the scope of the present invention. As used herein, the term "and/or" includes any and all combinations of one or more of the respective listed positions. Elements mentioned in the singular do not exclude the plurality of elements, unless otherwise specified.

The functionality of an element specified in the description or claims as a single element may be practiced by means of several components of the device, and conversely, the functionality of elements indicated in the description or claims as several separate elements may be practiced by means of a single component.

Although exemplary embodiments have been described in detail and shown in the accompanying drawings, it should be understood that such embodiments are illustrative only and are not intended to limit the present invention, and that this invention should not be limited to the specific arrangements and structures shown and described, since various other modifications and embodiments of the invention may be obvious to a person skilled in the art based on the information set forth in the description and knowledge of the prior art, without going beyond the essence and scope of this invention.

Claims (24)

1. A doctor's fatigue assessment system, comprising: a display unit configured to display a medical image for viewing by a physician, the medical image comprising an area to be diagnosed; a segmentation unit configured to divide an area in the medical image to be diagnosed into segments using a neural network and transmit the segmentation data to the measurement unit; a tracking unit configured to track a physician's gaze while viewing the displayed medical image and transmit gaze tracking data to the measurement unit, the tracking unit configured to fix the position of the physician's gaze on the displayed image; a measurement unit configured to measure the gaze coverage of each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data, and transmitting the measurement results to the assessment unit; And an evaluation unit configured to evaluate the doctor's fatigue based on the results of measurements of the amount of coverage of the gaze of the segments. 2. The system according to claim. 1, in which the tracking unit is additionally configured to fix the point in time corresponding to the location of the doctor's gaze on the displayed image, the measurement unit is further configured to measure the time taken to cover each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data, and the estimator is further configured to evaluate the doctor's fatigue based on the segment gaze time measurements. 3. The system according to claim 1, in which, in order to measure the coverage of the gaze of each segment, the measurement unit is configured to: determining the visual coverage of the image for each time point based on the two-dimensional pixel coordinate of the gaze in the image and the distance between the monitor and the doctor's eyes for each time point, and measuring the amount of eye coverage of the image by accumulating said determined visual coverage of the image for all pixels for each segment according to the segmentation data. 4. A method for assessing physician fatigue, comprising the steps of: displaying on the display unit a medical image for viewing by a physician, the medical image containing an area to be diagnosed; dividing the area on the medical image to be diagnosed into segments using a neural network; tracking the physician's gaze while viewing the displayed medical image, the tracking comprising fixing the location of the physician's gaze on the displayed image; measuring gaze coverage of each segment in the region to be diagnosed based on the gaze tracking data and the segmentation data; And perform an assessment of the doctor's fatigue based on the results of measurements of the amount of coverage of the gaze of the segments. 5. The method according to claim. 4, in which the tracking further comprises fixing the point in time corresponding to the location of the doctor's gaze on the displayed image, the method further comprising measuring the time taken to cover each segment in the area to be diagnosed based on the gaze tracking data and the segmentation data, moreover, the assessment of the doctor's fatigue is performed additionally based on the results of measurements of the time of covering the segments with a glance. 6. The method of claim 4, wherein measuring eye coverage of each segment comprises the steps of: determine the visual coverage of the image for each point in time based on the two-dimensional pixel coordinate of the gaze in the image and the distance between the monitor and the doctor's eyes for each point in time, and measuring the amount of view coverage of the image by accumulating said determined visual coverage of the image for all pixels for each segment according to the segmentation data.