JP2021020054A - Medical image processing apparatus and program - Google Patents

Abstract

To easily grasp abnormality of a brain.SOLUTION: A medical image processing apparatus comprises a processing circuit configured to: obtain image data representative of a brain of a subject; obtain data representing a clinical sign or symptom of the subject, the clinical sign or symptom being relevant to a brain condition; process the image data to obtain an estimation of an abnormality in the brain of the subject; and determine whether the estimation of the abnormality is consistent with the data representing the clinical sign or symptom.SELECTED DRAWING: Figure 1

Description

The embodiments disclosed in this specification and drawings relate to medical image processing devices, methods and programs.

Clinicians are known to look back at a scan of a patient's brain to look for signs of stroke, such as ischemic stroke. Such scans may be non-contrast computed tomography (NCCT) scans. Interpreting NCCT scans to look for signs of ischemic stroke can be particularly difficult due to the subtle nature of imaging findings. For example, searching for ischemic areas can involve spotting very subtle differences in image contrast.

Clinician interpretations of findings in imaging may be weighted by the clinical picture of the patient they have, depending on the circumstances. For example, if the clinician has already seen the patient, the clinician may know the side (right or left) of the patient who has the symptoms. However, such clinical information may not always be readily available, or at least may not be immediately available to the clinician when looking back at imaging.

Patients who have had a stroke often have a history of stroke or small vascular disease. Imaging features derived from past strokes and / or microvascular disease can confuse the task of identifying imaging features specific to the current acute episode. It can be difficult to distinguish between signs of a past stroke and signs of a recent stroke.

One of the problems to be solved by the embodiments disclosed in the present specification and the drawings is to easily grasp the abnormalities of the brain. However, the problems to be solved by the embodiments disclosed in the present specification and the drawings are not limited to the above problems. The problem corresponding to each effect by each configuration shown in the embodiment described later can be positioned as another problem.

The medical image processing apparatus according to the embodiment includes a processing circuit. The processing circuit acquires image data representing the subject's brain, acquires data representing the subject's clinical signs or symptoms, and the clinical signs or symptoms are related to the state of the brain and of the subject. It is configured to process image data to obtain an estimate of the abnormality in the brain and to determine if the estimate of the abnormality is consistent with data representing clinical signs or symptoms.

FIG. 1 is a schematic view of an apparatus according to an embodiment. FIG. 2 is a flowchart illustrating an overview of the method of the embodiment. FIG. 3 is a diagram illustrating a display screen according to the embodiment. FIG. 4 is a flowchart showing a first specific example in the embodiment. FIG. 5 is a flowchart showing a second specific example in the embodiment. FIG. 6 is a flowchart showing a third specific example in the embodiment.

Hereinafter, embodiments of the medical image processing apparatus will be described as non-limiting examples with reference to the drawings.

Certain embodiments are the acquisition of image data representing the subject's brain and the acquisition of data representing the subject's clinical signs or symptoms, wherein the clinical signs or symptoms are brain conditions. To obtain, to process the image data to obtain an estimate of the abnormality in the brain of the subject, and to obtain the data in which the estimate of the abnormality represents the clinical sign or symptom. Provided is a medical image processing apparatus including a processing circuit configured to determine and perform a match.

Certain embodiments are the acquisition of image data representing the subject's brain and the acquisition of data representing the subject's clinical signs or symptoms, wherein the clinical signs or symptoms are brain conditions. To obtain, to process the image data to obtain an estimate of the abnormality in the brain of the subject, and to obtain the data in which the estimate of the abnormality represents the clinical sign or symptom. Provided is a medical image processing method comprising determining whether or not they match.

The medical image processing apparatus 10 according to the embodiment is schematically illustrated in FIG. The medical image processing device 10 is configured to process and display a medical image of a patient or other subject.

The medical image processing device 10 comprises a computing device 12, which in this case is a personal computer (PC) or workstation. The computing device 12 is connected to, for example, a display device 16 such as a screen and one or more input devices 18 such as a computer keyboard and mouse. In some embodiments, the display device 16 is a touch screen, which also serves as an input device 18. The computing device 12 is connected to the data store 20.

The medical image processing apparatus 10 is connected to a CT scanner 14 configured to perform a non-contrast CT (NCCT) scan of a patient or other subject to obtain volumetric medical imaging data. In this embodiment, the scan is a brain scan. In other embodiments, any suitable body part can be scanned.

In alternative embodiments, the data can be obtained using any suitable modality and / or collection technique. The CT scanner 14 is, for example, a CT scanner, a cone beam CT scanner, an MRI (magnetic resonance imaging) scanner, an X-ray scanner, an ultrasonic scanner, a PET (positron emission tomography) scanner, or SPECT (single photon emission computed tomography). It may be replaced or supplemented by one or more scanners configured to acquire two-dimensional or three-dimensional imaging data at any suitable imaging modality, such as a scanner.

The data acquired by using the CT scanner 14 is stored in the data store 20 and supplied to the calculation device 12. In other embodiments, the calculator 12 may acquire data directly from the CT scanner 14. In an alternative embodiment, the medical image processor 10 receives medical imaging data and / or medical images from one or more additional data stores (not shown) on behalf of or in addition to the data store 20. .. For example, the medical image processing device 10 is one of other information systems such as a picture archiving and communication system (PACS) or, for example, a laboratory data archive, an electronic medical recording (EMR) system, or an entrance / exit transfer (ADT) system. Medical imaging data can be received from one or more remote data stores that can form a portion.

The computing device 12 provides processing resources for automatically or semi-automatically processing medical imaging data. The arithmetic unit 12 includes a central processing unit (CPU) 22.

The computing device 12 provides processing resources for automatically or semi-automatically processing a data set. In this embodiment, the dataset comprises medical imaging data. For clarity, the following refers to the processing and retrieval of medical images. However, the operations described below as being performed on a medical image may actually be performed on any suitable set of imaging data representing the medical image. Depending on the situation, the captured data may be processed internally by the calculator 12 without displaying any corresponding image.

In order to detect a discrepancy between the line-of-sight detection circuit 24 configured to determine the direction of the line-of-sight and the rendering circuit 26 configured to render an image from the imaged data, the computing device 12, for example, capture data. A comparison circuit 28 configured to compare data to detect discrepancies between the line of sight determined from and the line of sight determined in a different way, and, for example, ischemia and / / to identify anomalies. Alternatively, it includes an anomaly detection circuit 29 configured to process imaging data to detect a region of a hyperdense blood vessel.

In this embodiment, the circuits 24, 26, 28, 29 are each implemented in the computer 12 using a computer program having computer-readable instructions capable of executing the methods of the embodiment. However, in other embodiments, various circuits may be implemented as one or more ASICs (application specific integrated circuits) or FPGAs (field programmable gate arrays). In this embodiment, the circuits 24, 26, 28, and 29 are implemented as part of the CPU 22, respectively. In an alternative embodiment, the circuits 24, 26, 28, 29 may be implemented separately or form part of two or more CPUs. In a further embodiment, at least some of the methods may be performed on one or more graphic processing units (GPUs).

The computing device 12 includes a hard drive and other components of a PC including a RAM, a ROM, a data bus, an operating system including various device drivers, and a hardware device including a graphics card. Such components are not shown in FIG. 1 for clarity.

The system of FIG. 1 is configured to perform a series of stages as illustrated in the flowchart of FIG.

At stage 30, a CT scanner collects non-contrast CT scans of the brain of a patient suspected of having a stroke. The volumetric data from the NCCT scan is passed to the data store 20 and from such data store 20 to the line-of-sight detection circuit 24. In other embodiments, the line-of-sight detection circuit 24 can obtain a set of volumetric NCCT data from any suitable data store. Such NCCT data can be acquired by the line-of-sight detection circuit 24 at any suitable time after the NCCT scan is acquired.

At stage 32, the line-of-sight detection circuit 24 processes the NCCT scan data in order to obtain an estimate of the direction of the line of sight of the patient's eyes. References to the eye below refer to the eyeballs, which are sometimes referred to as eyeballs. Both eyes are expected to point in the same direction. In this embodiment, the line-of-sight detection circuit 24 outputs a classification into one of the three classes. In the first class, the direction of the line of sight is to the right. In the second class, the direction of the line of sight is to the left. In the third class, the direction of the line of sight is either left or right, or the direction of the line of sight is unknown. Such classifications are referred to below as "right", "left", and "neither / unknown".

The angle of gaze can be defined with respect to the opposite surface of the patient's skull. The angle of sight can be defined, for example, as described below. Kobayashi, M.M. , Horizontal deviation deviation on computed tomography: the visual criterion and lession charactic strokes in ischemic stroke. Acta Neurol Berg (2018) 118: 581. https://doi.org/10.1007/s13760-018-0949-1, or Spokoynyy, Ilana et al. , Visual Determination of Conjugate Eye Development on Computed Tomography Scan Predicts Diagnosis of Stroke Code Patients, Journal Type Patients, Journal

In some embodiments, the line-of-sight detection circuit 24 may return, for example, a line-of-sight angle as a number. The angle of gaze may be returned in addition to or instead of the determination that the direction of gaze is left, right, or neither / unknown.

In this embodiment, a trained model is used to determine whether the eyes are biased to the right, biased to the left, or neither / unknown. Such a trained model may be a deep learning classifier. For example, the R-CNN (Region with Convolutional Neural Network Features) method may be used, which may be similar to that described below. R. Girshick, J. Mol. Donahoe, T.M. Darrel, J. et al. Malik, “Rich IEEE hierarchy for accurate object detection and semantic segmentation”, The IEEE Conference on Computer Vision and Pattern (14)

In other embodiments, any suitable method can be used to determine whether the direction of the line of sight is left, right, or neither / unknown. In a further embodiment, any suitable classification of line-of-sight can be used.

In the present embodiment, the purpose of training the trained model is to make the trained model 100% unique or as close to 100% unique as possible. If the right is predicted, the eyes are always looking to the right. If left is predicted, then the eyes are facing left. Any uncertainty in determining the line of sight will lead to a classification of neither / unknown.

In some circumstances, a scan of the brain may have been obtained by excluding the patient's eyes from the anatomical area scanned using a CT scanner. In at least some such cases, it may be possible to obtain the direction of gaze based on other anatomy. In some embodiments, the trained model can output a classification based on a biological structure other than the eyeball. For example, the trained model can be trained to take into account the extraocular muscles. Extraocular muscle compression may indicate gaze direction. In other embodiments, determining the direction of the line of sight may be based on segmenting the extraocular muscles in an NCCT scan.

The determined gaze direction indicates with respect to the patient's skull whether the eyes are directed to the patient's right or to the patient's left. The line-of-sight detection circuit 24 outputs the direction of the line of sight as one of the three categories of "right", "left", and "neither / unknown".

In some embodiments, the line-of-sight detection circuit 24 can also output the line-of-sight angle. In a further embodiment, individual gaze directions and / or gaze angles can be determined individually for each of the patient's eyes. In some embodiments, the line-of-sight angle is compared to the threshold angle and the line-of-sight direction is reported to be right or left only if the line-of-sight angle exceeds the threshold angle.

In the present embodiment, the line-of-sight detection circuit 24 automatically acquires the line-of-sight direction by processing the volumetric NCCT data. In other embodiments, the direction of gaze can be provided by the clinician based on the clinical observation of the patient. In a further embodiment, the clinician can manually analyze the volumetric NCCT data to determine the direction of the line of sight. The clinician can use the input device 18 to input the direction of the line of sight into the calculator 12.

In clinical practice, clinical deviation of the eye is well documented as a symptom of stroke. The clinical deviation of the eye is called the Prevost's sign. Eye deviation is defined as an even and lasting deviation of both eyes from the midline position to the same side. If symptomatic, the eye is displaced to the hemispherical side of the stroke-damaged brain. The damaged hemisphere is the opposite of the side of the body where symptoms such as paralysis or facial dropping are present.

Déviation conjugation on CT scans is a highly specific imaging feature for the clinical manifestations of the line of sight. Depending on the situation, the line of sight as determined by the CT scan can be used as a substitute for the clinical symptoms obtained by examining the patient.

Eye deviations due to stroke are generally the result of damage to the frontal eye fields, the inferior frontal gyrus of the midline, or the caudal region of its corticopontine projections.

Ocular deviation has been found to occur in about 20% to 30% of patients who have had a cerebral hemisphere stroke. Ocular deviation has been shown to be hyposensitive but highly specific and have a positive predictive value of 93% to ensure lateralization into the ischemic hemisphere.

In the absence of clinical information, clinicians have been found to be sensitive to the interpretation of signal attenuation associated with early ischemia by confirming eye deviation.

After stage 32, the flowchart can be executed simultaneously or consecutively in any order, any one of four possible stages 34, 38, 46, 52 or Proceed to all.

At stage 34, the rendering circuit 26 receives suggestions for the eye region in the volumetric NCCT data from the line-of-sight detection circuit 24. For example, the eye area can be indicated by a bounding box obtained from segmentation. The rendering circuit 26 renders an image showing the eye region from the volumetric NCCT data. The rendering circuit 26 displays a rendered image of the eye region on the display device 16. The rendered image can provide a quick view of the line of sight. The rendered image can be displayed on any suitable user interface. For example, the rendered image can be displayed on a clinical dashboard that the user can use to look back at the image and / or other medical data.

For example, a user, such as a clinician, looks at the displayed image of the eye area. By looking at the image, the user can manually verify that the automated detection of the line of sight is correct.

In this embodiment, the rendered image represents a portion of the body axis slice. Axial slices are slices that pass through the crystalline lens of the eye and through the head. The user can view the entire body axis slice by clicking on the displayed portion. In other embodiments, any suitable image of the eye area can be displayed.

At stage 38, the rendering circuit 26 displays a simplified depiction of the line of sight 40 on the display device 16. A simplified depiction of such a line of sight may be described as a "cartoon." The simplified depiction 40 can be displayed on the same user interface as the rendered image. The simplified depiction 40 is intended to provide a basic indication of whether the eye deviation is to the left or to the right.

In the present embodiment, the depiction 40 includes a pair of first line arts 42 facing left and a pair of second line arts 44 facing right. The rendering circuit 26 either receives the first line art 42 (as shown in FIG. 2) or the second line art 44 according to the determination of the line-of-sight direction to be left or right, as received from the line-of-sight detection circuit 24. To highlight.

The depiction 40 can provide a simple way for the user to view a determined gaze direction at a glance.

In some embodiments, the depiction 40 can be displayed without displaying the rendered image. For example, the depiction 40 can be displayed as part of a clinical overview.

At stage 46, the comparison circuit 28 compares the line-of-sight direction determined in stage 32 with at least one other result. The other one or more outcomes comprise additional data that may be associated with the laterality of the stroke. The other one or more results may include, for example, at least one symptom manually entered by a user, such as a clinician. For example, the symptom may be unilateral facial drooping or unilateral paralysis. Symptoms may be one of the left or right indicators listed in the National Institutes of Health Stroke Scale / Score (NIHSS).

The comparison circuit 28 outputs a determination as to whether the line-of-sight direction determined in stage 32 is consistent with at least one other result. If the gaze direction does not match at least one other result, the flowchart proceeds to stage 48. At stage 48, the rendering circuit 26 displays to the user reference information that warns the user of the mismatch. In this embodiment, the reference information includes a flag 50 on the display device 16, such as on the depiction 40. In the embodiment of FIG. 2, the line of sight is shown as a discovery and includes flag 50 if there is a mismatch. In other embodiments, any suitable display of reference information may be used. Such reference information can provide the user with a warning that a mismatch has been detected.

At the stage 52, the line-of-sight detection circuit 24 passes the line-of-sight direction to the abnormality detection circuit 29. The anomaly detection circuit 29 uses the line-of-sight direction as an input for the anomaly detection procedure. In a further embodiment, any suitable information derived from imaging and / or any information regarding any suitable clinical condition can be used as input to the anomaly detection procedure.

Anomaly detection procedures are sometimes described as "automated stroke detection." The anomaly detection procedure comprises processing the volumetric NCCT data received at stage 30 to identify at least one type of anomaly in the NCCT scan.

In other embodiments, the gaze direction and / or gaze angle can be determined as part of the anomaly detection procedure.

In the embodiment of FIG. 2, the stage 52 abnormality detection procedure can be performed on imaging data representing the entire brain. The anomaly detection circuit 29 is configured to detect an ischemic region and a highly resorbed blood vessel region in the brain. The anomaly detection circuit 29 outputs one or more regions identified as ischemic regions or regions of highly resorbed blood vessels. In other embodiments, the anomaly detection circuit 29 can be configured to detect any suitable brain abnormality.

At stage 54, the rendering circuit 26 renders at least one image using the results of the anomaly detection procedure and displays at least one image (not shown in FIG. 2) on the display device 16. In this embodiment, at least one such image is displayed as part of the same user interface as the eye image and description 40. At least one identified area of ischemic and / or hyperabsorbent vessels is highlighted in at least one image.

In other embodiments, the rendering circuit 26 can display the results of the anomaly detection procedure using any suitable display method. For example, the rendering circuit 26 can display text indicating whether or not any abnormality has been detected. The rendering circuit 26 can label the area of the displayed image.

In this embodiment, the anomaly detection circuit 29 outputs a decision on the side of the brain in which all or most of the anomaly regions are detected. The rendering circuit 26 can display suggestions (eg, text or visual display) indicating the side of the brain where all or most of the abnormal areas are found.

The comparison circuit 28 compares the side of the brain output by the abnormality detection circuit 29 in the stage 52 with the direction of the line of sight determined in the stage 32. When the side of the brain output by the anomaly detection circuit 29 does not match the line of sight determined in stage 32, the rendering circuit 26 displays a flag 50 or a further flag to indicate that there is a discrepancy.

In other embodiments, the anomaly detection procedure can rule out results from the side of the brain that do not correspond to the determined gaze direction.

At stage 56, the user of the user interface (eg, a clinician) specifies the side of the brain that performs the anomaly detection procedure.

The user can specify the side of the brain based on the gaze direction determined in stage 32. For example, if the line of sight is pointing to the left, this indicates that the left side of the brain is damaged. This is the opposite side of the symptom. For example, damage to the left side of the brain can result in drooping or paralysis on the right side of the body.

The user can specify that anomaly detection be performed only on the left side of the brain when the line of sight is pointing to the left. When deciding on a side, the user can take into account whether the line of sight is found to be inconsistent with other information (eg, inconsistency as indicated to the user by flag 50 or by further flags).

At stage 58, the anomaly detection circuit 29 re-executes the anomaly detection procedure to detect a stroke sign on the side of the brain designated by the user.

The anomaly detection procedure outputs suggestions for any region of ischemia or highly resorbed blood vessels found in the anomaly detection procedure.

The comparison circuit 28 can compare the designated side of the brain with the side of the brain indicated by gaze or by any other data.

At stage 60, the rendering circuit 26 updates at least one displayed image according to the re-executed anomaly detection procedure of stage 58. Any anomalous areas found in stage 58 are highlighted in at least one displayed image. The rendering circuit 26 displays suggestions on the side of the brain where the anomaly detection procedure has been performed. When the rendering circuit 26 adds or removes flag 50 or additional flags to indicate any discrepancy between the designated side of the brain and the side of the brain indicated by line of sight or by any other data. There is also.

In this embodiment, the stage 52 abnormality detection procedure is performed on the entire brain. At stage 56, the user specifies the side of the brain that performs the anomaly detection procedure. At stage 58, an anomaly detection procedure is performed on the side of the brain selected by the user.

In other embodiments, the stage 52 abnormality detection procedure is performed on the first side of the brain. In some embodiments, the first side of the brain is selected by the user. In some embodiments, the first side of the brain is automated by the anomaly detection circuit 29 based on the gaze direction determined in stage 32 or based on other data, such as, for example, other clinical symptoms. Is selected. At stage 56, the user is given the option to switch to the other side of the brain. If the user chooses to switch to the other side of the brain, at stage 58, the anomaly detection circuit 29 re-executes the anomaly detection procedure for the other side of the brain.

Two examples of anomaly detection procedures are illustrated in FIG. 2 (stage 52 and stage 58). In other embodiments, examples of anomaly detection procedures can be performed any suitable number of times. For example, a clinician can alternate between performing anomaly detection procedures on one side of the brain and performing anomaly detection procedures on the other side of the brain. In some embodiments, the user is not provided with an algorithm that has an automatically detected gaze direction and / or other clinical information for the image. Is given the option to run the algorithm.

Based on clinical practice, the gaze direction and / or other relevant clinical information may be entered into an algorithm configured to automatically detect stroke signs, such as the algorithm used in anomaly detection procedures. You can think of it as reasonable. However, there are some situations where errors can occur in automated detection. Depending on the situation, an error may occur in the algorithm for detecting eye deviation. For example, an error may occur in detection. In some circumstances, for example, errors due to human or machine errors that cause false reports may occur in clinical information. The clinical information may not match the automatic algorithm or may provide incorrect information otherwise. Therefore, if gaze detection and / or other clinical information is always used as an input to an algorithm for detecting stroke signs, in some cases automatic algorithms will result based on incorrect information or assumptions. May be calculated, which can lead to significant errors in the results.

By flagging for mismatches as in the method of FIG. 2, the user can be provided with a warning of possible errors. The user may choose to re-execute the algorithm on the selected side or on the opposite side of what has already been executed.

The method of FIG. 2 can provide clinical application for stroke sign detection from a head scan, which features automatic gaze detection. The clinical application may be for the detection of stroke ischemia from NCCT.

Gaze detection can be input to a stroke sign algorithm for improved detection. The line-of-sight detection can be displayed to the user. Automatic axis alignment of the imaging surface with the crystalline lens of the eye can be performed to provide quick user confirmation. For example, scan-aligned slices can be displayed so that the user can confirm eye deviation detection. Flags may be set if gaze detection does not match stroke sign detection or does not match the clinical side of the patient's symptoms. For example, a flag may be displayed if gaze detection does not match automatic ischemia detection. A flag may be displayed if gaze detection does not match automatic high-absorption vessel detection. A flag may be displayed if gaze detection does not match one or more clinical symptoms (if any).

The user may be allowed to return an automatic algorithm that forces the other side or a particular side. Automatic algorithms may ignore some or all clinical inputs.

The method of FIG. 2 can provide assistance to the user in interpreting the result of anomaly detection. The method of FIG. 2 can provide information to the user in an intuitive and easy-to-interpret way. The user may be provided with an appropriate level of information to inform the user of clinical practice. Enough for the user to decide whether to accept the results of the automated anomaly detection, perform the automated detection again, and / or manually look back at the image to verify the automated detection. Information may be given.

Stroke may be an example of a serious, life-threatening medical condition and may require urgent medical attention. Ischemic stroke is an example of a condition in which a clinician has a limited time window in which treatment decisions should be made. The sooner patients receive the correct treatment, the less damage they can suffer. In this limited time window, quick decisions may need to be made if brain tissue should be protected. The method described above with reference to FIG. 2 can assist the clinician in performing a rapid and accurate diagnosis.

As described above, the anomaly detection procedure comprises processing volumetric NCCT data to detect one or more abnormalities, for example, one or more signs of stroke.

In some embodiments, the anomaly detection procedure comprises applying a trained model to volumetric NCCT data to identify one or more anomaly regions. The trained model may include a deep learning algorithm. The trained model may include, for example, a neural network such as a convolutional neural network (CNN). The trained model may include U-Net. In other embodiments, the trained model may include any suitable form of machine learning, such as a support vector machine (SVM) or regression forest. The trained model may utilize features such as intensity features, gradient features and / or Gabor filters, for example.

The trained model can be stored in the datastore 20 or in any suitable memory that may be local or remote. In other embodiments, the trained model can be implemented in hardware, for example by using at least one ASIC or FPGA.

The trained model is trained to detect abnormal regions in brain imaging data, for example, to segment regions of ischemic and / or highly resorbed vessels. The trained model is trained in multiple training sets of volumetric NCCT data. In some embodiments, the training set of volumetric NCCT data is labeled with ground truth.

In different embodiments, different methods of inputting the line of sight into the trained model may be used.

In some embodiments, deep learning algorithms with shared weights are trained to perform multiple tasks. Deep learning algorithms are learned using multitask learning. Multiple tasks can include both anomalous area detection and gaze direction detection. Gaze detection can be used to interpret other results.

In some embodiments, such as the embodiment of FIG. 2, the line of sight is detected as a pretreatment step. The determined gaze direction is then used as an input for stroke sign detection.

In other embodiments, gaze detection is used during post-processing of the algorithm results. For example, the output image may be masked so that only one side of the brain is displayed.

In some embodiments, the gaze direction is used to modify the activation function of the trained model, for example, the sigmoid or ReLU (normalized linear unit) activation function. The gaze direction may be used to correct one or more thresholds for each side of the brain. Applying the trained model to one side of the brain (eg, left side) may differ from applying the trained model to the other side of the brain (eg, right side).

In some embodiments, the trained model comprises a self-learning model. The self-learning model continues to learn while in use. The self-learning model can be automatically refined over time. The self-learning model can update its internal algorithm based on its output.

In the embodiment of FIG. 2, the line of sight is detected and used as an input to stroke detection. The line of sight can be thought of as a non-brain feature (non-brain feature) that is potentially associated with stroke. The line of sight can be considered a clinical sign or symptom.

In other embodiments, other non-brain signs (non-brain signs) or symptoms can be considered. Non-cerebral signs or symptoms may include other information that may indicate that a stroke has occurred. Non-cerebral signs or symptoms may carry information that may indicate the side of the brain where the stroke occurred. Non-brain signs or symptoms on one side of the body (eg, the left side of the body) indicate damage to the other side of the brain (in this example, the right hemisphere).

In some embodiments, at least one non-brain sign or symptom is determined from a volumetric NCCT scan. In some embodiments, at least one non-brain sign or symptom is determined from a further scan of the same patient. Such further scans can be obtained using any suitable modality and protocol. For example, a further scan may be a CT scan in which contrast is used.

When performing a CT scan of a patient's body area, it is common to perform a three-dimensional (3D) scanogram first. The 3D scanogram may have a larger imaging area (field of view) than the local scan of the area to be performed. A 3D scanogram may include a low resolution scan of a larger area of the patient's body, for example, the entire patient's body. In some embodiments, items of non-brain features or clinical information are determined from a 3D scanogram.

In some circumstances, an optical image of the patient can be taken during the acquisition of another imaging modality, for example during a CT acquisition. For example, images of the patient can be taken during the collection and stored with the imaging data from such collections. In some embodiments, non-brain features or items of clinical information are determined from optical imaging during acquisition. In some embodiments, non-brain signs or symptoms are determined from other optical imaging.

Non-brain features may be the relative angle of the patient's head (flexed neck) on the patient's trunk. The angle of the head on the trunk may include an angle at which the patient's head deviates to the right or left. The angle of the head on the trunk may be measured relative to a forward position in the center of the head. The angle of the head on the trunk can indicate the laterality of the stroke in a manner similar to the line of sight. The angle at which the head is located may indicate the side of the body where the stroke occurred.

In some embodiments, the angle of the head on the trunk is estimated from a 3D scanogram. In some embodiments, the angle of the head on the trunk is estimated from a CT angiography (CTA) scan. A CTA scan may include the carotid artery and therefore may include enough neck to estimate the angle of the head on the trunk. In some embodiments, the angle of the head on the trunk is estimated from optical imaging. Estimating the angle of the head on the trunk may use the assumption that the patient is aligned head-on with the table.

Non-cerebral signs or symptoms may include facial asymmetry or other asymmetry. In some embodiments, facial asymmetry may suggest that a stroke has occurred. If it is possible to determine which side of the face was affected by the stroke (eg, which side was affected by ptosis and / or paralysis), facial asymmetry provides an indication of the laterality of the stroke. There is. Non-brain signs or symptoms may include tongue deviation.

In some embodiments, a non-brain sign or symptom comprises compression of the laryngeal muscles. Compression of the laryngeal muscles may be a sign of dysphagia. Dysphagia is a clinical symptom of stroke that may be seen on imaging.

In some embodiments, the surface of the patient is determined using volumetric NCCT data and / or other imaging data. For example, slice to volume regression (SVR) can be used to determine the surface. The resulting surface and / or imaging data can be automatically analyzed to determine facial asymmetry or other asymmetry. In some embodiments, images from optical cameras are used to determine facial asymmetry or other asymmetry.

In some embodiments, eye muscle tension is used in place of or in addition to eye-based gaze determination. If the scan does not include the crystalline lens of the eye, eye muscle tension can be used. Determining eye muscle tone may include detection and / or segmentation of the extraocular muscles and identification of compression of the extraocular muscles indicating the gaze direction. Such compressions may cause the muscle density to appear higher.

In a further embodiment, the non-cerebral sign or symptom may be a clinical sign or symptom indicating any form of brain condition, eg, any suitable brain disease and / or brain injury. The area of abnormality detected in the anomaly detection procedure may be of any type of abnormality associated with such brain condition. Such signs or symptoms can include any type of behavior of any suitable body part, such as, for example, the position and / or orientation and / or movement of the body part.

In the embodiments described above, the gaze direction is determined for a single NCCT scan. In other embodiments, gaze directions can be compared over multiple collections. In some embodiments, the gaze direction determined for the NCCT scan is compared to the gaze direction determined for the subsequent CTA scan. Flags or other warnings are generated if the gaze direction determined for the CTA scan is not the same as the gaze direction determined for the NCCT scan. In other embodiments, gaze directions can be compared over any number of scans of any suitable modality and / or protocol to check for consistent deviations. In a further embodiment, any non-brain feature or clinical information can be compared across multiple scans.

The above embodiments have described the detection of abnormal areas of the brain for ischemic stroke. In other embodiments, the methods described above can be used for hemorrhagic stroke. In a further embodiment, the methods described above can be used for any suitable abnormal region indicating any suitable brain condition, such as intracerebral hemorrhage.

The patient may be any suitable human or animal subject. References to medicine can include veterinary medicine.

In a further embodiment, the signs or symptoms detected may be clinical signs or symptoms indicating any disease or condition that may not be a brain condition. Any suitable type of anomaly can be detected. Anomaly detection can utilize the detected signs or symptoms. Alternatively, the processing circuit can determine if the sign or symptom is consistent with the abnormality.

In some embodiments, swelling can be detected and used, for example, to indicate signs of trauma. Such swelling may be on the outside of the skull, or on the outside of another part of the body. In the head, swelling detection may be used to seek out counter-injury.

In some embodiments, detection of soft tissue edema can be used to indicate trauma.

In some embodiments, detection of signs and symptoms may be useful when some of the clinical features may be missing. For example, consider the case of an elderly woman with lung cancer. The radiologist is interpreting the scan without complete information. The radiologist is unaware of any information that the patient has fallen. In some situations, a rib fracture may look like a metastasis of cancer. Without information that the patient has had a fall, the chances of a rib fracture being correctly identified may be reduced. If soft tissue edema is identified, such soft tissue edema may be an indication of trauma.

Lower extremity edema may indicate a particular condition. For example, unilateral edema may facilitate the search for deep vein thrombosis. Bilateral edema may be a concomitant symptom of heart failure.

Non-brain signs or symptoms may include body temperature. Camera-based imaging can be used to capture non-brain signs or symptoms. For example, jaundice can be detected based on optical imaging.

Many of the above embodiments have been described for a single set of medical imaging data, which can be obtained in a single dose. A single determination of gaze direction or another clinical sign or symptom can be obtained.

In a further embodiment, for example, by taking multiple scans of the subject, longitudinal medical imaging data collected over time is obtained. Such longitudinal medical imaging data may represent the subject more than once. Gaze direction, or another clinical sign or symptom, may be obtained at each such time.

As mentioned above, in some embodiments, the gaze directions can be compared over multiple collections. The gaze direction determined for the NCCT scan can be compared to the gaze direction determined for subsequent CTA scans.

Persistent eye deviation can be eye deviation that shifts in a consistent direction over time. Persistent eye deviation can be eye deviation that shifts in a consistent direction across multiple scans. For example, if the gaze direction is left over two or more scans, or if the gaze direction is right over two or more scans, it can be determined that persistent eye deviation has occurred.

In some embodiments, the line-of-sight detection circuit 24 determines if there is a persistent eye deviation. The line-of-sight detection circuit 24 determines that there is persistent eye deviation if the line-of-sight direction is left for two consecutive scans or if the line-of-sight direction is right for two consecutive scans. Two consecutive scans may include NCCT scans and CTA scans.

In other embodiments, the line-of-sight detection circuit 24 is consistently left or right over any suitable number of scans, such as three, four, or more scans. , Judge that there is a persistent eye deviation. The line-of-sight detection circuit 24 may determine that there is a persistent eye deviation if the direction of the line of sight is consistently left or right over a predetermined time period.

The comparison circuit 28 can compare the side of the brain output by the abnormality detection circuit 29 with the direction of the continuous eyeball deviation. If there is a discrepancy, a flag may be displayed.

The rendering circuit 26 may display a simplified depiction of the direction of sustained eye deviation. In some embodiments, the simplified depiction comprises a separate depiction of the gaze direction in multiple scans. For example, two manga versions of a pair of eyes may be displayed, where the first pair of eyes represents the gaze direction in the NCCT scan and the second pair of eyes represents the gaze direction in the CTA scan.

Any suitable visual suggestion of persistent eye deviation may be presented to the user. For example, text suggestions and / or additional graphical elements can be used to indicate to the user persistent eye deviation.

Certain embodiments include collecting image data including the brain of the subject and clinical information about the brain disease of the subject, and estimating an abnormal portion due to the brain disease based on the image data. Provided is a medical image processing apparatus including a processing circuit configured to check the agreement between the clinical information and the estimation result by referring to the clinical information and the estimation result. .. The clinical information may be information regarding the behavior of the body part of the subject. The behavior of the body part can be specified based on the image data. The processing circuit can be further configured to display reference information when the clinical information and the estimation result do not match. The processing circuit can be further configured to limit the region of the treatment to each half region of the brain of the subject when the clinical information and the estimation result do not match.

Certain embodiments provide a means of displaying images (CT) collected for a stroke (such as an MPR view) and automatically detect multiple non-brain features (possibly clinical information). A means of displaying, a means of comparing the non-brain feature with a plurality of automatically detected signs of stroke, and a means of indicating when the non-brain feature and the automatically detected sign of stroke do not match. And, a medical imaging apparatus comprising.

Non-brain features may indicate laterality of the stroke. The device may further include means to re-execute an algorithm that forces detection on the other side or specific side of the brain. Signs of stroke may include either ischemic or highly resorbed vessels. Non-brain features may be déviation.

Clinically significant information can be presented via simplified image renditions (eye cartoons). The image data in the bounding box of the detected eye on the appropriate surface can be displayed so that the eye deviation can be directly interpreted. Eye localization and alignment can be performed automatically.

Non-brain features may be asymmetric muscle tone. Non-brain features may be head deviations on the trunk. Head deviations on the trunk may have been measured in at least one of CTA, 3D scanogram, and optical. Non-brain features may be facial asymmetry. Facial asymmetry may have been measured optically. Mismatches may be identified between multiple scans of the same patient.

Automatically detected non-brain features may be entered into the automatic detection of stroke signs. Non-brain features may be used as direct inputs to the detection of stroke signs (eg, as inputs to deep learning algorithms). Non-brain features may be input to multiple network layers. Non-brain features can modify the activation function (eg, sigmoid, ReLU, etc.) in some way. Non-brain features can be used as post-processing to assist user interpretation. Non-brain features and stroke sign detection can be combined. Both types of detection can be performed by a single deep learning algorithm, for example multitask learning. Non-brain features may now be input to some form of learning that is input to the CNN.

FIG. 3 is a diagram illustrating a display screen according to the embodiment. The display screen 62 of FIG. 3 is displayed on the display device 16. The display screen 62 includes, for example, a depiction 40, a CT image 64, an overlay image 66, and a line-of-sight image 70. The display screen 62 allows the user to get a bird's-eye view of the processing results executed in each stage of FIG.

The CT image 64 is, for example, a cross section of the brain rendered by the rendering circuit 26. Specifically, the CT image 64 shows, for example, a cross section of the brain in which an abnormal region is detected.

In the overlay image 66, the detected abnormal region 68 is overlaid with respect to the CT image 64. Specifically, the second CT image 66 shows, for example, an abnormal region 68 on the right brain side.

The line-of-sight image 70 is, for example, a part of the imaging surface including the crystalline lens of the eye rendered by the rendering circuit 26. Specifically, the line-of-sight image 70 shows an eye region. The user can directly confirm the deviation of the eyes by the line-of-sight image 70.

In the description 40, for example, since the first line art 42 is highlighted, it is shown that the deviation of the eyes is directed to the left. Further, in the drawing 40, for example, since the flag 50 is highlighted, it is shown that the line-of-sight direction determined in the stage 32 does not match the other results.

Other results include, for example, the location of the abnormal region of the brain detected by the CT scan, the gaze direction entered by the user, and the angle of the head detected by the CTA scan. All of these are related to the line-of-sight direction determined in stage 32.

The flowchart of FIG. 2 described above includes, for example, the following flowcharts of FIGS. 4 to 6. In the flowcharts of FIGS. 4 to 6, the line-of-sight direction and the object to be compared are different from each other.

FIG. 4 is a flowchart showing a first specific example in the embodiment. In the first specific example, the line-of-sight direction identified from the CT image is compared with the abnormal region of the brain detected from the CT image.

When the flowchart of FIG. 4 starts, the line-of-sight detection circuit 24 acquires the CT image collected by the CT scanner 14 (step ST110). After acquiring the CT image, the line-of-sight detection circuit 24 specifies the line-of-sight direction based on the acquired CT image (step ST120). Further, the abnormality detection circuit 29 detects an abnormal region of the brain based on the acquired CT image (step ST130).

After the line-of-sight direction is specified and the abnormal region of the brain is detected, the comparison circuit 28 compares the line-of-sight direction with the abnormal region of the brain and determines whether or not the comparison results are consistent (step ST140). .. If the comparison results are inconsistent, the process proceeds to step ST150, and if the comparison results are consistent, the process proceeds to step ST160.

In step ST150, the rendering circuit 26 displays a flag indicating inconsistency. The flag is shown, for example, in description 40 of FIG. In step ST160, the rendering circuit 26 displays the result on the display device 16. After step ST160, the flowchart of FIG. 4 ends.

FIG. 5 is a flowchart showing a second specific example in the embodiment. In the second specific example, the line-of-sight direction specified from the CT image is compared with the line-of-sight direction specified by another method. Another method is, for example, the line-of-sight direction input by the user.

When the flowchart of FIG. 5 starts, the line-of-sight detection circuit 24 acquires the CT image collected by the CT scanner 14 (step ST210). After acquiring the CT image, the line-of-sight detection circuit 24 identifies the first line-of-sight direction based on the acquired CT image (step ST220).

Next, the calculation device 12 acquires the second line-of-sight direction (step ST230). Specifically, the calculation device 12 acquires the second line-of-sight direction input by the user using the input device 18.

After the first line-of-sight direction is specified and the second line-of-sight direction is acquired, the comparison circuit 28 compares the first line-of-sight direction with the second line-of-sight direction, and whether or not the comparison result is consistent. (Step ST240). If the comparison results are inconsistent, the process proceeds to step ST250, and if the comparison results are consistent, the process proceeds to step ST260.

In step ST250, the rendering circuit 26 displays a flag indicating inconsistency. The flag is shown, for example, in description 40 of FIG. In step ST260, the rendering circuit 26 displays the result on the display device 16. After step ST260, the flowchart of FIG. 5 ends.

FIG. 6 is a flowchart showing a third specific example in the embodiment. In the third specific example, the line-of-sight direction identified from the CT image is compared with the abnormal region of the brain detected by the CTA scan.

When the flowchart of FIG. 6 starts, the line-of-sight detection circuit 24 acquires a CT image collected by the CT scanner 14 (step ST310). After acquiring the CT image, the line-of-sight detection circuit 24 specifies the line-of-sight direction based on the acquired CT image (step ST320).

Next, the computing device 12 detects an abnormal region of the brain based on the CTA scan (step ST330). Specifically, the calculation device 12 detects an abnormal region of the brain from the image of the CTA scan acquired by the CT scanner 14.

After the line-of-sight direction is specified and the abnormal region of the brain is detected, the comparison circuit 28 compares the line-of-sight direction with the abnormal region of the brain and determines whether or not the comparison results are consistent (step ST340). .. If the comparison results are inconsistent, the process proceeds to step ST350, and if the comparison results are consistent, the process proceeds to step ST360.

In step ST350, the rendering circuit 26 displays a flag indicating inconsistency. In step ST360, the rendering circuit 26 displays the result on the display device 16. After step ST360, the flowchart of FIG. 6 ends.

Although specific circuits have been described herein, in alternative embodiments, the functionality of one or more of these circuits is provided by a single processing resource or other component. It is possible, or the functionality provided by a single circuit, can be provided by a combination of two or more processing resources or other components. Reference to a single circuit includes multiple components that provide the functionality of such a single circuit, whether or not such components are separated from each other, and refers to multiple circuits. References include a single component that provides the functionality of such multiple circuits.

According to at least one embodiment described above, the abnormality of the brain can be easily grasped.

Although some embodiments have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other embodiments, and various omissions, replacements, changes, and combinations of embodiments can be made without departing from the gist of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, as well as in the scope of the invention described in the claims and the equivalent scope thereof.

10 Medical image processing device 30, 32, 34, 38, 46, 48, 52, 54, 56, 58, 60 Stage 40 Depiction 42 First line art 44 Second line art 50 Flag 62 Display screen 64 CT image 66 Second CT image 68 Abnormal area 70 Line-of-sight image

Claims (23)

Acquiring image data representing the brain of the subject and
Obtaining data representing the clinical signs or symptoms of the subject and the clinical signs or symptoms are related to the condition of the brain.
Processing the image data and obtaining an estimate of the abnormality in the brain of the subject.
Determining whether the presumption of the abnormality is consistent with the data representing the clinical sign or symptom.
A medical image processing apparatus comprising a processing circuit configured to perform the above. The medical image processing apparatus according to claim 1, wherein the condition of the brain is a stroke, and the clinical sign or symptom indicates the laterality of the stroke. The medical image processing apparatus according to claim 1, wherein the clinical sign or symptom comprises at least one of the behavior of a body part, the position of the body part, the orientation of the body part, and the movement of the body part. The clinical sign or symptom comprises at least one of gaze direction, gaze angle, asymmetric muscle tone, head angle on the trunk, facial asymmetry, tongue deviation, claim 1. The medical image processing apparatus according to. The medical image processing apparatus according to claim 1, wherein the clinical sign or symptom comprises a persistent ocular deviation. The medical image processing apparatus according to claim 1, wherein the clinical sign or symptom comprises an eye deviation that occurs in two or more scans and / or two or more times. The image data comprises data collected as a plurality of times and / or a plurality of scans, and the processing circuit exhibits the clinical signs or symptoms in each of the plurality of times and / or each of the plurality of scans. The medical image processing apparatus according to claim 1, which is configured to acquire the data to be displayed. The processing circuit is further configured to process the image data in order to obtain the data representing the clinical sign or symptom, and / or the processing circuit is said data representing the clinical sign or symptom. The medical image processing apparatus according to claim 1, further configured to process additional image data in order to obtain the above. The medical image processing apparatus according to claim 1, wherein the processing circuit is further configured to display reference information when the presumption of the abnormality does not match the data representing the clinical sign or symptom. The processing circuit displays an image rendered from at least a part of the image data, and displays a suggestion of at least one anomalous region dependent on the estimation of the anomaly on the rendered image. The medical image processing apparatus according to claim 1, further configured to perform the above. The processing circuit is further configured to receive a selection on the side of the brain, which determines that the presumption of the abnormality does not match the data representing the clinical sign or symptom. Performed in response,
The processing circuit processes the image data using the processing restricted to the selected side of the brain, thereby obtaining an estimate of anomalies on the selected side of the brain. The medical image processing apparatus according to claim 1, further configured to perform the above. The medical image processing apparatus according to claim 1, wherein the processing circuit is further configured to display a simplified depiction of the clinical sign or symptom. The processing circuit is configured to acquire individual data representing the clinical sign or symptom of the subject in each of a plurality of times and / or a plurality of scans, the simplified depiction of the subject. The medical image processing apparatus according to claim 12, wherein the clinical sign or symptom is exhibited in each of a plurality of times and / or each of the plurality of scans. The processing circuit is configured to select a portion of the image data associated with the clinical sign or symptom and to display the selected portion of the image data. The medical image processing apparatus according to 1. The medical image processing apparatus according to claim 1, wherein the abnormality comprises at least one of an ischemic region and a highly resorbable blood vessel region. The medical image processing apparatus compares the data representing the clinical sign or symptom with previously acquired data related to the clinical sign or symptom, and the data and the previously acquired data. The medical image processing apparatus according to claim 1, wherein the medical image processing apparatus according to claim 1 is configured to identify whether or not there is a mismatch between the two. The medical image processing apparatus according to claim 1, wherein the processing of the image data depends on the data representing the clinical sign or symptom. The medical image processing apparatus according to claim 1, wherein the processing of the image data includes applying a trained model. The medical image processing apparatus according to claim 18, wherein the data representing the clinical sign or symptom is used as an input to the trained model. a) The data representing the clinical sign or symptom is input to multiple network layers of the trained model.
b) The data representing the clinical sign or symptom is used to modify the activation function of the trained model.
c) A single trained model is used to process the image data for both obtaining the data representing the clinical sign or symptom and obtaining the presumption of the abnormality. Be done,
The medical image processing apparatus according to claim 19, which is at least one of a) to c). The medical image processing apparatus according to claim 1, wherein the data representing the clinical signs or symptoms of the subject is data input by the user. Acquiring image data representing the brain of the subject and
Acquiring data representing the clinical signs or symptoms of the subject, wherein the clinical signs or symptoms are related to the condition of the brain.
Processing the image data and obtaining an estimate of the abnormality in the brain of the subject.
Determining whether the presumption of the abnormality is consistent with the data representing the clinical sign or symptom.
A medical image processing method comprising. Acquiring image data representing the brain of the subject and
Acquiring data representing the clinical signs or symptoms of the subject, wherein the clinical signs or symptoms are related to the condition of the brain.
Processing the image data and obtaining an estimate of the abnormality in the brain of the subject.
Determining whether the presumption of the abnormality is consistent with the data representing the clinical sign or symptom.
A program that comprises computer-readable instructions that can be executed to do so.