JP2021058272A - Medical image processing device, tomographic device, medical image processing method and program - Google Patents

Abstract

To acquire an image analysis result as immediately as possible after imaging a subject.SOLUTION: A medical image processing device includes: acquisition means for acquiring measurement data from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data; detection means for detecting a specific injury or disease on the basis of the measurement data; and notification means for notifying detection of a specific injury or disease in accordance with detection of a specific injury or disease by the detection means.SELECTED DRAWING: Figure 1

Description

The present invention relates to a medical image processing apparatus, a tomography apparatus, a medical image processing method and a program.

In the medical field, in order to identify a disease of a subject and observe the degree of the disease, image diagnosis using images acquired by various imaging devices (modality) is carried out. Examples of the imaging device include an X-ray imaging device, an X-ray computed tomography device (CT), a magnetic resonance imaging device (MRI), a positron emission tomography device (PET), an ultrasonic diagnostic device, and the like. Examples of imaging devices in the field of ophthalmology include fundus cameras, scanning laser ophthalmoscopes (SLO), optical coherence tomography imaging devices (OCT), and OCT angiography imaging devices (OCTA).

Image diagnosis is basically carried out by a doctor observing the lesions depicted in the image, but in recent years, various information useful for diagnosis can be obtained by image analysis using a medical image processing device or the like. became. For example, image analysis has made it possible to detect small lesions that may be overlooked and prompt the doctor's confirmation, or to quantitatively measure the shape and volume of lesions.

To perform diagnostic imaging, a technician or doctor first operates a imaging device to image the subject. Next, the image acquired by the imaging device is stored in a database provided in the imaging device, an image management system connected to the network of the medical institution facility, or the like. Finally, diagnostic imaging is performed by reference to the stored images by a radiologist or a physician with a high degree of expertise in possible diseases and trauma.

Conventional image analysis is performed using an image observed by a doctor at the time of image diagnosis (hereinafter referred to as a diagnostic image) as in Patent Document 1. In other words, the image diagnosis by the doctor and the conventional image analysis cannot be started until the photograph of the subject and the reconstruction of the diagnostic image are completed, and the diagnostic result or the analysis result can be obtained. There was a risk that it would be after a considerable amount of time had passed since the start of shooting. Therefore, even if the subject is found to have a disease or trauma that requires urgent treatment by image diagnosis or image analysis, the subject has already returned home or other reasons at a medical institution facility. Since then, he has disappeared, and the treatment was delayed and became serious.

Further, according to Patent Document 2, it is possible to notify a doctor of an abnormality in a test value, but since it is possible to notify a doctor after the diagnostic image is stored in an image storage communication system (PACS), Patent Document 1 Similarly, the treatment was delayed and the subject may become serious.

Japanese Unexamined Patent Publication No. 2016-87109 Japanese Unexamined Patent Publication No. 4-236938

In view of the above circumstances, it is an object of the present invention to obtain an analysis result as quickly as possible after photographing a subject.

One aspect of the present invention is
An acquisition means for acquiring the measurement data from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data, and an acquisition means.
A detection means for detecting a specific injury or illness based on the measurement data,
In response to the detection of a specific injury or illness by the detection means, a notification means for notifying that a specific injury or illness has been detected, and a notification means.
It is a medical image processing apparatus provided with.

One aspect of the present invention is
A medical image processing method performed by a computer
An acquisition step of acquiring the measurement data from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data, and
A detection step for detecting a specific injury or illness based on the measurement data,
A notification step for notifying that a specific injury or illness has been detected according to the detection of the specific injury or illness, and a notification step.
It is a medical image processing method including.

According to the present invention, the analysis result can be obtained quickly after the subject is photographed.

Configuration diagram of the photographing system and the image processing apparatus according to the first and second embodiments. Diagram illustrating the relationship between the projected image and the diagnostic image The figure explaining the area label image with respect to the diagnostic image The figure explaining the area label image with respect to the projected image Diagram illustrating a disease area label image The figure which shows the example of the neural network which outputs the area label image by image segmentation. Diagram showing an example of a neural network that outputs an index of the presence or absence of anomalies Flow diagram showing the flow of processing performed by the notification server Flow chart showing the flow of processing in Embodiment 1-3 Configuration diagram of the photographing system and the image processing apparatus according to the fourth embodiment Flow chart showing the flow of processing in Embodiments 4, 5, 10 and 11. Schematic diagram of the configuration of the image processing apparatus according to the fifth to eleventh embodiment. Flow chart showing the flow of processing in Embodiment 6-9

Hereinafter, exemplary embodiments for carrying out the present invention will be described in detail with reference to the drawings. However, the dimensions, materials, shapes, relative positions of the components, etc. described in the following embodiments are arbitrary and can be changed according to the configuration of the device to which the present invention is applied or various conditions. Also, in the drawings, the same reference numerals are used between the drawings to indicate elements that are the same or functionally similar.

<Explanation of terms>
First, terms used in the present specification will be described. For the sake of clarity, the following terminology will be described by taking a CT (Computed Tomography) device as an example. It is assumed that the imaging device is a CT device, the projected image and the diagnostic image are acquired by the CT device, and the diagnostic image is a three-dimensional image composed of a plurality of tomographic images. However, the CT device may be replaced with any other modality.
Further, the diagnostic image may be a two-dimensional image.

(network)
A network refers to a connection that enables multiple devices (computers) to communicate with each other, or the entire connected system. Each device in the network may be connected by wire, wireless, or both. Lines connecting each device in the network include, for example, a dedicated line, a local area network (hereinafter referred to as LAN) line, a wireless LAN line, an Internet line, Wi-Fi (registered trademark), and Bluetooth (registered trademark). ..

(Medical image processing device)
The medical image processing device is a device that performs image processing on a medical image. Examples of image processing include diagnostic imaging processing including detection of lesions and quantitative measurement of lesion shape, volume, and the like. The medical image processing device may be configured by the same device as the control device (console) that controls the imaging device, or may be configured by a different device. The medical image processing device may be composed of two or more devices capable of communicating with each other, or may be composed of a single device. Further, each component of the medical image processing apparatus may be composed of a software module executed by a processor such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit). In addition, each component may be configured by a circuit or the like that performs a specific function such as an ASIC. Further, it may be configured by a combination of any other hardware and any software.

(Image management system)
The image management system is a device and a system that receives and stores an image taken by an imaging device such as a CT or an image processed image. The image management system can transmit an image in response to a request from the connected device, perform image processing on the stored image, and request another device to perform image processing. An example of an image management system is Picture Archiving and Communication Systems (PACS).
The image management system may store the information of the subject and the incidental information such as the shooting time associated with the received image as related information in another database. In addition, the image management system is connected to a network and can send and receive images, convert images, and send and receive various information associated with saved images in response to requests from other devices. ..

(Diagnostic image)
The diagnostic image is an image that a doctor observes at the time of image diagnosis. The diagnostic image is obtained by reconstructing the projected image. Diagnostic images are stored in a database provided in the imaging device, an image management system connected to a network of medical institution facilities, etc., and can be observed by a doctor.

(Measurement data / raw data)
The medical imaging device performs medical imaging on the subject according to the imaging principle according to the medical imaging device, and collects measurement data on the subject. The measured data is also referred to as raw data. The measurement data (raw data) of the X-ray computed tomography apparatus is projection data or synogram data. The measurement data of the magnetic resonance imaging device is k-space data, the measurement data of the ultrasonic diagnostic device is echo data, the measurement data of the PET device is coincidence data or synogram data, and the measurement data of the SPECT device is projection data or synogram data. .. The measurement data of the X-ray diagnostic apparatus is X-ray image data.

The measurement data itself is not an image showing the structure of the subject directly, but a diagnostic image showing the structure of the subject is obtained by the reconstruction process.

Hereinafter, projection data, which is measurement data (raw data) of the X-ray CT apparatus, will be described. The projection data is output data (raw data) of the detector of the photographing apparatus at each projection position. The projected data has a two-dimensional, three-dimensional or four-dimensional data structure and can be displayed as an image, so that it is also called a projected image. An image in which projection data are arranged in order according to the angle of projection is also called Sinogram data.

More specifically with reference to FIG. 2, when the gantry of the imaging device moves from the crown of the subject to the direction of the foot (Z-axis direction) and imaging is started, the projected tomographic images on the XY plane are sequentially displayed. It is acquired, and finally the three-dimensional projected image Im210 is acquired. The diagnostic image Im230 can be obtained by reconstructing the tomographic image Im220 of an arbitrary XY plane from the projected image Im210.

(Reconstruction)
Reconstruction is a process of obtaining a diagnostic image from a projected image. Examples of the reconstruction method are a filter-corrected back projection method and an iterative reconstruction method. The imaging device or the terminal connected to the imaging device reconfigures the projected image acquired by the imaging device, and the diagnostic image is acquired.

The reconstruction process takes time, and depending on the image conditions, it may take several minutes to several hours from the start of shooting until the diagnostic image is acquired. Since the projected image is usually not used for diagnosis, it may be deleted immediately after the diagnostic image is reconstructed, deleted after a certain period of time, or deleted when the data storage area of the photographing apparatus becomes small. To do.

(Imaging conditions / Imaging protocol)
The imaging condition is a group of setting information related to imaging that is set in the imaging device when photographing the subject. For example, the imaging conditions include the device model, the imaging range and position of the subject, the tube voltage of the imaging device, the tube current, the scan slice thickness, the scan time, the amount of contrast medium used for contrast, and the contrast medium injection time. The shooting time phase (timing) and the like are included. In general, it is possible to save the trouble of resetting by selecting one of a preset group of a plurality of imaging conditions prepared in advance according to the contents of the examination such as the medical condition and the site of the subject to be photographed. There are many. The imaging conditions are also referred to as imaging protocols.

(Image condition)
The image condition is a group of setting information used when generating a diagnostic image. Specifically, it is setting information used when reconstructing a diagnostic image from a projected image in CT or PET. Alternatively, it is setting information used when reconstructing a diagnostic image from K-Space data in MRI. The image conditions include, for example, a reconstruction method, FOV, image resolution, slice thickness, slice interval, reconstruction function, and the like. In general, as in the case of imaging conditions, by selecting one of a plurality of preset image conditions prepared in advance according to the content of the examination such as the medical condition and site of the photographed subject, the image can be re-created. It often saves the trouble of setting. Further, depending on the device, the image conditions may be automatically set according to the imaging conditions.

The imaging conditions and image conditions are stored in the data structure constituting the diagnostic image, stored as data separate from the image, or stored in a database or image management system related to the imaging device. Therefore, it can be acquired by a procedure corresponding to the photographing conditions of the photographing apparatus and the storage means of the image conditions. Specifically, the shooting conditions and the image conditions are obtained, for example, by analyzing the data structure of the image output by the shooting device, acquiring the data stored separately from the image, or acquiring from the database related to the shooting device. Obtained by accessing the interface for

(Area label image)
The area label image is a label image in which an area is labeled for each pixel. Specifically, among the region groups depicted in the image Im310 showing the tomographic image of the chest acquired by the imaging device in FIG. 3, a pixel value (hereinafter, region label value) group capable of identifying an arbitrary region. It is an image Im320 divided by.

The specified arbitrary region includes a region of interest (ROI: Region Of Interest), a volume of interest (VOI: Volume Of Interest), and the like. By specifying the coordinate group of the pixel having an arbitrary area label value from the area label image Im320, the coordinate group of the pixel depicting the corresponding lung region in the image Im310 can be specified. Specifically, for example, when an area label value 1 indicating the right lung is specified, a coordinate group having a pixel value of 1 among the pixel groups of the area label image Im320 is specified, and the image Im310 corresponds to the coordinate group. Pixel group to be extracted is extracted. Thereby, the region of the right lung in the image Im310 can be identified. Of course, if the area label image has a pixel value that can identify the lesion area, the coordinate group of the lesion area can be specified from the corresponding image.

The image in which the region corresponding to the region label image can be specified is not limited to the two-dimensional tomographic image Im310, and the type and dimension thereof vary depending on the embodiment. The area label image may be, for example, a two-dimensional image in which the area label value is attached to each pixel of the two-dimensional projected image, or a three-dimensional image in which the area label value is attached to each pixel of the three-dimensional diagnostic image. But it may be.

In some embodiments, the area label image may be reduced or enlarged. In the image interpolation processing used for reducing or enlarging the area label image, the nearest neighbor method or the like shall be used so as not to mistakenly generate an undefined area label value or an area label value that should not exist at the corresponding coordinates. ..

(Image segmentation processing)
The image segmentation process is a process for identifying an area called ROI or VOI such as an organ, a lesion, or a trauma depicted in an image for use in image diagnosis or image analysis. For example, the image segmentation process identifies the right lung, the left lung, the pneumothorax region, which is a disease of the lung, and the like from the images obtained by photographing the chest.

As long as a plurality of regions to be specified are depicted in the image, the number of regions to be specified may be a plurality, or may be one region surrounding the regions so as to include these regions. Further, if the area to be specified is not drawn in the image, the number of areas to be specified is 0.

The specified area group is output as information that can be used in other processing. Specifically, for example, the coordinate group of the pixel group constituting each of the specified region groups can be output as a numerical data group. Further, for example, a coordinate group indicating a rectangular region, an ellipsoid region, a rectangular parallelepiped region, an ellipsoid region, or the like including each of the specified region groups can be output as a numerical data group. Further, for example, a coordinate group indicating a straight line, a curve, a plane, a curved surface, or the like corresponding to the boundary of the specified region group can be output as a numerical data group. Further, for example, it is possible to output an area label image showing the specified area group. In the following, when it is expressed that the accuracy of the image segmentation processing is high or it is expressed as a region label image with high accuracy, it means that the ratio that the region can be correctly specified is high. On the contrary, when it is expressed that the accuracy of the image segmentation processing is low, it means that the ratio of erroneously specifying the region is high.

(Trained model and teacher data)
The trained model is a model in which training (learning) is performed in advance using appropriate teacher data (learning data) for a machine learning model that follows an arbitrary machine learning algorithm such as deep learning. .. However, the trained model does not require further learning, and additional learning can be performed.

The teacher data is composed of one or more pairs of input data and output data. The format and combination of the input data and output data of the pair group that composes the teacher data may be one of which is an image and the other of which is a numerical value, or one of which is composed of multiple image groups and the other of which is a character string. May be an image or the like, which is suitable for a desired configuration.

As a first example of teacher data, specifically, teacher data composed of a pair group in which a two-dimensional projected image acquired by CT is used as input data and an abnormality label corresponding to the image is used as output data ( Hereinafter, the first teacher data) can be mentioned. The abnormality label is a unique numerical value or character string indicating whether or not there is an abnormality such as an injury or illness or an image artifact in the shooting range corresponding to the projected image. For example, 0 is set if there is no abnormality, and 1 is set if there is an abnormality. Depending on the design, the input data may be a three-dimensional or four-dimensional projected image. The four-dimensional image includes, for example, a three-dimensional moving image or an image in which parameters at each pixel position of the three-dimensional image are shown in different hues.

As a second example of teacher data, teacher data composed of a pair group in which a two-dimensional diagnostic image acquired by CT is used as input data and an abnormality label corresponding to the image is used as output data (hereinafter, second). Teacher data). The abnormality label is a unique numerical value or character string indicating whether or not there is an abnormality such as an injury or illness or an image artifact in the imaging range corresponding to the diagnostic image. For example, 0 is set if there is no abnormality, and 1 is set if there is an abnormality. Depending on the design, the input data may be used as a three-dimensional or four-dimensional diagnostic image.

As a third example of the teacher data, a pair group using a two-dimensional projected image acquired by CT as input data and a two-dimensional area label image capable of identifying the region of interest drawn in the image as output data. The constructed teacher data (hereinafter referred to as a third teacher data) can be mentioned. Areas of interest are, for example, areas of the lung field and pneumothorax. The input data is, for example, the two-dimensional projected image Im410 which is a part of the three-dimensional projected image shown in FIG. 4, and the output data is the two-dimensional image which can identify the region of the chest depicted in the projected image Im410. It is the area label image Im420 of. Depending on the design, the input data may be a three-dimensional or four-dimensional projected image, and the output data may be a two-dimensional to four-dimensional area label image.

As a fourth example of teacher data, a pair group in which a two-dimensional diagnostic image acquired by CT is used as input data and a two-dimensional area label image capable of identifying an area of interest drawn in the image is used as output data. The teacher data composed by (hereinafter, the fourth teacher data) can be mentioned. The input data is, for example, the two-dimensional diagnostic image Im510 which is a part of the three-dimensional diagnostic image shown in FIG. 5, and the output data can identify the region of the chest drawn on the diagnostic image Im510. Two-dimensional area label image Im520. Depending on the design, the input data may be a three-dimensional or four-dimensional diagnostic image, and the output data may be a two-dimensional to four-dimensional area label image.

When the input data is input, the trained model outputs the output data according to the design of the trained model. For example, a trained model designed for image segmentation problems and regression problems outputs output data that is likely to correspond to the input data according to the tendency of the output data corresponding to the input data of the teacher data used for training. To do. Also, for example, a trained model designed for a classification problem is classified into each of the classes defined by the design according to the tendency of the output data corresponding to the input data of the teacher data used for training. Output the possibility of obtaining as a numerical group.

Specifically, for example, when a projected image is input to a trained model trained with the first teacher data, the trained model outputs a value indicating whether or not there is an abnormality in the projected image, or an abnormality due to design. It outputs a numerical value indicating the degree of presence or absence.

Further, for example, when a diagnostic image is input to the trained model trained by the second teacher data, the trained model outputs a value indicating whether or not there is an abnormality in the diagnostic image, or an abnormality is caused by the design. It outputs a numerical value indicating the degree of presence or absence.

Further, for example, when a projected image is input to the trained model trained by the third teacher data, the trained model outputs a region label image that can identify the lung field and pneumothorax region depicted in the projected image. ..

Further, for example, when a diagnostic image is input to the trained model trained by the fourth teacher data, the trained model produces a region label image that can identify the lung field and pneumothorax regions depicted in the diagnostic image. Output.

(Machine learning algorithm)
Machine learning algorithms include techniques for deep learning such as convolutional neural networks (CNNs). In the method related to deep learning, the layers and nodes constituting the neural network are set, the parameters for the optimization algorithm regarding the weights and biases set in the nodes and the like are set, and the loss function is appropriately designed. Performance such as accuracy rate, precision rate, and recall rate may differ depending on the design of these parameters and the loss function.

For example, in the trained model of deep learning using the first teacher data or the second teacher data, if more appropriate parameters are set, there is no abnormality such as injury or illness in the input image. May be more likely to be output correctly. Further, for example, in the trained model using the third teacher data or the fourth teacher data, if more appropriate parameters are set, a more accurate area label image may be output.

Specifically, the parameters in the CNN layer group and node group are, for example, the kernel size of the filter, the number of filters, the stride value, and the dilation value set for the convolutional layer, and the fully connected layer. It can include the number of nodes to be output from. The parameter group and the number of training epochs can be set to preferable values for the usage pattern of the trained model based on the teacher data. For example, based on the teacher data, it is possible to set a parameter group and the number of epochs that can correctly output whether or not there is an abnormality with a higher probability and output a more accurate area label image.

A method for determining the above parameter group and the number of epochs will be illustrated. In the method called the holdout method, first, 70% of the pair group constituting the teacher data is selected for training, and the remaining 30% is selected for evaluation. Next, the trained model is trained using the training pair group, and the training evaluation value is calculated using the evaluation pair group at the end of each training epoch. The training evaluation value is, for example, the average value of a value group in which the output when the input data constituting each pair is input to the trained model during training and the output data corresponding to the input data are evaluated by the loss function. Is. Finally, the parameter group and the number of epochs at the time when the training evaluation value is the best (depending on the design of the loss function, the time of the minimum value or the time of the maximum value) are set as the parameters of the trained model. Determined as a group or number of epochs. In this way, by dividing the pair group that constitutes the teacher data into training and evaluation and determining the number of epochs, it is possible to prevent the trained model from overtraining the training pair group. be able to.

Further, as an example of other methods, there is also a method called a cross-validation method. In the cross-validation method, the pair group constituting the teacher data is divided into a plurality of sets, one set is used as evaluation data, and the other set group is used as training data for performance evaluation. Then, the parameter group and the number of epochs are determined according to the average performance of each result of evaluating each set as evaluation data.

An example of the above loss function is given. When targeting a regression problem, for example, the absolute value of the difference between the value of a certain output data of the teacher data and the value output by inputting the input data corresponding to the output data into the trained model, or the square. The error may be the output of the loss function. Further, when the image segmentation problem is targeted, for example, with respect to the image of a certain input data of the teacher data, the average of the area label image forming the pair and the area label image output by the trained model regarding the pixel value group. The squared error may be the output of the loss function.

(Image segmentation engine)
The image segmentation engine is a module that performs image segmentation processing and outputs an area label image corresponding to the input input image. Examples of the input image include a two-dimensional or three-dimensional projected image of CT, a diagnostic image, K-Space data of MRI, a diagnostic image, a B-scan image of OCT, a three-dimensional image, and the like. Further, as an example of the area label image, the area label image showing the area of injury or illness when the input image is a two-dimensional or three-dimensional image of CT, or the retinal layer when the input image is a B scan image of OCT. There is an area label image showing each layer and the like.

In the image processing method constituting the image segmentation processing method in the following embodiment, processing using a trained model according to various machine learning algorithms such as deep learning may be performed. The image processing method may be performed not only by a machine learning algorithm but also by any other existing processing. The image processing includes, for example, various image filter processing, matching processing using a database of area label images corresponding to similar images, image registration processing of reference area label images, knowledge base image processing, and the like. .. Further, in the image processing method, the final area label image may be output by making a majority decision or averaging the results of a plurality of image processings by ensemble processing.

In particular, there are various configurations of a convolutional neural network (CNN) that performs image segmentation processing on an image input as an input image and outputs a region label image. As an example, there is a configuration shown in FIG.

Specifically, when the projected image is input by training the configuration with, for example, the third teacher data, learning to output a region label image capable of identifying the pneumothorax region depicted in the projected image. A finished model is obtained. Further, when a diagnostic image is input by training the configuration with, for example, the fourth teacher data, a learned area label image that can identify the pneumothorax area drawn on the diagnostic image is output. A model is obtained.

The configuration includes a plurality of layers that are responsible for processing and outputting an input value group. In addition, it should be noted.
As shown in FIG. 6, the types of layers included in the configuration include a convolution layer, a downsampling layer, an upsampling layer, and a synthetic layer.

The convolutional layer is a layer that performs convolutional processing on an input value group according to parameters such as the kernel size of the set filter, the number of filters, the stride value, and the dilation value. The downsampling layer is a layer that performs processing to reduce the number of output value groups to be smaller than the number of input value groups by thinning out or synthesizing input value groups. Specifically, as the process performed in the downsampling layer, for example, there is a Max Pooling process.

The upsampling layer is a layer that performs processing to increase the number of output value groups to be larger than the number of input value groups by duplicating the input value group or adding the interpolated value from the input value group. Specifically, as the process performed in the upsampling layer, for example, there is a linear interpolation process. The composite layer is a layer in which a value group such as an output value group of a certain layer or a pixel value group constituting an image is input from a plurality of sources, and the processing is performed by concatenating or adding them.

As parameters set in the convolutional layer group included in the above configuration, for example, by setting the kernel size of the filter to 3 pixels in width and 3 pixels in height and the number of filters to 64, image segmentation processing with a certain accuracy is performed. Is possible. However, it should be noted that if the parameter settings for the layers and nodes that make up the neural network are different, the degree to which the tendency trained from the teacher data can be reproduced in the output data may differ. That is, in many cases, the appropriate parameters for each layer group and each node group differ depending on the embodiment, and may be changed as necessary.

Further, depending on the embodiment, better characteristics of the CNN may be obtained by changing the configuration of the CNN as well as the method of changing the parameters as described above. Better characteristics include, for example, high accuracy of image segmentation processing, short time of image segmentation processing, short training time of trained model, and the like. Examples of modification of the CNN configuration include incorporating a batch normalization layer and an activation layer using a rectifier linear unit after the convolutional layer.

When it is necessary to process a one-dimensional image, a three-dimensional image, or a four-dimensional image, the kernel size of the filter may correspond to the one-dimensional image, the three-dimensional image, or the four-dimensional image.

Further, the image segmentation process may be performed by only one image processing method, or may be performed by combining two or more image processing methods. Further, it is possible to carry out a plurality of image segmentation processing methods to generate a plurality of area label images.

Further, depending on the embodiment, the input image is divided into small area groups, and image segmentation processing is performed on each of them to obtain a small area area label image group, and the small area area label image group is combined. Therefore, there is a method of generating an area label image. If the input image is a three-dimensional image, the small area may be a three-dimensional image smaller than the input image, a two-dimensional image, or a one-dimensional image. Further, the small area may be a two-dimensional image smaller than the input image or a one-dimensional image as long as the input image is a two-dimensional image. Further, depending on the embodiment, a plurality of area label image groups may be output.

In addition, parameters may be input to the image segmentation engine together with the input image. The parameters input in this case may include parameters that specify the degree of the range in which image segmentation processing is performed, such as the upper limit of the size of the lesion, and parameters that specify the image filter size used in the image processing method. it can. Depending on the embodiment, the image segmentation engine may output other images or coordinate data groups that can specify the area instead of the area label image.

When performing a plurality of image segmentation processing methods or performing image segmentation processing on a plurality of small area groups, the processing time can be shortened by performing the image segmentation processing in parallel.

When using some image processing methods such as image processing using CNN, it is necessary to pay attention to the image size. Specifically, it should be noted that different image sizes may be required for the input image and the output area label image in order to deal with the problem that the peripheral part of the area label image is not segmented with sufficient accuracy. Is.

For the sake of clarity, although not specified in the embodiments described later, when an image segmentation engine that requires different image sizes for the image input to the image segmentation engine and the image to be output is adopted, the image size is appropriately adjusted. It is assumed that it is being adjusted. Specifically, the input image such as the image used for the teacher data for training the trained model or the image input to the image segmentation engine is padded or the shooting area around the input image is combined. Adjust the image size by doing. The area to be padded is filled with a constant pixel value, filled with neighboring pixel values, or mirror padded according to the characteristics of the image segmentation processing method so that the image segmentation processing can be effectively performed.

(Anomaly detection engine)
The anomaly detection engine performs anomaly detection processing and checks the input input image for abnormalities such as healthy subjects, normal shooting environments, normal shooting methods, and other injuries and illnesses and image artifacts. It is a module to judge.

In the process of determining the presence or absence of an abnormality in the abnormality detection process, a value indicating the degree of abnormality is output, or a value indicating the degree of abnormality such as the severity of injury or illness is output. In that case, the abnormality detection engine may output the value together with the determination.

Examples of the input image include a two-dimensional or three-dimensional projected image of CT, a diagnostic image, K-Space data of MRI, a diagnostic image, a B-scan image of OCT, a three-dimensional image, and the like.

In addition, as an example of an abnormality due to injury or illness, when the input image is a CT image, an abnormality in which pneumothorax or aortic dissection, body fluid or blood retention, fracture, etc. are depicted, and the input image is an OCT B scan image. In some cases, there is an abnormality in which edema or a thin nerve cell layer is visualized.

Further, as an example of an abnormality due to an image artifact, when the input image is a CT image or an MRI image, an image caused by a metal inside the subject's body, a movement of the subject during shooting, a malfunction of the device, or the like. There is an anomaly in which the artifact is depicted.

In the image processing method constituting the method of abnormality detection processing in the following embodiment, processing using a trained model according to various machine learning algorithms such as deep learning may be performed. The image processing method may be performed not only by a machine learning algorithm but also by any other existing processing. The image processing includes, for example, various image filter processing, matching processing using a database of similar images, image registration processing of normal images, knowledge-based image processing, and the like. Further, in the image processing method, the presence or absence of an abnormality may be finally determined by, for example, making a majority decision or averaging the results of a plurality of image processings by ensemble processing.

In particular, there are various configurations of CNNs that output index values (probabilities, etc.) for determining the presence or absence of abnormalities in an image input as an input image. As an example, there is a configuration shown in FIG. ..

Specifically, a trained model that outputs a numerical value that makes it possible to determine whether or not there is an abnormality in the projected image when the projected image is input by training the configuration with, for example, the first teacher data. Is obtained. Further, when the diagnostic image is input by training the configuration with, for example, the second teacher data, a trained model that outputs a numerical value that makes it possible to determine whether or not there is an abnormality in the diagnostic image is available. can get.

The configuration includes a plurality of convolutional processing blocks composed of a convolutional layer, a batch normalization layer, and an activation layer using a rectified linear function. The configuration also includes a final convolutional layer, a Fully Connected layer, and an output layer. The fully connected layer fully connects the output value group of the convolution processing block. Further, the output layer uses the sigmoid function to output the degree of whether or not an abnormality is depicted in the input image as an estimation result (Result). That is, the value of the estimation result approaches 1 if an abnormality is drawn in the input image, and approaches 0 if it is not drawn.

By setting the number of convolutional processing blocks included in the configuration to 16 and the parameters of the convolutional layer group to be 3 pixels in width and 3 pixels in height and 64 in the number of filters, the accuracy is constant. The presence or absence of abnormalities can be estimated. However, in practice, as described in the above description of the trained model, a better parameter group can be set by using the teacher data according to the usage pattern of the trained model. When it is necessary to process a one-dimensional image, a three-dimensional image, or a four-dimensional image, the kernel size of the filter may be expanded to one-dimensional, three-dimensional, or four-dimensional. The anomaly detection processing method may be carried out using only one image and data processing method, or may be carried out by combining two or more image and data processing methods.

Further, as another method of abnormality detection processing, there is also a method of using a machine learning algorithm called Generative Adversarial Network (GAN) or Auto Encoder (AE). Specifically, in order to determine the presence or absence of an abnormality by outputting a pseudo normal image close to the appearance of the input image by GAN or AE and calculating the difference between the pseudo normal image and the input image. Outputs a value that serves as an index for. In other words, the magnitude of the difference indicates the closeness between the pseudo normal image and the input image, so if the difference is small, it is judged that there is no abnormality because it is close to normal, and if the difference is large, there is an abnormality. to decide. In addition, by applying a look-up table to the value of the difference between images, it is possible to output a heat map image that visually shows which region in the image the difference was in.

Further, the output result of the image segmentation engine may be used as a part of the abnormality detection processing method. For example, when an image is input to an image segmentation engine that outputs an injury / illness area label image and the area or volume of the injury / illness area or the number of pixels in the output area label image exceeds a set threshold value, an abnormality occurs. There is a way to assume that it is depicted.

(console)
The console is an information processing device (computer) connected to the photographing device and having a user interface capable of operating the photographing device. When the imaging device is a CT device, the console is connected to a gantry of the CT device and controls the gantry to perform tomography.

In medical institution facilities, users such as technicians and doctors can operate the console to set imaging conditions and perform imaging procedures to photograph the subject. In addition, the user can operate the console to set image conditions and perform an image processing procedure for reconstructing the projected image or K-Space data obtained by shooting into a diagnostic image. In addition, the console is provided with a storage device for temporarily storing the projected image and K-Space data captured and acquired by the photographing device. Depending on the type of the photographing device, the console for the photographing procedure and the console for the image processing procedure may be integrated or may be separate housings.

(Notification server)
The notification server transfers certain information to a set destination group, informs the related parties of the information, instructs the related parties to take an action, and executes a program of the transfer destination terminal. It is a target server equipped with a notification service function. The information is information composed of text, an image, program execution information necessary for executing a program of a transfer destination terminal, and the like.

The notification server is connected to, for example, a related network of a medical institution facility, and can transmit information to an electronic medical record terminal or a mobile information terminal such as a PHS or a smartphone owned by a staff member of the medical institution facility. For example, when the electronic medical record terminal receives text or image information from the notification server, the received text or image is displayed on the screen of the electronic medical record terminal in a style suitable for viewing. This display is executed by the client application of the electronic medical record system or other application that cooperates with the notification server. Further, when the electronic medical record terminal or the smartphone receives the program execution information from the notification server, the program available on the terminal is executed according to the program execution information. An example of the program is a diagnostic image reference viewer used to reference a diagnostic image. For example, if the program execution information includes information that can identify the test to be displayed on the diagnostic image reference viewer, the diagnostic image reference viewer is executed on the terminal and the diagnostic image related to the test is displayed. To do.

If the program execution information does not include information that specifies the program to be executed, the program is executed based on other information, or if the execution method is unknown, nothing is done. The other information is, for example, a file path, the format of the file pointed to by the file path is specified by an extension or the like, and the file path is given as a parameter to the program associated with the format on the terminal and executed. To do. Further, the other information is, for example, a Uniform Resource Locator (URL), and the URL is given as a parameter to a program such as a WEB browser installed on the terminal and executed.

Further, for example, when a smartphone receives information composed of text or an image from a notification server, an instant messenger, an e-mail client, an SMS (Short Message Service) client, or the like displays the information.

In addition, depending on the type of distribution destination terminal such as an electronic medical record terminal or a smartphone, the notification server is notified of opening information, which is information on whether or not the user has confirmed the information transmitted by the notification server. As an example of use, a case where a patient is urgently transported to a medical medical institution facility by an ambulance will be described with reference to FIG. First, in step S110, the emergency personnel in charge transmits information such as a memo and a photograph of the patient's condition and the arrival time at the transportation destination to the notification server to the doctor in charge at the transportation destination. Next, in step S120, the notification server transfers the information received in step S110 to the smartphone possessed by the doctor in charge, who is the set destination user. Next, in step S130, after a certain period of time has elapsed since step S120 was performed,
When the notification server receives the opening information from the smartphone of the doctor in charge, it can be considered that the doctor in charge has learned that the patient is being transported. If the opening information has not been received, in step S140, the notification server attempts to communicate the information by voice electronically synthesized to the PHS of the doctor in charge, which is set as the second destination. In the case of PHS, the transfer of the image may be skipped as the information of only the voice, and the voice may convey that the image has already been transferred to the smartphone in step S120. Further, the number of destinations set in step S110 may be plural, and the same procedure as in steps S120 to S140 is performed for each destination. Further, in step S110, as soon as a certain amount of biometric information is accumulated by the vital sign monitor equipped in the ambulance, the destination may be automatically set and the biometric information may be transmitted to the notification server. In addition, after the image acquired by the X-ray imaging device installed in the ambulance is transmitted to the medical medical institution facility of the transport destination during transportation by the ambulance, the PACS of the storage destination of the image is automatically automatically set. The destination may be set and the image may be transmitted to the notification server.

Hereinafter, embodiments of the medical image processing apparatus according to the present invention will be illustrated. In the following, for the sake of simplicity, the medical image processing device is simply referred to as an image processing device.

<First Embodiment>
Hereinafter, the image processing apparatus according to the first embodiment will be described with reference to FIGS. 1 and 9.

FIG. 1 shows an X-ray computed tomography apparatus 100 (hereinafter, simply referred to as an imaging apparatus 100) according to the present embodiment. The photographing device 100 includes a gantry 110 and a console 120. For example, the gantry 110 is installed in the CT examination room, and the console 120 is installed in the control room adjacent to the CT examination room. The gantry 110 and the console 120 are connected to each other so as to be able to communicate with each other. The gantry 110 is equipped with an imaging mechanism for X-ray CT imaging. The console 120 is a computer that controls the gantry 110. Further, the console 120 is connected to a PACS (not shown). An examiner (a camera operator or the like) can photograph an examinee and acquire a projected image by operating the console 120 via the input device 126 to give an instruction. The console 120 can acquire a diagnostic image by reconstructing the projected image, and the examiner can confirm the processing result of the console 120 by the output device.

As shown in FIG. 1, the gantry 110 includes a gantry control circuit 111, an X-ray generating unit 112, a rotating frame 113, an X-ray detecting unit 114, and a data collecting circuit 115. The gantry control circuit 111 controls the drive of the X-ray generation unit 112, the rotating frame 113, the X-ray detection unit 114, and the data collection circuit 115. The X-ray generating unit 112 shapes the X-rays generated by the X-ray tube with a collimator, and irradiates the shaped X-rays toward the rotating frame 113. The rotating frame 113 is a substantially cylindrical metal frame having an opening, and is rotatably supported in the gantry 110. Further, a sleeper is provided in the opening of the rotating frame 113, and the subject is placed on the sleeper.

The X-ray detection unit 114 detects X-rays emitted from the X-ray generation unit 112 to the subject placed on the bed of the rotating frame 113 and transmitted through the subject by the detection element. The detection element has a scintillator and an optical sensor. In the detection element, the scintillator converts the X-rays incident on the detection element into light having a photon amount corresponding to the incident X-ray dose, and the optical sensor amplifies the light and converts it into an electric signal. The data collection circuit 115 reads out and amplifies the electric signal corresponding to the X-ray dose detected by the X-ray detector 114, and integrates the amplified electric signal over the view period to digitally correspond to the X-ray dose. Generate subject projection data with values.

The console 120 is used by the examiner to operate the photographing device 100 or the gantry 110. The console 120 is a computer including an arithmetic processor, a main storage device, an auxiliary storage device, a communication interface, an output device 125, and an input device 126. When the arithmetic processor executes a computer program, it functions as an acquisition unit 121, a detection unit 122, a notification unit 123, and a reconstruction unit 124. The console 120 may be composed of a plurality of devices provided with some of these components. The output device 125 is, for example, a display, a light emitting device, and a speaker. The input device 126 is, for example, a mouse, a keyboard, and a touch panel. The console 120 corresponds to the medical image processing apparatus in this embodiment.

The acquisition unit 121 acquires a projected image (projection data) from the photographing device 100 via a circuit or a network while the photographing device 100 is photographing the subject or after the photographing is completed. The acquisition unit 121 corresponds to an acquisition means for acquiring measurement data from an imaging device 100 capable of acquiring a diagnostic image by reconstructing the measurement data.

The detection unit 122 includes an abnormality detection engine (injury / disease detection engine), and when a projected image is input, the detection unit 122 can detect the abnormality drawn on the projected image and determine the presence or absence of the abnormality. In the present embodiment, the detection unit 122 detects a specific injury or illness as an abnormality. The detection unit 122 corresponds to a detection means for detecting a specific injury or illness based on measurement data (projected image). In the present embodiment, the measurement data to be detected for abnormality is the projection data before the reconstruction, and the detection unit 122 detects a specific injury or illness from the projection data. In another embodiment, the measurement data to be detected for injury or illness is synogram data based on the projection data, and the detection unit 122 may detect a specific disease from the synogram data.

The anomaly detection engine utilizes a trained model trained with teacher data having the same configuration as the second teacher data. Therefore, the anomaly detection engine outputs a true value when a projected image in which a specific injury or illness is drawn is input, and outputs a false value when a projected image in which a specific injury or illness is not drawn is input. The true value is an example of the estimation result indicating that there is a specific injury or illness, and the false value is an example of the estimation result indicating that there is no specific injury or illness.

An abnormality is detected when the input data group constituting the teacher data comprehensively includes projection images having the same imaging conditions and image conditions as the projection image expected to be input to the detection unit 122. It is suitable because it may increase the accuracy of the operation. Specifically, by comprehensively covering the imaging conditions and image conditions included in the input data group, it is possible to stably detect injuries and illnesses even for input data whose accuracy is low with a conventional abnormality detection engine. The anomaly detection engine is trained so that. Therefore, the detection unit 122 can stably maintain high injury / illness detection accuracy for projected images under various conditions by using an abnormality detection engine including a trained model in which such training has been performed. it can.

In addition, the output data group constituting the teacher data is created by a specialist with reference to the input data group, is created by arbitrary injury / illness detection processing, or is created by modifying the output data separately created by the specialist. It can be prepared by doing something like that.

Of the pairs of input data and output data constituting the teacher data, the pair that does not contribute to the accuracy of the injury / illness detection process may be removed from the teacher data. For example, if the output data constituting the pair of teacher data is incorrect, the accuracy of injury / illness detection of the trained model trained using the teacher data is likely to decrease. Therefore, it may be possible to improve the accuracy of the trained model included in the anomaly detection engine by removing pairs with incorrect output data from the teacher data.

Further, in the description of the present embodiment, the projected image input to the abnormality detection engine is shown in FIG.
It will be described as a three-dimensional projection image (projection data) including a plurality of two-dimensional projection data as shown in the projection image Im210 of the above. However, depending on the embodiment, a two-dimensional or four-dimensional projected image may be used as long as the abnormality detection engine can handle it. Specifically, in the case of a two-dimensional projected image, the input projected image may be a two-dimensional projected image corresponding to the XY plane in the projected image Im210, or the XZ plane in the projected image Im210. It may be a two-dimensional projected image corresponding to.

Further, when the injury or illness is detected, the detection unit 122 may record the information that can identify the inspection related to the projected image in which the injury or illness is detected and the information that the injury or illness is detected. Specifically, the detection unit 122 records the above information in a recording device connected to a console 120 (not shown) or in an external system such as PACS connected to the console 120 (not shown) via a network. May be good.

Further, in the description of the present embodiment, the abnormality to be detected is regarded as an injury or illness (disease or trauma) that requires urgent treatment, such as pneumothorax, aortic dissection, retention of body fluid or blood, fracture, etc. As described above, depending on the embodiment, other abnormalities that are desired to be notified may be included.

The notification unit 123 can notify the examiner that an injury or illness has been detected by graphics, light, or sound via the output device 125. The notification unit 123 corresponds to a notification means for notifying that a specific injury or illness has been detected in response to the detection of the specific injury or illness by the detection unit 122. The output device 125 is, for example, a light emitting device such as a display, a speaker, or an LED light, and any other output device. When a touch panel display is used as the output device 125, the touch panel may also be used as the input device 126.

The reconstruction unit 124 generates a diagnostic image by performing a reconstruction process on the projected image. The setting information (image condition) for reconstruction may be input by the examiner or may be automatically determined according to the imaging condition.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. FIG. 9 is a flow chart showing the flow of the medical image processing method in the present embodiment. First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S910.

In step S910, the examiner operates the console 120, so that the imaging device 100 starts photographing the subject.

In step S920, the acquisition unit 121 acquires measurement data (projection image (projection data)) from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data. Specifically, the acquisition unit 121 acquires the projected image (projection data) acquired by the X-ray detection unit 114 from the data collection circuit 115 of the photographing apparatus 100.

In step S930, the detection unit 122 detects a specific disease based on the measurement data. Specifically, the detection unit 122 determines the presence or absence of a specific injury or illness with respect to the projected image acquired by the acquisition unit 121. Although not shown in particular for the sake of simplicity, when a projected image of imaging conditions for which injury / illness detection processing cannot be performed is input, a series of procedures of the present embodiment may be completed, or the notification unit 123 may be provided. The examiner may be notified of this via the output device provided.

In step S940, when it is determined in step S930 that there is a specific injury or illness, the process proceeds to step S950. Alternatively, when it is determined that there is no specific injury or illness, the series of procedures of the present embodiment is terminated.

In step S950, the notification unit 123 notifies that a specific injury or illness has been detected in response to the detection of the specific injury or illness. Specifically, in response to the detection of a specific injury or illness by the detection unit 122, the notification unit 123 notifies the examiner that the specific injury or illness has been detected via the output device 125 provided in the console 120.

In the present embodiment, since the abnormality is detected for the projected image (projection data), the abnormality can be detected without waiting for the completion of the reconstructing process of the diagnostic image. That is, the analysis result can be obtained quickly.

In the present embodiment, the detection of an abnormality based on the projected image and the notification of the abnormality can be completed before the reconstruction of the diagnostic image is completed. Therefore, the image processing apparatus of the present embodiment has an abnormality such as a disease, trauma, or poor imaging of the subject that can be visualized in the diagnostic image at a time earlier than the time when the image diagnosis or the conventional image analysis can be started. Can be detected and notified.

In addition, by notifying the medical staff that an abnormality has been detected, it is appropriate for the subject to perform re-imaging, additional imaging, another examination, surgical procedure, etc. before the subject becomes seriously ill. It is possible to encourage medical staff to take appropriate measures.

<Second embodiment>
Hereinafter, the image processing apparatus according to the second embodiment will be described with reference to FIGS. 1 and 9. The image processing apparatus according to the present embodiment detects anomalies not for a three-dimensional projected image but for a two-dimensional projected image.

FIG. 1 shows an example of a schematic configuration of an image processing apparatus according to the present embodiment. Hereinafter, the parts different from those of the first embodiment will be mainly described, and the description of the same configuration as that of the first embodiment will be omitted.

In the present embodiment, the detection unit 122 detects injuries and illnesses on a two-dimensional projected image. When the two-dimensional projection image is input, the abnormality detection engine included in the detection unit 122 outputs an inference result indicating whether or not the input two-dimensional projection image includes a specific injury or illness. In other words, the detection unit 122 in the present embodiment detects a specific injury or illness by using at least one of the plurality of two-dimensional projection data as a mass before the photographing device acquires all of the plurality of two-dimensional projection data. To do. The abnormality detection engine in the present embodiment is trained by teacher data (first teacher data) composed of a pair group in which a two-dimensional projected image is used as input data and an abnormality label corresponding to the image is used as output data. Have a trained model.

Next, a series of image processing according to the present embodiment will be described with reference to the flow chart of FIG. First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S910. Step S910 is the same as the first embodiment.

In step S920, the acquisition unit 121 acquires the projected image captured by the photographing device 100 from the photographing device 100 connected via a circuit or a network. The difference from the first embodiment is that the projected image is acquired even during the shooting of the subject. As shown in FIG. 2, for example, the image pickup apparatus 100 sequentially captures a two-dimensional projection image IM 220 which is a tomographic image of the XY plane, and acquires a three-dimensional projection image IM 210 when the imaging is completed. In the present embodiment, when the imaging device 100 captures the two-dimensional projected image IM 220, the acquisition unit 121 acquires the two-dimensional projected image from the imaging device 100.

In step S930, each time the acquisition unit 121 acquires a two-dimensional projected image, the detection unit 122 sequentially determines the presence or absence of a specific injury or illness on the two-dimensional projected image. Since injury or illness detection is performed each time a two-dimensional projected image is acquired, the two-dimensional projected image constituting the three-dimensional projected image IM210 is before the photographing device 100 acquires all of the three-dimensional projected image IM210. Injury and illness detection for IM220 is performed. Although not shown in particular for the sake of simplicity, when a projected image of imaging conditions for which injury / illness detection processing cannot be performed is input, a series of procedures of the present embodiment may be completed, or the notification unit 123 may be provided. The examiner may be notified of this via the output device provided.

In step S940 and step S950, the same procedure as in the first embodiment is carried out.

This embodiment can exert the same effect as that of the first embodiment. Further, by performing anomaly detection on a part of the two-dimensional projected images constituting the three-dimensional projected image, it is possible to prompt the medical staff to deal with it at an earlier point than the first embodiment. ..

<Third embodiment>
Hereinafter, the image processing apparatus according to the third embodiment will be described with reference to FIGS. 9 and 10. When a specific injury or illness is detected, the image processing device according to the present embodiment notifies the console and also instructs the notification server outside the console to notify the user.

FIG. 10 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the second embodiment, but the present embodiment may be based on the first embodiment. Hereinafter, the configuration not specifically stated is the same as that of the second embodiment.

The imaging system according to the present embodiment includes an X-ray computed tomography apparatus 1000 (imaging apparatus 1000) and a notification server 1030. The imaging device 1000 includes a gantry 1010 and a console 1020. Since the configuration of the gantry 1010 is the same as that of the gantry 110 of the first embodiment, the description thereof will be omitted. The gantry 1010 and the console 1020, and the console 1020 and the notification server 1030 are connected via a circuit or a network. Further, the gantry 1010 and the console 1020 may be directly connected, and the console 1020 and the notification server 1030 may also be directly connected. Although these devices are separate devices in the present embodiment, a part or all of these devices may be integrally configured. Further, these devices may be connected to any other device via a circuit or network, or may be integrally configured with any other device.

The examiner can take a picture of the subject and acquire a projected image by operating the console 1020 and giving an instruction. The console 1020 is used by the examiner to operate the gantry 1010. Further, as a configuration necessary for explaining the present embodiment, at least an acquisition unit 1021, a detection unit 1022, a notification unit 1023, and a reconstruction unit 1024 are provided. The console 1020 may be composed of a plurality of devices provided with some of these components.

The acquisition unit 1021, the detection unit 1022, the reconstruction unit 1024, the output device 1025, and the input device 1026 are the acquisition unit 121, the detection unit 122, the reconstruction unit 124, and the output device 125 in the first embodiment or the second embodiment. , The same as the input device 126. Therefore, detailed description will be omitted.

In addition to the same functions as the notification unit 123 in the second embodiment, the notification unit 1023 has a function of transmitting information related to notification (hereinafter, notification-related information) to the notification server 1030. The notification-related information is at least one of information such as information on the subject being photographed by the imaging device or an examination order number capable of identifying the examination action, and information indicating that an abnormality is depicted in the projected image. Including. Depending on the setting, the notification unit 1023 may add information regarding the type of abnormality to the notification-related information. Further, if a numerical value indicating the degree of abnormality is calculated, the notification unit 1023 may add this numerical value to the notification-related information.

The notification server 1030 can transfer the received information to the set destination group, inform the related parties of the received information, instruct the related parties to take an action, and execute the program of the transfer destination terminal. The destination is a destination group selected as a destination to which the received information should be transferred from the destination information group that can be referred to by the broadcast server 1030. The destination information group may be acquired from a database related to staff information provided in an electronic medical record system connected via a network (not shown), or the user may directly store the destination information (not shown) provided in the notification server 1030. May be prepared by inputting.

Further, the destination to be transferred may be a preset specified destination. The default destination is, for example, a destination of a doctor who is supposed to perform image diagnosis on a diagnostic image acquired by the imaging device 1000, a destination of a mailing list or the like representing a department to which a diagnostic imaging specialist belongs.

In addition, the destination to be transferred may be selected by using the information of the subject and the examination included in the received information. Specifically, if the received information includes the information of the subject, the notification server 1030 may add the doctor in charge of the subject to the destination.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 9, but a part of the description will be omitted because it is the same as the second embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S910. In step S910, the examiner operates the console 1020, and the imaging device 1000 starts photographing the subject.

In step S920, the acquisition unit 1021 acquires the projected image acquired by the X-ray detection unit of the photographing device 1000 from the data collection circuit of the photographing device 1000 connected via a circuit or a network. The acquisition unit 1021 acquires the two-dimensional projected image each time the two-dimensional projected image is acquired by the photographing apparatus 1000.

In step S930, each time the acquisition unit 1021 acquires a two-dimensional projected image, the detection unit 1022 sequentially determines the presence or absence of a specific disease on the two-dimensional projected image.

In step S940, when it is determined in step S930 that there is a specific injury or illness, the process proceeds to step S950. Alternatively, when it is determined that there is no specific injury or illness, the series of procedures of the present embodiment is terminated.

In step S950, the notification unit 1023 notifies the examiner that a specific injury or illness has been detected via the output device 1025 provided on the console 1020. Further, the notification unit 1023 transmits the notification-related information shown above to the notification server 1030 connected via a circuit or a network. When the notification server 1030 receives the notification-related information, the notification server 1030 notifies according to this information.

This embodiment can exert the same effect as that of the second embodiment. Further to mention further, by notifying the notification server, it is possible to inform the related parties other than the examiner that the abnormality has been detected, and it is possible to promptly start the preparation for dealing with the examinee.

<Fourth Embodiment>
Hereinafter, the image processing apparatus according to the fourth embodiment will be described with reference to FIGS. 10 and 11. The image processing apparatus according to the present embodiment also provides a provisional diagnostic image when notifying an abnormality so that an image interpreter can make a judgment.

FIG. 10 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the third embodiment, but the present embodiment may be based on the first or second embodiment. Hereinafter, the configuration not specifically stated is the same as that of the third embodiment.

In the present embodiment, when the detection unit 1022 detects a specific injury or illness in the console 1020, the reconstruction unit 1024 performs a reconstruction process on the projected image in which the specific injury or illness is detected. At this time, the reconstruction unit 1024 performs the reconstruction process according to predetermined image conditions. In the present specification, the reconstructed image obtained according to the specified image conditions is referred to as a tentative diagnostic image in order to distinguish it from the diagnostic image reconstructed under appropriate image conditions and registered in PACS. This defined image condition is a simple setting that prioritizes the processing speed of reconstruction as compared with the image condition when reconstructing an actual diagnostic image.

The detection unit 1022 detects injuries and illnesses on a temporary diagnostic image generated by the reconstruction. In general, reconstruction is a time-consuming process, but a provisional diagnostic image is reconstructed according to a specified image condition, and this specified image condition is a setting that prioritizes processing speed. As described above, since human judgment is not required to determine the image conditions and the load of the reconstruction process is small, a temporary diagnostic image can be generated in a relatively short time.

The notification unit 1023 has the following functions in addition to the same functions as the notification unit 1023 in the third embodiment. The notification unit 1023 has a function of adding a temporary diagnostic image to the notification-related information. Alternatively, the notification unit 1023 adds a function of saving the temporary diagnostic image in an image management system such as PACS so that the persons concerned can observe it, and information for referring to the temporary diagnostic image to the notification-related information. Has the function of

The image conditions used for reconstructing the diagnostic image are preferably set appropriately according to the inspection content, but when generating a temporary diagnostic image, the prescribed image conditions are adopted. The defined image conditions may be predetermined for each imaging condition, for example. Further, the temporary diagnostic image may be generated by an external device instead of being generated by the reconstruction unit 1024 or the console 1020.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 11, but a part of the description will be omitted because it is the same as the third embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1110. In steps S111-10 to S1140, the same procedure as in the third embodiment is carried out.

In step S1150, the reconstruction unit 1024 reconstructs the projected image in which a specific injury or illness is detected according to a predetermined image condition to generate a temporary diagnostic image. In step S1160, the notification unit 1023 notifies the examiner that a specific injury or illness has been detected together with a temporary diagnostic image via an output device provided in the console 1020. In addition, the notification unit 1023 generates notification-related information including a temporary diagnostic image and transmits it to the notification server 1030. The notification server 1030 performs notification together with a temporary diagnostic image according to the notification-related information.

This embodiment can exert the same effect as that of the third embodiment. Furthermore, since a temporary diagnostic image is added to the notification-related information in the present embodiment, the person concerned makes an image diagnosis as to whether or not an abnormality really exists on the terminal that has received the notification-related information. Can be judged. Further, since the generation of the temporary diagnostic image does not take much time as described above, the notification is not delayed by this.

<Fifth Embodiment>
Hereinafter, the image processing apparatus according to the fifth embodiment will be described with reference to FIGS. 11 and 12. In the fifth embodiment, the image processing device is mounted on the console of the photographing device, but the image processing device according to the present embodiment is mounted on a device different from the console, and this device performs abnormality detection and notification.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the fourth embodiment, but the present embodiment may be based on any one of the first to third embodiments. Hereinafter, the configuration not specifically stated is the same as that of the fourth embodiment.

The imaging system according to the present embodiment includes an X-ray computed tomography apparatus 1200 (imaging apparatus 1200), a detection apparatus 1240, and a notification server 1230. In this embodiment, the detection device 1240 corresponds to a medical image processing device. The photographing device 1200 and the detection device 1240, and the detection device 1240 and the notification server 1230 are connected via a circuit or a network. Further, the console 1220 and the detection device 1240 may be directly connected. Further, the detection device 1240 and the notification server 1230 may also be directly connected. Although these devices are separate devices in the present embodiment, a part or all of these devices may be integrally configured. Further, these devices may be connected to any other device via a circuit or network, or may be integrally configured with any other device.

The photographing apparatus 1200 includes a gantry 1210 and a console 1220. The gantry 1210 and console 1220 are similar to the gantry 1010 and console 1020 of the fourth embodiment. However, the console 1220 in this embodiment does not have to have the detection unit 1022 and the notification unit 1023.

The console 1220 is used by the examiner to operate the photographing apparatus 1200. Further, the console 1220 is provided with a storage device (not shown), and as soon as the projected image captured by the photographing device 1200 can be partially acquired, it is sequentially acquired and stored. In addition, the console 1220 stores the imaging conditions under which the examiner performed the imaging in association with the projected image.

The notification server 1230 is the same as the notification server 1030 of the fourth embodiment.

The detection device 1240 includes at least an acquisition unit 1241, a detection unit 1242, a notification unit 1243, a reconstruction unit 1244, an output device 1245, and an input device 1246 as configurations necessary for the description of the present embodiment. .. The detection device 1240 may be composed of a plurality of devices provided with some of these components.

The acquisition unit 1241 acquires the projected image stored in the console 1220. The detection unit 1242, the notification unit 1243, and the reconstruction unit 1244 are each the detection unit 1022 of the fourth embodiment.
, The same as the notification unit 1023 and the reconstruction unit 1024.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 11, but a part of the description will be omitted because it is the same as the fourth embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1110. In step S1110, the same procedure as in the fourth embodiment is carried out.

In step S1120, the acquisition unit 1241 acquires the projected image acquired by the imaging device 1200 during imaging, which is stored in the console 1220.

Steps S1130 to S1160 are the same as those in the fourth embodiment, except that the detection device 1240 performs processing instead of the console 1220.

This embodiment can exert the same effect as that of the fourth embodiment. Further, it is possible to reduce the processing load of the console by having a detection device other than the console take charge of the detection process. In addition, since the device can be installed later on the already installed photographing device and console, it is possible to flexibly respond to the demand for the abnormality detection and notification function and the convenience of the introduction cost.

<Sixth Embodiment>
Hereinafter, the image processing apparatus according to the sixth embodiment will be described with reference to FIGS. 12 and 13. The image processing apparatus according to the present embodiment converts the projected image into a temporary diagnostic image and performs abnormality detection on the temporary diagnostic image, instead of detecting the abnormality on the projected image.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the fifth embodiment, but the present embodiment may be based on any one of the first to fourth embodiments. Hereinafter, the configuration not specifically stated is the same as that of the fifth embodiment.

The abnormality detection engine provided in the detection unit 1242 and the detection unit 1242 is almost the same as the detection unit in the fifth embodiment, but the image to be detected is not a projection image but a diagnostic image. Is different. That is, the detection unit 1242 detects a specific injury or illness on the diagnostic image obtained by reconstructing the measurement data. The diagnostic image to be detected for injury or illness is obtained by reconstructing the measurement data according to predetermined image processing conditions.

In the present embodiment, when a projected image is input to the detection device 1240, the reconstruction unit 1244 first generates a temporary diagnostic image by reconstructing the input projected image under predetermined image conditions. Next, the detection unit 1242 determines the presence or absence of a specific injury or illness by detecting the specific injury or illness drawn on the temporary diagnostic image using the abnormality detection engine. The anomaly detection engine utilizes a learning model trained with teacher data (second teacher data) consisting of a pair of diagnostic images and anomaly labels. Therefore, the abnormality detection engine or the detection unit 1242 outputs a true value when a diagnostic image in which a specific injury or illness is drawn is input, and a false value when a diagnostic image in which a specific injury or illness is not drawn is input. Is output.

In the following description, the diagnostic image input to the trained model used by the abnormality detection engine will be described as a two-dimensional diagnostic image as shown in the diagnostic shadow image Im230 of FIG. However, depending on the embodiment, a three-dimensional or four-dimensional diagnostic image may be used as long as the abnormality detection engine can handle it. In that case, the detection unit 1242 waits for the acquisition unit 1241 to acquire a plurality of projected image groups, then constructs a three-dimensional or four-dimensional projected image and inputs it to the abnormality detection engine.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 13, but a part of the description will be omitted because it is the same as the fifth embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1310. The processing of steps S1310 to S1320 is the same as that of steps S110 to S1120 of the fifth embodiment.

In step S1330, the detection unit 1242 reconstructs the projected image to acquire a temporary diagnostic image. Step S1340 is the same as step S1130 of the fifth embodiment, except that the estimation result of injury / illness detection is output for a provisional diagnostic image instead of the projected image.

Steps S1350 to S1360 are the same as steps S1140 and S1160 of the fifth embodiment. During the notification process in step S1360, the notification unit 1243 outputs a temporary diagnostic image in which a specific injury or illness is detected from the output device 1245 and the notification server 1230 provided in the detection device 1240.

This embodiment can exert the same effect as that of the fifth embodiment. Further, in the present embodiment, since the projected image is reconstructed into a temporary diagnostic image and then the abnormality is detected, it is easy to apply the abnormality detection process developed for the conventional diagnostic image.

In addition, the diagnostic image used for the actual image diagnosis is reconstructed under various image conditions according to the settings of the examiner at each time, so that the image quality is not stable, and for the conventional abnormality detection process, It was the cause of unstable abnormality detection accuracy. However, according to the present embodiment, since the abnormality detection process is performed on the temporary diagnostic image reconstructed under the specified image conditions, the image quality is stable and the abnormality detection accuracy is also stable. Further, as the input data of the teacher data, it is sufficient to acquire a temporary diagnostic image reconstructed under the specified image conditions, and it is not necessary to acquire the temporary diagnostic image reconstructed under various image conditions. Creation of teacher data is also facilitated.

<7th Embodiment>
Hereinafter, the image processing apparatus according to the seventh embodiment will be described with reference to FIGS. 12 and 13. The image processing apparatus according to the present embodiment detects anomalies by using a segmentation process for tentative diagnostic images.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the sixth embodiment, but the present embodiment may be based on any one of the first to fifth embodiments. Hereinafter, the configuration not specifically stated is the same as that of the sixth embodiment.

In addition to the same functions as the notification unit 1243 in the sixth embodiment, the notification unit 1243 provides a temporary diagnostic image generated by the reconstruction unit 1244 and a region label image colored by a predetermined look-up table. Add to the information related to notification. Alternatively, the notification unit 1243 stores the temporary diagnostic image and the area label image in an image management system such as PACS so that the persons concerned can observe them, and adds information for referencing them to the notification-related information. to add.

The detection unit 1242 has an image segmentation function and an abnormality detection function. Detection unit 1
242 outputs a region label image capable of identifying an organ or the like and an abnormal region drawn on a temporary diagnostic image by region segmentation processing. The domain segmentation process is performed by a segmentation engine that includes a trained model according to a machine learning algorithm. The image segmentation engine accepts a diagnostic image as input and identifies an injured area included in the diagnostic image. The detection unit 1242 further calculates the area of the injured area in the area label image, and determines the presence or absence of a specific injury or illness depending on whether or not the area of the injured area exceeds a preset threshold value. Specifically, the detection unit 1242 determines that the diagnostic image has a specific disease when the area of the injured area included in the diagnostic image exceeds the threshold value. For example, when the area label field can identify the lung field and the pneumothorax area, and the value obtained by dividing the pneumothorax area by the lung field area exceeds 3%, it is determined that there is a specific injury or illness. Alternatively, the detection unit 1242 determines that there is a specific injury or illness when the area of the pneumothorax exceeds 2 square cm.

The abnormality detection engine or the image segmentation engine uses a trained model trained by the teacher data having the same configuration as the fourth teacher data. Further, the above-mentioned temporary diagnostic image may be a two-dimensional image or a three-dimensional image.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 13, but a part of the description will be omitted because it is the same as the sixth embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step 1310. The processing of steps S131 to S1330 is the same as that of the sixth embodiment.

In step S1340, the detection unit 1242 first performs image segmentation processing on the temporary diagnostic image to acquire an area label image, and then the area of the injured area in the area label image exceeds a preset threshold value. Abnormality is judged based on whether or not it is present.

In step S1360, when the notification unit 1243 notifies that a specific injury or illness has been detected, the notification unit 1243 sends the area label image in which the abnormality is detected and the temporary diagnostic image to the output device and the notification provided in the detection device 1240. Output from server 1230.

In the present embodiment, the image segmentation process is performed on the temporary diagnostic image, but the image segmentation process is performed on the projected image to detect a specific injury or illness according to the area of the obtained injury or illness area. May be good.

This embodiment can exert the same effect as that of the sixth embodiment. Further, in the present embodiment, the provisional diagnostic image and the area label image capable of identifying the area where the abnormality is drawn are included in the information related to the notification, so that a user who is not a specialist in image diagnosis can use the information. It is easy to judge the presence or absence of an abnormality even when referring to.

<8th Embodiment>
Hereinafter, the image processing apparatus according to the eighth embodiment will be described with reference to FIGS. 12 and 13. It is recommended that the image processing apparatus according to the present embodiment performs an inspection according to another imaging protocol when an abnormality is detected in the inspection according to one imaging protocol.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the sixth embodiment, but the present embodiment may be based on any one of the first to fifth and seventh embodiments. Hereinafter, the configuration not specifically stated is the same as that of the sixth embodiment.

The detection unit 1242 is the same as the abnormality detection engine of the sixth embodiment, but the input projected image is limited to the one taken by the non-contrast examination. In addition, the detection target is limited to diseases such as aortic dissection and hepatocellular carcinoma, in which contrast examination is usually performed together with non-contrast examination.

In addition to the same function as the notification unit 1243 in the sixth embodiment, the notification unit 1243 adds a text to the effect that it is recommended to add a contrast examination to the information related to the notification.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 13, but a part of the description will be omitted because it is the same as the sixth embodiment unless otherwise specified. First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1310. Steps S131-10 to S1350 are the same as those in the sixth embodiment.

In step S1360, the notification unit 1243 outputs a text indicating that a specific injury or illness has been detected and recommending the addition of a contrast examination via the output device provided in the console 1220 and the notification server 1230. ..

Here, an example is taken in which it is recommended to perform a contrast examination when a specific injury or illness is detected in the non-contrast examination, but the imaging protocol to be changed may be other than the use or absence of a contrast medium. Examples of imaging protocols for which changes are recommended include the imaging device model, the imaging range and position of the subject, the tube voltage, tube current, scan slice thickness, scanning time, and imaging time phase of the imaging device.

This embodiment can exert the same effect as that of the sixth embodiment. Furthermore, in the present embodiment, it is possible to detect a disease for which a contrast examination is generally performed, and by recommending the person concerned to perform a contrast examination, as a result, it is discovered only by the non-contrast examination. It is possible to prevent overlooking difficult diseases.

<9th embodiment>
Hereinafter, the image processing apparatus according to the ninth embodiment will be described with reference to FIGS. 12 and 13. It is recommended that the image processing apparatus according to the present embodiment detects an image artifact as an abnormality and re-photographs when the abnormality is detected.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described on the basis of the sixth embodiment, but the present embodiment may be based on any one of the first to fifth and seventh to eighth embodiments. Hereinafter, the configuration not specifically stated is the same as that of the sixth embodiment.

The detection unit 1242 is the same as the abnormality detection engine of the sixth embodiment, but limits the detection target to the image artifact. The abnormality detection engine included in the detection unit 1242 is learned according to teacher data (second teacher data) using a temporary diagnostic image as input data and an abnormality label indicating the presence or absence of an image artifact as output data. Therefore, the detection unit 1242 accepts the diagnostic image as an input and outputs the presence or absence of an image artifact in the diagnostic image.

The detection unit 1242 determines the presence or absence of an image artifact on a two-dimensional or three-dimensional projected image as in the first and second embodiments, instead of determining an abnormality on a temporary diagnostic image. You may.

In addition to the same function as the notification unit 1243 in the sixth embodiment, the notification unit 1243 adds a text to recommend re-shooting to the information related to the notification.

Next, a series of procedures according to the present embodiment will be described with reference to the flow charts of FIGS. 12 and 13, but a part of the description is omitted because the procedure is the same as that of the sixth embodiment unless otherwise specified. To do. First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1310. In steps S131 to S1350, the same procedure as in the sixth embodiment is carried out.

In step S1360, the notification unit 1243 outputs a text to the effect that an abnormality has been detected and that it is recommended to take a picture again because there is an image artifact via the output device provided in the console 1220.

In addition to the abnormality detection engine that detects image artifacts, the detection unit 1242 in the present embodiment also includes an abnormality detection engine that detects an injury or illness (disease or trauma) similar to the first, second, and seventh embodiments. , Injury and illness detection may be performed at the same time. If no image artifact is detected and a disease or trauma is detected, the notification unit 1243 does not recommend re-imaging.

This embodiment can exert the same effect as that of the sixth embodiment. Further, in the present embodiment, it is possible to detect an image artifact that interferes with the image diagnosis, and it is possible to take an image that can be normally image-diagnosed before the patient disappears due to returning home or the like. ..

<10th Embodiment>
Hereinafter, the image processing apparatus according to the tenth embodiment will be described with reference to FIGS. 11 and 12. The image processing apparatus according to the present embodiment has a plurality of abnormality detection engines, and the abnormality detection engine used for abnormality detection is changed according to the imaging conditions.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the fifth embodiment, but the present embodiment may be based on any one of the first to fourth and sixth to ninth embodiments. Hereinafter, the configuration not specifically stated is the same as that of the fifth embodiment.

In addition to the functions of the fifth embodiment, the acquisition unit 1241 acquires the projected image from the console 1220, and also acquires the imaging conditions of the projected image.

The detection unit 1242 includes a plurality of abnormality detection engines. These plurality of anomaly detection engines are trained using different teacher data. Specifically, the imaging conditions of the input data constituting the teacher data are different for each teacher data. Therefore, a certain abnormality detection engine has high abnormality detection accuracy when a projected image having the same imaging conditions as the imaging conditions of the teacher data related to the abnormality detection engine is input.

The detection unit 1242 accepts the imaging condition of the projected image as an input together with the projected image. The detection unit 1242 selects an abnormality detection engine that is expected to have the highest detection accuracy based on the input imaging conditions. That is, an anomaly detection engine trained with teacher data that includes input data for imaging conditions similar to the input imaging conditions is selected.

In addition, instead of detecting the projected image as a target, the detection unit 1242 generates a temporary diagnostic image by a reconstruction process using the specified image conditions as in the sixth embodiment, and the temporary diagnostic image is generated. Anomaly detection may be performed on the diagnostic image. In this case, the input data of the teacher data of the plurality of abnormality detection engines is a tentative diagnostic image obtained by reconstructing the projected images taken under different imaging conditions under the specified image conditions.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 11, but a part of the description will be omitted because it is the same as the fifth embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1110. Step S1110 is the same as the fifth embodiment. In step S1120, the acquisition unit 1241 acquires the projected image and the imaging conditions from the console 1220.

In step S1130, the detection unit 1242 detects a specific injury or illness. Specifically, the detection unit 1252 first selects an abnormality detection engine according to the imaging conditions of the projected image, and then specifies the projection image acquired by the acquisition unit 1241 using the selected abnormality detection engine. Determine if there is any injury or illness.

Steps S1140 to S1160 are the same as those in the fifth embodiment.

This embodiment can exert the same effect as that of the fifth embodiment. Further, in the present embodiment, by performing the optimum abnormality detection processing on the captured projected image, the abnormality detection processing with higher accuracy is performed.

<11th Embodiment>
Hereinafter, the image processing apparatus according to the eleventh embodiment will be described with reference to FIGS. 11 and 12. The image processing apparatus according to the present embodiment determines the abnormality detection engine to be used and the start timing of the abnormality detection processing according to the abnormality to be detected.

FIG. 12 shows an example of a schematic configuration of the image processing apparatus according to the present embodiment. Hereinafter, the present embodiment will be described based on the fifth embodiment, but the present embodiment may be based on any one of the first to fourth and sixth to tenth embodiments. Hereinafter, the configuration not specifically stated is the same as that of the fifth embodiment.

In the present embodiment, the acquisition unit 1241 also acquires the imaging conditions when the measurement data is acquired by the imaging device, and the detection unit 1242 starts the detection of a specific injury or illness at a timing according to the imaging conditions.

In addition to the functions of the fifth embodiment, the acquisition unit 1241 acquires the projected image from the console 1220 and at the same time acquires the imaging conditions of the projected image.

The detection unit 1242 is almost the same as the fifth embodiment, but inputs the imaging conditions together with the image. Further, the detection unit 1242 includes a plurality of abnormality detection engines. These plurality of abnormality detection engines have different types of injuries and illnesses to be detected. Further, the timing at which the detection unit 1242 starts the abnormality detection process differs depending on the imaging conditions.

As a specific example, a case where one of the abnormality detection engines is an abnormality detection engine that detects pneumothorax will be described. Since the abnormality detection engine performs detection processing on the pneumothorax drawn in the chest region, it is sufficient that the chest region is visualized in the input projected image. Therefore, for example, when the imaging range included in the imaging condition information is the whole body, the image is input at the timing when the image of the chest region can be acquired without waiting for the completion of the imaging of the whole body, and the pneumothorax The detection process may be started. If the imaging range does not include the chest region, it is not necessary to perform the detection process, and if the entire chest region is not included, the detection process is performed according to the design and settings of the abnormality detection engine. The process may or may not be performed.

Further, as another specific example, a case where one of the abnormality detection engines is an abnormality detection engine that detects aortic dissection will be described. Since the abnormality detection engine performs detection processing on the aorta drawn in the mediastinal region, it is sufficient that the mediastinal region is visualized in the input projected image. Therefore, for example, when the imaging range included in the imaging condition information is the whole body, the image is input at the timing when the image of the mediastinal region can be acquired without waiting for the completion of the imaging of the whole body, and the aorta is input. The dissection detection process may be started. If the shooting range does not include the mediastinal area, it is not necessary to perform the detection process, and if the entire mediastinal area is not included, depending on the design and settings of the abnormality detection engine. , The detection process may or may not be performed.

Further, as another specific example, when one of the abnormality detection engines is an abnormality detection engine that detects hepatocellular carcinoma for an image in which the imaging phase is the arterial phase in imaging using a contrast medium. Will be explained. Since the abnormality detection engine performs detection processing on the liver region visualized in the abdominal region, it is sufficient that the input projected image is the arterial phase and the liver region is visualized. Therefore, for example, in the imaging of the arterial phase, when the imaging range included in the imaging condition information is the whole body, the image is taken at the timing when the image of the liver region can be acquired without waiting for the completion of the imaging of the whole body. May start the detection process for hepatocellular carcinoma. If the arterial phase is not photographed or the liver region is not included in the imaging range, the detection process may not be performed. Further, in the imaging of the arterial phase, in a situation where the entire liver region is not included in the imaging range, the detection process may or may not be performed according to the design and setting of the abnormality detection engine.

As another specific example, one of the abnormality detection engines is imaging using a contrast medium, and hepatocellular carcinoma uses two projected images whose imaging phases are the arterial phase and the portal vein phase as input images. The case where the abnormality detection engine detects the above will be described. Since the abnormality detection engine performs detection processing on the liver region visualized in the abdominal region, at least the images of the arterial phase and the portal vein phase are included in the input image group, and the liver region is visualized. Is enough. Therefore, for example, in the imaging including the arterial phase and the portal vein phase, the image of the liver region of the two imaging phases can be acquired without waiting for the completion of the imaging of the entire imaging range (for example, the whole body). An image group may be input and the detection process for hepatocellular carcinoma may be started. If both the arterial phase and the portal vein phase cannot be obtained, or if the imaging range does not include the liver region, the detection process may not be performed. In addition, in the case of imaging including the arterial phase and portal vein phase, if the entire liver region is not included in the imaging range, the detection process may be performed according to the design and setting of the abnormality detection engine. It does not have to be.

Generally, the imaging conditions include information on the imaging range and the imaging phase of the contrast medium, but if the information is unreliable or missing, the imaging range and imaging will be taken separately from the input image. Image processing for estimating the time phase may be performed, and the result may be used as a shooting condition. For example, the shooting range may be estimated using a machine learning model that accepts an image as input data and outputs a shooting range. Alternatively, the shooting time phase may be estimated using a machine learning model that accepts an image as input data and outputs a shooting time phase. These machine learning models can be learned by a machine learning algorithm such as deep learning using teacher data having an image and a correct label of a shooting range or a shooting time phase as a pair.

The detection unit 1242 selects an abnormality detection engine that can correspond to the input imaging conditions from the plurality of abnormality detection engines. For example, when the information of the imaging range included in the imaging condition includes the chest, an abnormality detection engine for detecting pneumothorax and an abnormality detection engine for detecting aortic dissection are selected. It should be noted that a specific abnormality detection engine may be set so as not to be selected depending on the specialty and intention of the facility in which the medical image processing apparatus according to the present embodiment is introduced.

In addition, the detection unit 1242 generates a temporary diagnostic image by a reconstruction process using the specified image conditions as in the sixth embodiment instead of detecting the abnormality on the projected image, and the temporary diagnostic image is generated. Anomaly detection may be performed on the diagnostic image of. In this case, the input data of the teacher data of the plurality of abnormality detection engines is a tentative diagnostic image obtained by reconstructing the projected images taken under different imaging conditions under the specified image conditions.

Next, a series of procedures according to the present embodiment will be described with reference to the flow chart of FIG. 11, but a part of the description will be omitted because it is the same as the fifth embodiment unless otherwise specified.

First, when a series of procedures according to the present embodiment is started, the process first proceeds to step S1110. Step S1110 is the same as the fifth embodiment. In step S1120, the acquisition unit 1241 acquires the projected image and the imaging conditions from the console 1220.

In step S1130, the detection unit 1242 determines the abnormality detection engine used for detection and the timing at which the presence or absence of a specific injury or illness can be determined based on the imaging conditions acquired by the acquisition unit 1241. The detection unit 1242 uses the selected abnormality detection engine at the timing when the presence or absence of a specific injury or illness can be determined, and detects the injury or illness on the projected images acquired by the acquisition unit 1241 up to that point. carry out.

Steps S1140 to S1160 are the same as those in the fifth embodiment.

This embodiment can exert the same effect as that of the fifth embodiment. Further, in the present embodiment, more efficient abnormality detection processing is performed by selecting an abnormality detection engine that can handle the captured image and performing the processing when the abnormality can be detected. Will be done.

(Other Examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or device via a network or storage medium, and one or more processors in the computer of the system or device reads and executes the program. It is also possible to realize the processing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above embodiments. The present invention also includes inventions modified to the extent not contrary to the gist of the present invention, and inventions equivalent to the present invention. In addition, the above-described embodiments can be appropriately combined as long as they do not contradict the gist of the present invention.

For example, the abnormality detected by the detection unit is not limited to a specific injury or illness (disease and trauma), and may be any abnormality. Further, the detection unit may detect an abnormality in either the projected image or the temporary diagnostic image. Further, the projected image or the temporary diagnostic image of the abnormality detection target may be any of a two-dimensional image, a three-dimensional image, and a four-dimensional image. Further, the detection unit may have an engine that outputs whether or not the input data (projected image or a temporary diagnostic image) contains an abnormality, and an engine that outputs the probability that the input data contains the abnormality. Further, the detection unit may have an engine for extracting the abnormal region included in the input data, and may determine the abnormality according to the area of the extracted abnormal region.

In addition, at the time of notification, not only the fact that an abnormality is included is output, but also the probability that an abnormality is included, a temporary diagnostic image reconstructed according to a specified image condition, and at least an area label image from which an abnormal area is extracted. Either of them may be output together. Even in the embodiment in which the provisional diagnostic image or the area label image is not used at the time of abnormality detection, these data may be generated and notified.

Further, the detection unit may carry out a plurality of abnormality detection processes according to the above-described embodiment. For example, the detection unit has a plurality of anomaly detections based on a projected image, anomaly detection based on a tentative diagnostic image, anomaly detection based on a region label image of a tentative diagnostic image, and anomaly detection specialized for an image artifact. Anomaly detection may be performed. The content of the notification when an abnormality is detected may be changed according to what kind of abnormality is detected.

Further, although the embodiment of the X-ray CT apparatus is described above, the present invention is not limited to the X-ray CT apparatus, but any imaging apparatus that reconstructs raw data (measurement data) to acquire a diagnostic image. Applicable to (modality). For example, the present invention is also applicable to an MRI apparatus (magnetic resonance imaging apparatus), an ultrasonic diagnostic apparatus, a PET apparatus (positron emission tomography apparatus), and a SPECT apparatus (single photon emission tomography apparatus).

100, 1000, 1200 X-ray computed tomography device 120, 1020, 1240 Console 1240 Detection device 121, 1021, 1241 Acquisition unit 122, 1022, 1242 Detection unit 123, 1023, 1243 Notification unit

Claims (24)

An acquisition means for acquiring the measurement data from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data, and an acquisition means.
A detection means for detecting a specific injury or illness based on the measurement data,
In response to the detection of a specific injury or illness by the detection means, a notification means for notifying that a specific injury or illness has been detected, and a notification means.
A medical image processing device. The measurement data is projection data before reconstruction, and is
The detection means detects a specific injury or illness from the projection data.
The medical image processing apparatus according to claim 1. The measurement data is synogram data based on projection data, and is
The detection means detects a specific injury or illness from the synogram data.
The medical image processing apparatus according to claim 1. The measurement data is three-dimensional projection data including a plurality of two-dimensional projection data.
The detection means detects a specific injury or illness by targeting the three-dimensional projection data.
The medical image processing apparatus according to claim 1. The detection means detects a specific injury or illness by targeting at least one of the plurality of two-dimensional projection data before the photographing apparatus acquires all of the plurality of two-dimensional projection data.
The medical image processing apparatus according to claim 4. The detection means detects a specific injury or illness by targeting a diagnostic image obtained by reconstructing the measurement data.
The medical image processing apparatus according to claim 1. The diagnostic image is obtained by reconstructing the measurement data according to predetermined image processing conditions.
The medical image processing apparatus according to claim 6. The detection means includes a detection engine that accepts the diagnostic image as input and outputs an estimation result indicating whether or not a specific injury or illness is depicted in the diagnostic image.
The medical image processing apparatus according to claim 6. The detection means
It is equipped with an image segmentation engine that accepts the diagnostic image as input and identifies an abnormal region included in the diagnostic image.
When the area of the abnormal region included in the diagnostic image exceeds the threshold value, it is determined that the diagnostic image has a specific injury or illness.
The medical image processing apparatus according to claim 6. The acquisition means also acquires the imaging conditions when the measurement data is acquired by the photographing apparatus.
The detection means includes a plurality of abnormality detection engines, and detects a specific injury or illness by using the abnormality detection engine according to the imaging condition.
The medical image processing apparatus according to any one of claims 1 to 9. The acquisition means also acquires the imaging conditions when the measurement data is acquired by the photographing apparatus.
The detection means starts detection of a specific injury or illness at a timing corresponding to the imaging condition.
The medical image processing apparatus according to any one of claims 1 to 10. The detection means completes the detection of a specific injury or illness based on the measurement data before the imaging device completes the reconstruction of the diagnostic image.
The medical image processing apparatus according to any one of claims 1 to 11. With more display means
The notification means uses the display means to perform notification.
The medical image processing apparatus according to any one of claims 1 to 12. The medical image processing device is connected to the notification server so as to be communicable.
The notification means uses the notification server to perform notification.
The medical image processing apparatus according to any one of claims 1 to 13. The information notified by the notification means includes the fact that a specific injury or illness has been detected, and a diagnostic image obtained by reconstructing the measurement data according to specified image conditions.
The medical image processing apparatus according to any one of claims 1 to 14. The information notified by the notification means includes the fact that a specific injury or illness has been detected, and an image that identifies an abnormal region included in the diagnostic image obtained by reconstructing the measurement data according to specified image conditions. Including,
The medical image processing apparatus according to any one of claims 1 to 15. The information broadcast by the notification means includes recommendations for changing the imaging protocol.
The medical image processing apparatus according to any one of claims 1 to 16. The detection means includes an anomaly detection engine that detects image artifacts.
The notification means notifies that an image artifact has been detected and a recommendation for prompting re-imaging in response to the detection of the image artifact by the detection means.
The medical image processing apparatus according to any one of claims 1 to 17. The medical image processing apparatus is mounted on the same apparatus as the imaging apparatus.
The medical image processing apparatus according to any one of claims 1 to 18. The medical image processing device is mounted on a device different from the imaging device.
The medical image processing apparatus according to any one of claims 1 to 18. The imaging device is a tomography device.
The measurement data is two-dimensional projection data or three-dimensional projection data including a plurality of two-dimensional projection data.
The medical image processing apparatus according to any one of claims 1 to 20. X-ray generating means for irradiating X-rays and
X-ray detection means that detects X-rays that have passed through the subject and outputs them as measurement data,
The medical image processing apparatus according to any one of claims 1 to 21.
A tomography device equipped with. A medical image processing method performed by a computer
An acquisition step of acquiring the measurement data from an imaging device capable of acquiring a diagnostic image by reconstructing the measurement data, and
A detection step for detecting a specific injury or illness based on the measurement data,
A notification step for notifying that a specific injury or illness has been detected according to the detection of the specific injury or illness, and a notification step.
Medical image processing methods, including. A program for causing a computer to execute each step of the medical image processing method according to claim 23.

Images