JP2020062355A - Image processing apparatus, data generation apparatus, and program - Google Patents

Abstract

To improve the accuracy of determining a learned model.SOLUTION: An image processing apparatus according to an embodiment includes an extraction unit and a learning unit. The extraction unit extracts, from medical data for training, first data relating to an image of a first region which is a region inside a lesion, second data relating to an image of a second region which is a region around the lesion, and third data relating to an image of a third region which is a region outside the lesion. The learning unit generates a learned model for determining input medical data by using the first data, the second data, and the third data extracted by the extraction unit as training data.SELECTED DRAWING: Figure 1

Description

  Embodiments of the present invention relate to an image processing device, a data generation device, and a program.

  In fields such as so-called Computer Assisted Detection / Diagnostics, in which diagnosis is supported using a computer, for example, learning is performed by using a feature amount of a lesion known in advance as teacher data to generate a learned model. Moreover, in recent years, even if a human does not give the feature amount as prior knowledge by deep learning or reinforcement learning, the program self-learns the feature amount itself, and the human being labels (signs) the feature amount. A trained model may be generated by giving.

  A large amount of training data is generally required for learning in the process of generating a learned model that makes a determination based on the feature amount of a lesion. For example, when learning is performed using deep learning, medical data for generating a learned model may be required as training data, for example, about tens of thousands. However, it may be difficult to obtain a sufficient number of medical data for training. As a result, the determination accuracy of the learned model that has been learned may decrease.

JP, 2016-7270, A

  The problem to be solved by the present invention is to improve the determination accuracy of a learned model that has been learned.

  The image processing apparatus according to the embodiment includes an extraction unit and a learning unit. The extraction unit extracts, from the medical data for training, first data relating to the image of the first region which is the region inside the lesion, and second data relating to the image of the second region which is the region surrounding the lesion. , And third data relating to the image of the third region, which is a region outside the lesion. The learning unit generates a learned model that determines the input medical data by using the first data extracted by the extraction unit, the second data, and the third data as training data.

FIG. 1 is a diagram showing an image processing apparatus according to an embodiment. FIG. 2 is a diagram illustrating processing performed by the image processing apparatus according to the embodiment. FIG. 3 is a flowchart illustrating a flow of image extraction processing performed by the image processing apparatus according to the embodiment. FIG. 4 is a diagram illustrating processing performed by the image processing apparatus according to the embodiment. FIG. 5 is a flowchart illustrating the flow of learning processing performed by the image processing apparatus according to the embodiment. FIG. 6 is a diagram illustrating processing performed by the image processing apparatus according to the embodiment. FIG. 7 is a flowchart illustrating a flow of processing for executing a learned model performed by the image processing apparatus according to the embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Here, components that are the same as each other are assigned the same reference numerals, and redundant description is omitted.

(Embodiment)
It is a block diagram which shows the image processing apparatus 100 which concerns on embodiment of FIG.

  The medical image diagnostic apparatus 10 is an apparatus that captures or images a subject and generates medical data. The medical image diagnostic apparatus 10 also transmits medical data generated by imaging the subject to the image processing apparatus 100. Here, the medical image diagnostic apparatus 10 is, for example, an X-ray CT (Computed Tomography) apparatus. In such a case, the medical image diagnostic apparatus 10 performs X-ray CT imaging on the subject to generate X-ray CT data or X-ray CT images. The medical image diagnostic apparatus 10 uses the generated X-ray CT data or X-ray CT image as input medical data to be a target for determination by the learned model, or medical data for training for generating the learned model. Is transmitted to the image processing apparatus 100. The image processing apparatus 100 uses the interface function 120a of the first processing circuit 120 or the interface function 130a of the second processing circuit 130, which will be described later, to transmit X-ray CT data or X-ray CT images from the medical image diagnostic apparatus 10. To get.

  The medical image diagnostic apparatus 10 may generate an X-ray CT image based on the X-ray CT data obtained by performing X-ray CT imaging on the subject, or the image processing apparatus 100 may An X-ray CT image may be generated based on the X-ray CT data acquired from the medical image diagnostic apparatus 10.

  The image processing apparatus 100 is connected to the medical image diagnostic apparatus 10 and generates a learned model, executes the learned model, and executes various image processes. The image processing apparatus 100 uses the medical data generated by the medical image diagnostic apparatus 10 as input medical data for which the learned model is to be determined, or as training medical data for generating the learned model, To use. The image processing apparatus 100 uses these input medical data and training medical data in the form of an input medical image and a training medical image, respectively.

  The image processing apparatus 100 includes a memory 132, an input device 134, a display 135, a first processing circuit 120, and a second processing circuit 130. The first processing circuit 120 includes an interface function 120a, a control function 120b, a generation function 120c, a training data creation function 1, and a learning function 2. The training data creation function 1 includes an extraction function 4, and more specifically, the extraction function 4 includes an abnormal area extraction function 4a, a boundary area extraction function 4b, and a normal area extraction function 4c. Further, the learning function 2 includes an abnormal area learning function 2a, a boundary area learning function 2b, and a normal area learning function 2c. The second processing circuit 130 also includes an interface function 130a, a control function 130b, a generation function 130c, and a determination function 3.

  In the embodiment, the interface function 120a, the control function 120b, the generation function 120c, the training data creation function 1 (including the extraction function 4, more specifically, the abnormal area extraction function 4a, the boundary area extraction function 4b, and the normal area extraction function are included. 4c), learning function 2 (including abnormal area learning function 2a, boundary area learning function 2b, normal area learning function 2c), interface function 130a, control function 130b, generation function 130c, and determination function 3. The processing function and the learned model are stored in the memory 132 in the form of a program executable by a computer. The first processing circuit 120 and the second processing circuit 130 are processors that read a program from the memory 132 and execute the program to realize a function corresponding to each program. In other words, the first processing circuit 120 in the state where each program is read out has each function shown in the first processing circuit 120 of FIG. Further, the second processing circuit 130 in the state where each program is read out has the respective functions shown in the second processing circuit 130 of FIG. Further, when the second processing circuit 130 in the state where the program corresponding to the learned model is read out, the processing according to the learned model is performed. Note that, in FIG. 1, the first processing circuit 120 and the second processing circuit 1330 are described as having these processing functions realized by a single processing circuit, respectively. The first processing circuit 120 and the second processing circuit 130 may be configured in combination to realize the function by each processor executing a program. In other words, each of the above functions may be configured as a program, and one processing circuit may execute each program. Further, both the first processing circuit 120 and the second processing circuit 130 may be realized by a single processing circuit. As another example, a specific function may be implemented in a dedicated independent program execution circuit. In FIG. 1, the extraction function 4, the learning function 2, the determination function 3, the interface functions 120a and 130a, the control functions 120b and 130b, and the generation functions 120c and 130c are an extraction unit, a learning unit, a determination unit, and a reception unit, respectively. It is an example of a control unit and an image generation unit.

  The word "processor" used in the above description is, for example, a CPU (Central Processing Unit), a GPU (Graphical Processing Unit), or an application-specific integrated circuit (ASIC), a programmable logic device (for example, simple logic device). Programmable logic device (SPLD), complex programmable logic device (Complex Programmable Logic Device: CPLD), field programmable gate array (Field Programmable Gate Array: FPGA), and the like. Means. The processor realizes the function by reading and executing the program stored in the memory 132.

  Further, instead of storing the program in the memory 132, the program may be directly incorporated in the circuit of the processor. In this case, the processor realizes the function by reading and executing the program incorporated in the circuit. Therefore, for example, instead of storing the learned model in the memory 132, the program related to the learned model may be directly incorporated in the circuit of the processor.

  The first processing circuit 120 is a processing circuit mainly responsible for creating training data and generating a learned model by learning using the training data, and the second processing circuit 130 generates It is a processing circuit mainly having a function of applying the learned model to the data to be classified and performing the classification.

  The first processing circuit 120 uses the training data creation function 1 to generate training data for performing learning by the learning function 2 described below based on the data acquired from the medical image diagnostic apparatus 10. The first processing circuit 120 performs learning using the learning function 2 on the training data generated by the training data creation function 1 to generate a learned model. In addition, the second processing tentative 130 performs discrimination on the generated learned model and makes a determination. These training data creation function 1, learning function 2 and determination function 3 will be described later.

  Further, the first processing circuit 120 transmits, to the medical image diagnostic apparatus 10, information related to control of imaging of the medical image diagnostic apparatus 10 related to acquisition of training data by the interface function 120a, and the medical image diagnostic apparatus. From 10, the training data obtained by photographing is acquired. Further, the first processing circuit 120 having the interface function 120a stores the received training data in the memory 132.

  Similarly, the second processing circuit 130 transmits, to the medical image diagnostic apparatus 10, information related to control of imaging of the medical image diagnostic apparatus 10 relating to acquisition of data to be distinguished by the interface function 130a, and the medical image diagnostic apparatus 10. From the diagnostic device 10, the data of the discrimination target obtained by photographing is acquired. The first processing circuit 120 has the interface function 120a. The first processing circuit 120 stores the received discrimination target data in the memory 132.

  The first processing circuit 120 is controlled by the control function 120b, and the second processing circuit 130 is controlled by the control function 130b to perform overall control of the medical image diagnostic apparatus 10 and the image processing apparatus 100 to perform imaging and image generation. , Control the display of images, etc.

  Further, the first processing circuit 120 uses the generation function 120c to generate a medical image for training based on the training data, or performs various image processing on the generated medical image for training. . Similarly, the second processing circuit 130 uses the generation function 120c to generate a medical image to be classified based on the data to be classified, or to perform various image processes on the generated medical image to be classified. I do.

  The memory 132 stores various data such as the training data and the discrimination target data described above. The memory 132 is a RAM (Random Access Memory), a semiconductor memory device such as a flash memory, a hard disk, an optical disk, or the like.

  The input device 134 receives various instructions and information inputs from the operator. The input device 134 is, for example, a pointing device such as a mouse or a trackball, a selection device such as a mode changeover switch, or an input device such as a keyboard. Under the control of the control function 133, the display 135 displays a GUI (Graphical User Interface) for receiving the input of the imaging condition, an image generated by the generation function 120c or 130c, and the like. The display 135 is, for example, a display device such as a liquid crystal display. The display 135 is an example of a display unit.

  Next, the background of the embodiment will be briefly described.

  In fields such as so-called Computer Assisted Detection / Diagnostics, in which diagnosis is supported using a computer, for example, learning is performed by using a feature amount of a lesion known in advance as teacher data to generate a learned model. Also, in recent years, by deep learning and reinforcement learning, even if the human does not give the feature amount as prior knowledge, the program self-learns the feature amount itself and labels the feature amount with a label (symbol). It may be given by a human to generate a trained model.

  A large amount of training data is generally required for the learning performed in the process of generating the learned model that makes a determination based on the feature amount of the lesion. For example, when learning is performed using deep learning, medical data for generating a learned model may be required as training data, for example, about tens of thousands. However, it may be difficult to obtain a sufficient number of medical data for training. For example, in lung cancer and the like, there are only about several thousand clinically available clinical data, and this is not always sufficient when performing learning using deep learning. As a result, the generated learned model may not have sufficient discrimination accuracy.

  In view of this background, the image processing apparatus 100 according to the embodiment has the first processing circuit 120 and the second processing circuit 130. The first processing circuit 120 uses the extraction function 4 to extract, from the medical data for training, the first data relating to the image of the first area (abnormal area) that is the area inside the lesion and the area around the lesion. The second data related to the image of the second area (boundary area) and the third data related to the image of the third area (normal area) that is the area outside the lesion are extracted. In addition, the first processing circuit 120 uses the learning function 2 to calculate the learned model for determining the input medical data by using the first data extracted by the extraction function 4, the second data, and the third data. And the data as training data.

  That is, in the embodiment, the first processing circuit 120 uses the extraction function 4 to generate the first data, which is the data regarding the first region (abnormal region) that is the region inside the lesion, and the region around the lesion. The second data, which is data regarding the second region (boundary region), and the third data, which is data regarding the third region (normal region) that is a region outside the lesion, are extracted. An example of such a situation is shown in FIG. FIG. 2 is a diagram illustrating processing performed by the image processing apparatus according to the embodiment.

  In FIG. 2, the abnormal area 11 represents a first area that is a lesion in a certain organ. The boundary area 12 represents a second area that is an area that spreads around the abnormal area 11 that is a lesion. Further, since the normal regions 13a, 13b, 13c, 13d, 13e, and 13f are sufficiently separated from the abnormal region 11, they represent the third region that is considered to be a normal region outside the lesion. The difference between the normal area outside the lesion of a person who has a lesion and the normal area of a person who has no lesion is targeted at highly localized lesions such as solid cancer that is not a so-called systemic disease. In this case, since it is very small, in at least such a case, the third region, which is a region sufficiently separated from the abnormal region 11, can be treated as a normal region.

  In the example of FIG. 2, the first processing circuit 120 extracts the abnormal area 11, the boundary area 12, and the normal areas 13 a to 13 f by the extraction function 4. After that, the second processing circuit 130 performs learning by using the extracted abnormal region 11, boundary region 12, and normal regions 13a to 13f as training data, and generates a learned model.

  In the example of FIG. 2, firstly, the first processing circuit 120 extracts a plurality of normal regions 13a to 13f from one clinical data, and performs learning using these plurality of data. As a result, the first processing circuit 120 can generate a large number of effective training data with a small number of clinical cases, and the efficiency of learning can be improved.

  Secondly, the first processing circuit 120 extracts not only the abnormal region 11 but also the boundary region 12 that is a region that spreads around the abnormal region 11, and uses the data related to the extracted boundary region 12. Learn by doing. Further, the second processing circuit 130 distinguishes the lesion by using not only the information on the abnormal region 11 but also the information on the boundary region 12. As a result, the feature amount related to the boundary region 12 as well as the abnormal region 11 is used for the discrimination of the lesion, so that the accuracy of the determination performed by the second processing circuit 130 can be improved.

  The above will be described in more detail with reference to FIGS.

  First, a procedure for creating training data, which is performed by the first processing circuit 120 using the training data creation function 1, particularly the extraction function 4, will be described with reference to FIGS. 3 and 4.

  First, the first processing circuit 120 extracts the first region (abnormal region), which is the region inside the lesion, from the medical data for training acquired from the medical image diagnostic apparatus 10 by the extraction function 4 (step S100). ). Specifically, the first processing circuit 120 uses, for example, the medical image generated by the generation function 120c based on the medical data for training acquired from the medical image diagnostic apparatus 10 by the abnormal area extraction function 4a, for example, The contour of the lesion site is extracted using a contour extraction technique such as the Level set method, and the abnormal region 11 is extracted as the first region that is the region inside the lesion. Based on this processing, the first processing circuit 120 uses the abnormal area extraction function 4a to extract the first data relating to the image of the first area, which is the area inside the lesion, from the medical image for training.

  Then, the first processing circuit 120 uses the extraction function 4 to extract the second area, which is the area around the lesion, from the medical data for training acquired from the medical image diagnostic apparatus 10 (step S110). Specifically, the first processing circuit 120 uses, for example, the boundary area extraction function 4b for the medical image generated by the generation function 120c based on the medical data for training acquired from the medical image diagnostic apparatus 10, By performing the dilation process of morphological filtering, the boundary region 12 is extracted as the second region which is the region around the lesion.

  FIG. 4 shows an example of dilation processing of morphological filtering. FIG. 4 is a diagram illustrating processing performed by the image processing apparatus according to the embodiment.

  In FIG. 4, the first region 20 is a region that represents the region inside the lesion. The first area 20 is an area corresponding to the abnormal area 11 in FIG. The first processing circuit 120 uses the abnormal area extraction function 4a to set an area within a predetermined distance from the first area 20, that is, an area surrounded by the first area 20 and the dilation envelope 22 to the second area. As the area of. Here, the dilation envelop 22 is the center of the circle 21 when the circle 21 is moved around the first area 20 so that the circle 21 whose radius is the determined distance is circumscribed with the first area 20. It is the trajectory drawn by. Here, the range of the determined distance (expansion width of dilation) is a value set as a value according to the type of lesion. As an example, the first processing circuit 120 sets the determined distance to “X mm” when the lesion type is “liver cancer”, and sets the determined distance when the lesion type is “lung cancer”. Is set to “Y mm”. Note that, for example, when two cancers are adjacent to each other, the first processing circuit 120 may collectively treat them as one lesion peripheral region.

  Based on these processes, the first processing circuit 120 extracts the second data relating to the image of the second region which is the region around the lesion from the medical image for training by the boundary region extraction function 4b.

  Returning to FIG. 3, subsequently, the first processing circuit 120 extracts the third region, which is a region outside the lesion, from the medical data for training acquired from the medical image diagnostic apparatus 10 by the normal region extraction function 4c. (Step S120).

  Specifically, the first processing circuit 120 uses the normal area extraction function 4c to extract the contour of the organ itself to be determined using a contour extraction technique such as the Level set method. Then, the first processing circuit 120 uses the normal area extraction function 4c to set the area outside the first area extracted in step S100 and the second area extracted in step S110 as the area outside the lesion. It is extracted as a region of 3. At this time, the first processing circuit 120 differs from the target tissue to be trained when the third region (third data related to the image of the third region) is extracted by the normal region extraction function 4c. Excluding the tissue, extraction is performed so as to obtain a VOI (Volume Of Interest) of a certain size, for example. Here, examples of the tissue different from the tissue to be trained include a main blood vessel and a main tubular tissue such as a bronchus. As described above, the first processing circuit 120 improves the quality of the training data by removing the main blood vessels and the bronchial region itself, which may be affected by cancer, and their surroundings by the normal region extraction function 4c. Can be made. The first processing circuit 120 uses the normal area extraction function 4c to be an area that satisfies these conditions, for example, an area where a 5 mm sphere can be inscribed, and a plurality of non-overlapping spherical areas are normal areas. A plurality of third regions are extracted.

  Based on these processes, the first processing circuit 120 uses the normal region extraction function 4c to extract the third data related to the image of the third region, which is the region outside the lesion, from the medical image for training.

  In the above example, the case where the image to be extracted is the image of the patient having a lesion has been described. The first processing circuit 120 may use an image of a normal person having no lesion by the extraction function 4. In such a case, the first processing circuit 120 uses only the normal area extraction function 4c without using the abnormal area extraction function 4a and the boundary area extraction function 4b, and extracts only the third data related to the image of the third area. To do.

  Through the above steps S100 to S120, the first processing circuit 120 causes the extraction function 4 to generate the first data relating to the image of the first region which is the region inside the lesion and the second data which is the region surrounding the lesion. The second data relating to the image of the region of No. 3 and the third data relating to the image of the third region which is the region outside the lesion were extracted. The first processing circuit 120 subsequently performs learning on the extracted data by the learning function 2 to generate a learned model.

  Subsequently, a learning procedure performed by the first processing circuit 120 based on the data extracted by the extraction function 4 will be described with reference to FIGS. 5 and 6. FIG. 5 is a flowchart illustrating the flow of learning processing performed by the image processing apparatus 100 according to the embodiment.

  Here, as one example of performing learning, for example, the weighting of a learning target model whose weighting has not been adjusted is adjusted using training data, and a learned model whose weighting has been adjusted is generated. An example of such a learning target model is a neural network configured by gathering two or more nodes that return a constant output according to a weighting with respect to a constant input. Examples of such a neural network include, for example, a two-layer neural network and a multilayer neural network of three or more layers. As an example of a multilayer neural network used in image processing, for example, a convolutional neural network (CNN: Convolutional). Neural Network). Further, the learning of the neural network is performed, for example, on the supervised data by using the inverse error propagation method or the like.

  Further, as another example of performing learning, a feature amount can be extracted, but an identification method such as SVM (Support Vector Machine) or other classifier that does not perform labeling of the extracted feature amount is used. The machine learning method that was used was also included.

  Hereinafter, in the embodiment, a case of using deep learning will be described as an example of learning, but the embodiment is not limited to this, and the embodiment is similarly applicable to a method using SVM and the like.

  In FIG. 5, first, the first processing circuit 120 performs learning by the normal area learning function 2c using the third data relating to the image of the third area, which is an area outside the lesion, as training data, A learned model for the third region, which is a region outside the lesion, is generated (step S200). Specifically, for example, the first processing circuit 120 performs the labeling based on the third data extracted by the normal area extraction function 4c in step S120 of FIG. 3 by the training data creation function 1. Generate supervised data. The first processing circuit 120 performs learning using the supervised data generated based on the third data as the training data by the normal area learning function 2c, and in the area outside the lesion by, for example, the backward error propagation method. A trained model for a certain third region is generated.

  Similarly, the first processing circuit 120 uses the boundary area learning function 2b to perform learning by using the second data relating to the image of the second area, which is the area around the lesion, as the training data, and performs learning using the boundary area learning function 2b. A learned model for the second area, which is an area, is generated (step S210). Specifically, for example, the first processing circuit 120 performs the labeling based on the second data extracted using the boundary area extracting function 4b in step S110 of FIG. 3 by the training data creating function 1. Generate supervised data. The first processing circuit 120 performs learning by using the supervised data generated based on the second data as the training data by the boundary area learning function 2b, and in the area around the lesion by, for example, the backward error propagation method. A trained model for a second region is generated. In the case of performing deep learning, the first processing circuit 120 may perform learning by the boundary region learning function 2b based on the second data, in addition to this, using, for example, an automatic encoder.

  Further, the first processing circuit 120 performs learning by using the abnormal region learning function 2a using the first data relating to the image of the first region, which is the region inside the lesion, as the training data, and the region inside the lesion. A trained model for the first region is generated (step S220). Specifically, for example, the first processing circuit 120 performs labeling by the training data creation function 1 based on the first data extracted using the abnormal area extraction function 4a in step S100 of FIG. Generate supervised data. The first processing circuit 120 performs learning by using the supervised data generated based on the first data as training data by the abnormal area learning function 2a, and in an area inside the lesion by, for example, the backward error propagation method. A trained model for a certain first region is generated. In addition, when performing deep learning, the first processing circuit 120 may perform learning by the abnormal region learning function 2a based on the first data in addition to this, for example, using an auto encoder.

  Note that the first processing circuit 120 may perform learning by additionally using the data generated based on the second data and the third data as the training data when generating the learned model in step S220. good.

  Subsequently, the first processing circuit 120 integrates and aggregates the learned model generated in step S200, the learned model generated in step S210, and the learned model generated in step S220 by the learning function 2. Then, one learned model is generated (step S230). For example, the first processing circuit 120 linearly combines the learned model generated in step S200, the learned model generated in step S210, and the learned model generated in step S220 by the learning function 2, Generate one trained model.

  In this way, the first processing circuit 120 uses the first data, the second data, and the third data extracted by the extraction function 4 in steps S100 to S120 of FIG. 3 as training data. By using the learning function 2, learning is performed to generate a learned model that determines the input medical image that is the medical image to be distinguished.

  FIG. 6 shows an example of such processing. FIG. 6 is a diagram illustrating processing performed by the image processing apparatus 100 according to the embodiment.

  In FIG. 6, the data set 30 represents a data set extracted in the same case. The first processing circuit 120 uses the extraction function 4 to extract the third clinical data 31a, 31b, which is the data related to the normal region (region outside the lesion), from the single clinical data from the patient having the lesion. 31c, 31d, 31e, the second data 33 that is the data related to the boundary area (area around the lesion), and the first data 32 that is the data related to the abnormal area (area inside the lesion) are extracted. On the other hand, the first processing circuit 120 uses the normal region extraction function 4c to extract the third data 34a, which is data relating to the normal region (region outside the lesion), from the clinical data of the patient having no lesion. Only 34b, 34c, 34d and 34e are extracted.

  In step S200, the first processing circuit 120 performs learning by the normal area learning function 2c based on the third data 31a, 31b, 31c, 31d, 31e, 34a, 34b, 34c, 34d, 34e, and performs learning. The completed model 5c is generated. Further, in step S210, the first processing circuit 120 performs learning by the boundary function learning function 2b based on the second data 33, and generates the learned model 5b. In step S220, the first processing circuit 120 performs learning by the abnormal function learning function 2a based on the first data 32 and integrates the results with the learned models 5b and 5c to obtain the learned model 5a. To generate.

  Next, the execution of the learned model generated by the first processing circuit 120 based on the learning function 2 will be described. FIG. 7 is a flowchart illustrating the flow of execution of the learned model performed by the image processing apparatus according to the embodiment.

  The second processing circuit 130 uses the determination function 3 to make a determination (differentiation of lesion) for the input medical data based on the learned model generated in step S230 of FIG. 5 (step S300). For example, the second processing circuit 130 uses the determination function 3 to determine the presence / absence of a lesion in the image based on the learned model generated in step S230 of FIG. That is, the second processing circuit 130 uses the determination function 3 to determine whether the image related to the input medical data is a lesion image.

  In addition, the second processing circuit 130 may additionally perform other determination on the image related to the input medical data by the determination function 3. For example, when it is determined in step S300 that the image related to the input medical data is a lesion image, the second processing circuit 130 causes the determination function 3 to determine whether the lesion is a benign lesion, or It is also determined whether the lesion is malignant. In addition, the second processing circuit 130 may further specify the lesion area or calculate the likelihood of the lesion by the determination function 3.

  In addition, the second processing circuit 130 further calculates the feature amount of the image region used for the lesion classification by the determination function 3. For example, when the learned model is a neural network, the second processing circuit 130 uses the determination function 3 to calculate the feature amount of the image region used for the lesion classification by using a predetermined neuron model of the neural network. To do. Further, when performing machine learning using SVM, the second processing circuit 130 calculates the feature amount of the image region used for the lesion classification based on the predetermined eigenvector obtained using SVM.

  Regarding the calculated feature quantities, the second processing circuit 130 uses the determination function 3 to determine the calculated feature quantities for the input medical data according to, for example, the pathological characteristics of the lesion. It may be used as the auxiliary determination information. By using the auxiliary determination information, the second processing circuit 130 can further improve the accuracy of determination regarding whether or not the image related to the input medical data is a lesion image.

  As an example of using the auxiliary determination information, the second processing circuit 130 uses the determination function 3 to perform learning based on the learned model generated in step S230 of FIG. 5 and the positional relationship between the first region and the second region. Then, the input medical data may be determined. For example, the second processing circuit 130 uses the determination function 3 based on the learned model generated in step S230 of FIG. 5 and whether the first region is located so as to be surrounded by the second region. The input medical data may be determined. For example, the second processing circuit 130 uses the determination function 3 to determine that the larger the degree to which the first region is surrounded by the second region, the higher the possibility of the presence of the lesion.

  Further, the second processing circuit 130 may determine whether or not the image related to the input medical data is the image of the lesion, taking into consideration the information on the region around the lesion. For example, the second processing circuit 130 may use the determination function 3 to perform determination on the input medical data based on the feature amount and the learned model in the second region that is the region around the lesion. Further, for example, the second processing circuit 130 may perform texture analysis on a region around the lesion and make a determination on the input medical data based on the result of the texture analysis performed.

  Subsequently, the second processing circuit 130 causes the control function 130b to display the determination result performed by the determination function 3 in step S300 on the display 135 (step S310).

  Subsequently, the second processing circuit 130 causes the control function 130b to display the feature amount calculated by the determination function 3 in step S300 on the display 135 (step S320).

  The embodiment is not limited to the above example.

  In the embodiment, the case where the medical image diagnostic apparatus 10 is the X-ray CT apparatus has been described, but the embodiment is not limited to this.

  As an example, the medical image diagnostic apparatus 10 is, for example, a magnetic resonance imaging apparatus. In such a case, the medical image diagnostic apparatus 10 performs magnetic resonance imaging imaging on the subject to generate magnetic resonance imaging data or magnetic resonance imaging images. The medical image diagnostic apparatus 10 uses the generated magnetic resonance imaging data or magnetic resonance imaging image as an input medical image that is a target for which the learned model makes a determination, or as a training medical image for generating the learned model. Is transmitted to the image processing apparatus 100. The image processing apparatus 100 acquires the magnetic resonance imaging data or the magnetic resonance imaging image from the medical image diagnostic apparatus 10 by the interface function 120a or the interface function 130a.

  The medical image diagnostic apparatus 10 may generate the magnetic resonance imaging image based on the magnetic resonance imaging data obtained by performing the magnetic resonance imaging imaging, or the image processing apparatus 100 may cause the medical image diagnostic apparatus 10 to generate the image. A magnetic resonance imaging image may be generated by the generation function 120c or the generation function 130c based on the magnetic resonance imaging data acquired from.

  Further, the medical image diagnostic apparatus 10 may be other types of medical equipment such as an ultrasonic diagnostic apparatus, a nuclear medicine diagnostic apparatus such as a positron emission tomography apparatus, a digital X-ray diagnostic apparatus, or the like. In such a case, the medical image diagnostic apparatus 10 uses the data generated by imaging or imaging the subject as the input medical data to be the target of the learning model determination, or the training for generating the learned model. It is transmitted to the image processing apparatus 100 as medical data for medical use.

  In addition, the format of medical data represented by input medical data and medical data for training includes image data reconstructed based on volume data, image data reconstructed based on raw data, and image data. On the other hand, it includes image data in various formats such as image data further processed, for example, processed image data such as data in which edges of image data are emphasized.

  Further, the case where the image processing apparatus 100 acquires the medical data for training or the input medical data from the medical image diagnostic apparatus 10 has been described, but in the embodiment, the image processing apparatus 100 transmits the medical image diagnostic apparatus 10 from the medical image diagnostic apparatus 10. It is not limited to the case of directly acquiring the medical data for training or the input medical data. For example, the medical image diagnostic apparatus 10 transmits medical data to a storage device such as an image storage server that exists separately as training medical data or input medical data, and the medical data stored in the storage device is used as an image processing device. 100 may acquire. In this case, the medical data stored in the storage device is not limited to the one generated by the single medical image diagnostic device 10 and may be generated by a plurality of medical image diagnostic devices 10.

  Further, the input medical data to be distinguished is not limited to the image captured by the medical image diagnostic apparatus 10, and may be data such as a pathological image captured by a microscope by collecting cells of the subject.

  Further, in step S120 of FIG. 3, when the first processing circuit 120 extracts the third data of the third region or the image of the third region, the target tissue to be trained, such as blood vessels and bronchi. The case where the extraction is performed by excluding the tissue different from the above has been described, but the embodiment is not limited to this. For example, the first processing circuit 120 uses the extraction function 4 to perform training such as blood vessels and bronchus even when extracting the first data (first region) or the second data (second region). The extraction may be performed by excluding the tissue different from the target tissue.

  The lesion peripheral area is not limited to the area adjacent to the lesion area. The boundary region extraction function 4b causes the first processing circuit 120 to be located around the blood vessel or bronchus connected to the first region, which is the region inside the lesion, as the second region, which is the region surrounding the lesion. Areas of cells may be included.

  In addition, the user gives feedback to the determination made by the second processing circuit 130 by the determination function 3, and the first processing circuit 120 performs learning by the learning function 2 based on the feedback given by the user. The internal algorithm of the completed model may be updated. That is, the first processing circuit 120 may perform self-learning by continuing to update the learned model based on the feedback from the user by the learning function 2.

  Further, in the image processing apparatus 100 of the embodiment, the part excluding the learning function 2 may be configured as an independent data generation device. The data generating apparatus has the same function as the image processing apparatus 100 of the embodiment except for the learning function 2. For example, the data generation device includes an extraction unit and an output unit. The extraction unit extracts, from the medical data for training, first data relating to the image of the first region which is the region inside the lesion, and second data relating to the image of the second region which is the region surrounding the lesion. , And third data relating to the image of the third region, which is a region outside the lesion. The output unit outputs the first data, the second data, and the third data extracted by the extraction unit for use as training data of a learned model that makes a determination on the input medical data. Here, the extraction unit and the output unit respectively have the same configurations as the extraction function 4 and the control function 120b in FIG. 1, for example.

  In addition, the program installed in the data generation device includes, from the medical data for training, the first data relating to the image of the first region which is the region inside the lesion and the second region which is the region surrounding the lesion. The second data relating to the image and the third data relating to the image of the third region, which is the region outside the lesion, are extracted, and the extracted first data, second data, and third data are extracted. The computer is caused to execute the respective processes for outputting and to be used as the training data of the trained model for making a judgment on the input medical data.

  According to the image processing device of at least one embodiment described above, it is possible to generate a large number of training data to be used for learning, and thus it is possible to improve learning determination accuracy.

  Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the invention described in the claims and the equivalents thereof as well as included in the scope and the gist of the invention.

1 training data creation function 2 learning function 2a abnormal area learning function 2b boundary area learning function 2c normal area learning function 3 determination function 4 extraction function 4a abnormal area extraction function 4b boundary area extraction function 4c normal area extraction function 100 image processing device 120 First processing circuit 130 Second processing circuit

Claims (11)

From the medical data for training, the first data relating to the image of the first region which is the region inside the lesion, the second data relating to the image of the second region which is the region surrounding the lesion, and the external data of the lesion An extraction unit that extracts the third data relating to the image of the third region, which is the region,
A learning unit that generates a learned model that makes a determination regarding input medical data by using the first data extracted by the extraction unit, the second data, and the third data as training data. ,
With
Image processing device.   The image processing apparatus according to claim 1, further comprising a determination unit that performs the determination on the input medical data based on the learned model.   The image processing apparatus according to claim 2, wherein the determination unit makes the determination based on a feature amount in the second area.   The image processing apparatus according to claim 2, wherein the determination unit makes the determination based on a positional relationship between the first area and the second area.   The image processing apparatus according to claim 4, wherein the determination unit makes the determination based on whether the first region is located so as to be surrounded by the second region.   The image processing device according to claim 2, wherein the extraction unit excludes a tissue different from a tissue to be trained when extracting the third data.   The image processing apparatus according to claim 6, wherein the different tissue is a blood vessel or a bronchus.   The image processing apparatus according to claim 1, wherein the extraction unit extracts, as the second area, an area within a range of a predetermined distance from the first area.   The image processing device according to claim 8, wherein the range of the distance is a value set as a value according to the type of lesion. From the medical data for training, the first data relating to the image of the first region which is the region inside the lesion, the second data relating to the image of the second region which is the region surrounding the lesion, and the external data of the lesion An extraction unit that extracts the third data relating to the image of the third region, which is the region,
An output that outputs the first data, the second data, and the third data extracted by the extraction unit for use as training data of a learned model that makes a determination on input medical data. And a section,
Data generator. From the medical data for training, the first data relating to the image of the first region which is the region inside the lesion, the second data relating to the image of the second region which is the region surrounding the lesion, and the external data of the lesion Extracting the third data relating to the image of the third region, which is a region,
Each process for outputting the extracted first data, the second data, and the third data for use as training data of a trained model that makes a determination on input medical data is output to a computer. The program to run.

Images