Classifications

• • A—HUMAN NECESSITIES
  • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
  • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
  • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment

Definitions

• This disclosure relates to an information processing device, an information processing method, and a storage medium.
• Patent Document 1 describes the output of pre-registered similar cases for the breast image to be diagnosed. Specifically, Patent Document 1 describes that first, breast images are associated with breast gland density information and feature amounts of lesion candidates detected from the breast images and stored in a database. Then, using the breast image to be diagnosed, breast gland density information and feature amounts of the lesion candidates are calculated, and breast images similar to the position information and/or breast gland density information of the lesion candidates in the diagnosis target are output from the database as similar cases.
• Patent Document 1 has a problem in that the diagnostic accuracy decreases for breast images with high breast density. This is because when the breast density is high, the breast is imaged overlapping with the lesion, which changes the imaging tendency of the lesion compared to when the breast density is low, and the feature values of the lesion candidate cannot be calculated sufficiently. Furthermore, not only due to the level of breast density, but also due to factors such as the quality of the captured image, the feature values of the lesion candidate cannot be calculated appropriately from the breast image, making it difficult to improve the diagnostic accuracy for breast images.
• the purpose of this disclosure is therefore to resolve the aforementioned issue of the difficulty in improving diagnostic accuracy for breast images.
• An information processing device includes: an estimation unit that estimates a state of mammary gland tissue contained in an input breast image; A setting unit that sets an image diagnosis method based on the estimated state; a diagnostic unit that performs image diagnosis on the breast image using the set image diagnosis method; Equipped with The structure is as follows.
• an information processing method includes: Estimating the state of mammary gland tissue contained in the input breast image; Setting an image diagnostic technique based on the estimated state; performing image diagnosis on the breast image using the set image diagnosis technique;
• the structure is as follows.
• a program includes: Estimating the state of mammary gland tissue contained in the input breast image; Setting an image diagnostic technique based on the estimated state; performing image diagnosis on the breast image using the set image diagnosis technique; Have a computer carry out the process,
• the structure is as follows.
• this disclosure can improve diagnostic accuracy for breast images.
• FIG. 1 is a block diagram showing a configuration of a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• FIG. 2 is a diagram showing a process performed by a first information processing device according to the present disclosure.
• 5 is a flowchart showing a processing operation of the first information processing device according to the present disclosure.
• 10 is a flowchart showing a processing operation of a second information processing device according to the present disclosure.
• FIG. 11 is a diagram showing a process performed by a third information processing device according to the present disclosure.
• 13 is a flowchart showing a processing operation of a third information processing device according to the present disclosure.
• FIG. 13 is a block diagram showing a configuration of a fourth information processing device according to the present disclosure.
• 13 is a flowchart showing a processing operation of a fourth information processing device according to the present disclosure.
• FIG. 13 is a block diagram showing a configuration of a fifth information processing device according to the present disclosure.
• 13 is a flowchart showing a processing operation of a fifth information processing device according to the present disclosure.
• FIG. 13 is a block diagram showing a hardware configuration of a sixth information processing device according to the present disclosure.
• FIG. 13 is a block diagram showing a configuration of a sixth information processing device according to the present disclosure.
• the information processing device can be used to perform image diagnosis on images of the human body, particularly breast images, captured using imaging devices such as mammography, breast tomosynthesis, PET (Positron Emission Tomography), and MRI (Magnetic Resonance Imaging).
• imaging devices such as mammography, breast tomosynthesis, PET (Positron Emission Tomography), and MRI (Magnetic Resonance Imaging).
• the information processing device 10 is composed of one or more information processing devices each having a calculation device and a storage device.
• the information processing device 10 is composed of an image acquisition unit 11, a state estimation unit 12, a diagnostic method setting unit 13, an image diagnosis unit 14, and a display processing unit 15.
• Each function of the image acquisition unit 11, the state estimation unit 12, the diagnostic method setting unit 13, the image diagnosis unit 14, and the display processing unit 15 can be realized by the calculation device executing a program for realizing each function stored in the storage device.
• the information processing device 10 is also composed of an image storage unit 16 and a model storage unit 17.
• the image storage unit 16 and the model storage unit 17 are composed of storage devices.
• the information processing device 10 is connected to an image capturing device 20 and a display device 30.
• the image capturing device 20 is, for example, an X-ray image capturing device that captures breast images.
• the image capturing device 20 is not limited to being an X-ray image capturing device, and may be an image capturing device that captures breast images using any principle.
• the display device 30 is a display that can display image data and text data, and displays diagnostic information to, for example, a diagnostician such as a doctor who performs image diagnosis, or a patient who has had a breast image captured.
• the display device 30 may display diagnostic information in any form. Each component will be described in detail below.
• the image acquisition unit 11 acquires breast images from the above-mentioned image capturing device 20 and stores them in the image storage unit 16. At this time, the image acquisition unit 11 stores the breast images in association with patient identification information (patient ID) that identifies the patient whose breast image was captured, and may further store the breast images in association with patient data such as the patient's attributes.
• patient ID patient identification information
• the image acquisition unit 11 may acquire one breast image for the same patient, or may acquire and store multiple breast images.
• the image acquisition unit 11 may also acquire breast images via a network from another information processing device that has stored breast images.
• the state estimation unit 12 performs image processing on the breast image to estimate the state of the mammary gland tissue contained in the breast image.
• the state of the breast tissue is estimated by estimating the degree to which normal mammary gland tissue is contained in the breast image, and the image is classified into classes according to the estimated degree to which normal mammary gland tissue is contained.
• the state of the mammary gland tissue can be indicated, for example, by the mammary gland density.
• Mammary gland density indicates the ratio when the area of the area in the imaged breast region where mammary gland tissue is thought to have originally been present is used as the denominator, and the area of the area actually occupied by mammary gland tissue is used as the numerator.
• the state estimation unit 12 performs classifying the breast density using, for example, a classifying model using a trained neural network or the like.
• the classification criteria can be a four-value classification based on BI-RADS, known as the international comprehensive guideline for breast imaging diagnosis, or a two-value classification that classifies breasts as dense or low density (fatty breasts).
• the state estimation unit 12 estimates whether the breast density is dense or low density.
• the class classification model using a neural network or the like is generated in advance by supervised learning in a format in which a breast image is input and a class classification label is output, and is stored in the model storage unit 17.
• the class classification of breast tissue state, that is, breast gland density, performed by the state estimation unit 12 is not limited to being performed using the model described above, and may be performed by any method.
• the estimation of the breast tissue state by the state estimation unit 12 is not necessarily limited to performing class classification of breast gland density, and the value of breast gland density may be estimated.
• the state estimation unit 12 estimates the mammary gland density as the state of the mammary gland tissue (hereinafter also referred to as the mammary gland state), but the state of the mammary gland tissue may be estimated using other indices.
• the state estimation unit 12 may use the degree of spread of the mammary gland as an index and estimate a numerical value representing the degree of spread of the mammary gland or a class classification based on the degree of spread.
• the degree of spread of the mammary gland may be, for example, a numerical value representing the degree to which the mammary gland is distributed uniformly within the breast or is distributed in agglomerated form in a specific area, or a classification according to such a numerical value.
• the state of the mammary gland tissue may be any information about the mammary gland tissue obtained from a breast image.
• the diagnostic method setting unit 13 sets an image diagnostic method for breast images based on the class of mammary gland condition classified by the above-mentioned condition estimation unit 12.
• a pre-set model diagnostic model that outputs diagnostic information in response to input of a breast image is used as the image diagnostic method, and the diagnostic method setting unit 13 selects the model to be used depending on the class.
• a number of models to be selected are stored in advance in the model storage unit 17.
• Each model corresponds to a class related to the state of the mammary gland, and has been trained in advance as a model that provides the best diagnostic accuracy for breast images classified into the corresponding class.
• the model has been trained to output the malignancy score, the type of lesion, and the corresponding location in the image (e.g., a rectangular display) as diagnostic information for an input breast image.
• the model may be one in which the parameters of the neural network are optimally trained by fine-tuning or the like using training data in which the malignancy score, the type of lesion, and the rectangular display of the image unit or the lesion candidate region unit that is the corresponding location in the breast image are associated as teacher data for breast images belonging to the corresponding class, or one in which the network structure itself is designed for each class, or one that combines a number of these models.
• the classes classified based on the mammary gland density are two values, high density and low density, so two models, a high density model and a low density model, corresponding to each of them, can be selected.
• the diagnostic method setting unit 13 is not limited to setting the image diagnostic method by selecting a model according to the class as described above, and may set the image diagnostic method by other methods.
• the diagnostic method setting unit 13 may set the image diagnostic method by setting an output rule in a model that outputs diagnostic information according to the input of a breast image according to the class.
• the output rule may be set by adjusting the threshold value with a function of a mammary gland condition class designed in advance.
• a specific function f(S) that calculates the malignancy score S may be set according to the mammary gland condition class.
• the diagnostic method set by the diagnostic method setting unit 13 described above is only an example, and any diagnostic method may be set according to the class.
• the image diagnosis unit 14 (diagnosis unit) performs diagnosis on the breast image using the image diagnosis method set by the diagnosis method setting unit 13.
• the image diagnosis unit 14 inputs the breast image into a model selected according to the class of breast density estimated from the breast image as described above, and obtains the information output from the model as the diagnosis result.
• the image diagnosis unit 14 obtains from the model diagnostic information such as the malignancy score, the type of lesion suspected to develop (mass, architectural distortion, local asymmetric shadow, calcification, etc.), the area of the lesion in the breast image, and further the malignancy score for each area, the properties of the lesion shape, etc., and passes this information to the display processing unit 15.
• the image diagnosis unit 14 also passes information such as the breast image and patient ID to the display processing unit 15.
• the display processing unit 15 outputs the diagnostic information etc. passed from the image diagnosis unit 14 to be displayed on the display device 30. For example, as shown in FIG. 2, the display processing unit 15 displays a breast image G, as well as the class "Breast Density: D” D1 classified as the estimated breast condition, "Diagnostic Mode: High (High Density Diagnostic Mode)” D2 as the selected image diagnosis method, "Malignancy Score: 90%” D3, and "Diagnostic Result: Mass” D4, which is the type of suspected lesion.
• the breast condition is classified into two classes, such as "high density” or "low density”
• a model corresponding to "high density” is selected as the image diagnosis method, and the selected model is displayed in the "Diagnostic Mode” D2 column. That is, in the example of FIG. 2, the character “high” is displayed in an emphasized manner, indicating that a model corresponding to "high density” has been selected.
• the display processing unit 15 may display on the breast image G the type of lesion suspected to develop, "mass” D4, the area of the lesion (dotted rectangle) A1, and may also display the characteristics of the lesion shape, etc., such as "Note: circular" D5.
• the display processing unit 15 may simultaneously display the diagnostic information of multiple patients, etc., passed from the image diagnosis unit 14 in list format, for example, as shown in FIG. 3. In this case, the display processing unit 15 does not necessarily have to display the breast images or areas corresponding to the lesions on the screen, as shown in FIG. 3. However, when displaying the diagnostic information of multiple patients, the display processing unit 15 may display the breast images or areas corresponding to the lesions on the screen.
• the image acquisition unit 11 acquires breast images from the image capturing device 20, an image storage medium, etc. (step S11 in FIG. 4).
• the acquired images are output to the condition estimation unit 12, the image diagnosis unit 14, and the display processing unit 15 for use, as described below.
• the state estimation unit 12 estimates the state of the mammary gland tissue contained in the breast image.
• the density values of the mammary gland tissue are classified (step S12 in FIG. 4).
• the state estimation unit 12 then outputs the estimation result of the mammary gland density to the diagnostic method setting unit 13, the image diagnosis unit 14, and the display processing unit 15.
• the diagnostic method setting unit 13 sets the image diagnostic method according to the class classification result of the mammary gland condition.
• a diagnostic model corresponding to the class is selected (step S13 in FIG. 4).
• it is assumed that a model corresponding to the high density class is selected.
• the image diagnosis unit 14 receives the set image diagnosis method, performs image diagnosis on the breast image using the image diagnosis method, and outputs the diagnosis result to the display processing unit 15.
• the breast image is input to a model corresponding to the high density class, and diagnostic information is output from the model (step S14 in FIG. 4).
• the display processing unit 15 outputs the diagnostic information from the image diagnosis unit 14 to the display device 30 (step S15 in FIG. 4).
• the display processing unit 15 outputs the estimated breast density class (breast density) D1, the model (diagnosis mode) D2 selected as the image diagnosis method, the malignancy score D3 which is the output result of the model, the diagnosis result D4, and the like, together with the breast image G.
• an image diagnosis method is set according to the state of the mammary gland tissue contained in the breast image, and image diagnosis is performed using this method. This makes it possible to apply an appropriate image diagnosis method to breasts in various states, such as mammary gland density, and to perform a highly accurate diagnosis.
• information indicating the image diagnostic method that was actually set is displayed and output together with the diagnostic result.
• the diagnostician can take measures such as immediately rejecting the diagnostic result obtained by the device, and the final diagnostic accuracy throughout the entire diagnostic process by the diagnostician can be improved.
• the information processing device 10 in this embodiment has a configuration almost similar to that of the information processing device described in the above-mentioned embodiment 1. That is, as shown in Fig. 1, the information processing device 10 includes an image acquisition unit 11, a state estimation unit 12, a diagnostic method setting unit 13, an image diagnosis unit 14, a display processing unit 15, an image storage unit 16, and a model storage unit 17.
• the information processing device 10 in this embodiment further differs from embodiment 1 in the configuration described below. Below, the configuration that differs from the above-mentioned embodiment 1 will be mainly described in detail.
• the state estimation unit 12 performs image processing on the breast image, and when estimating the state of the mammary gland tissue contained in the breast image, it particularly estimates an index related to the mammary gland state using continuous values.
• the state estimation unit 12 obtains a mammary gland density value by inputting the breast image into a regression model using a trained neural network or the like.
• a neural network is obtained by supervised learning using training data in which, for example, a breast image is input and continuous values between 0 and 1 are associated with the mammary gland density value.
• the state estimation unit 12 is not necessarily limited to outputting mammary gland density values, and may use other models to output continuous values representing other states of the mammary gland.
• the diagnostic method setting unit 13 receives the breast density value output from the state estimation unit 12 and sets the diagnostic method. For example, the diagnostic method setting unit 13 may classify the received breast density value from a threshold value set in advance, and set an image diagnostic method corresponding to the class classification, as in the first embodiment. The diagnostic method setting unit 13 may also set an image diagnostic method corresponding to the received breast density value. Alternatively, the diagnostic method setting unit 13 may set an output rule in a model that outputs diagnostic information in response to an input of a breast image, in accordance with the class, to set the image diagnostic method.
• the image diagnostic method may be set by setting the threshold value related to the malignancy score using a function of the breast density value defined in advance.
• the diagnostic method set by the diagnostic method setting unit 13 described above is only an example, and any diagnostic method may be set according to the breast density value.
• the image diagnosis unit 14 performs diagnosis on the breast image using the image diagnosis method set by the diagnostic method setting unit 13. At this time, the image diagnosis unit 14 inputs the breast image and the mammary gland density value to the neural network used as the model set as the image diagnosis method as described above, and obtains the information output from the neural network as the diagnosis result.
• the neural network is obtained by supervised learning so that it receives the mammary gland density value and the breast image as input and outputs the diagnosis result.
• the display processing unit 15 outputs the diagnostic information and the like passed from the image diagnosis unit 14 to be displayed on the display device 30.
• the display processing unit 15 may display the mammary gland density value estimated by the condition estimation unit 12 in addition to the diagnostic mode indicating the image diagnosis method set together with the breast image and the diagnostic result, as shown in FIG. 2.
• the image acquisition unit 11 acquires breast images from the image capturing device 20, an image storage medium, etc. (step S21 in FIG. 5).
• the acquired images are output to the condition estimation unit 12, the image diagnosis unit 14, and the display processing unit 15 for use, as described below.
• the state estimation unit 12 estimates the state of the mammary gland tissue contained in the breast image using the mammary gland density value, which is a continuous value (step S22 in FIG. 5). The state estimation unit 12 then outputs the mammary gland density value to the diagnostic technique setting unit 13, the image diagnosis unit 14, and the display processing unit 15.
• the diagnostic method setting unit 13 sets the image diagnostic method according to the breast gland density value.
• a diagnostic model corresponding to the breast gland density value is selected (step S23 in FIG. 5).
• a model corresponding to high density is selected based on the breast gland density value.
• the image diagnosis unit 14 receives the set image diagnosis method, performs image diagnosis on the breast image using the image diagnosis method, and outputs the diagnosis result to the display processing unit 15.
• the breast image and the breast density value are input to a model corresponding to high density, and diagnostic information is output from the model (step S24 in FIG. 5).
• the display processing unit 15 outputs the diagnostic information from the image diagnosis unit 14 to the display device 30 (step S15 in FIG. 4).
• the display processing unit 15 outputs the estimated breast density value, the model selected as the image diagnosis method, the malignancy score and the diagnostic result that are the output results of the model, etc., together with the breast image G.
• the mammary gland condition is estimated as a continuous value by regression, and an image diagnosis method is set according to this continuous value. This makes it possible to set an image diagnosis method that is more appropriate according to the mammary gland condition, and to perform a diagnosis with even higher accuracy.
• the information processing device 10 in this embodiment has a configuration similar to that of the information processing device described in the above-mentioned embodiment. That is, as shown in Fig. 1, the information processing device 10 includes an image acquisition unit 11, a state estimation unit 12, a diagnostic method setting unit 13, an image diagnosis unit 14, a display processing unit 15, an image storage unit 16, and a model storage unit 17.
• the information processing device 10 in this embodiment further differs from the other embodiments in the following configuration. Below, the configuration that differs from the other embodiments described above will be described in detail.
• the state estimation unit 12 performs image processing on the breast image in the same manner as described above, and estimates the mammary gland density value as the state of the mammary gland tissue in the entire breast image.
• the state estimation unit 12 estimates the local mammary gland state in the breast image.
• the state estimation unit 12 sets partial regions in the breast image in pixel units or in region units consisting of multiple adjacent pixels, and estimates the mammary gland density value for each partial region.
• the state estimation unit 12 generates a mammary gland density value map in which a mammary gland density value is set for each pixel of the breast image using a model having a supervised trained neural network that outputs mammary gland density values in pixel units.
• the mammary gland density value map is corrected for each pixel according to the mammary gland density value of the entire image, and is output as local mammary gland density information.
• the state estimation unit 12 is not limited to estimating local mammary gland density values, and may estimate any value that represents the local mammary gland state, such as the probability that a mammary gland has been present at that location.
• the threshold used for classifying the breast density class for each pixel may be set using a value fixed in advance, rather than using the breast density value of the entire image.
• the state estimation unit 12 is not limited to classifying partial areas such as pixels into binary classes, and may classify into more classes, or may calculate continuous values of the breast density value on a pixel-by-pixel or region-by-region basis.
• the above-mentioned processing by the state estimation unit 12 is merely an example, and the state of the breast may be locally estimated using other methods.
• the diagnostic method setting unit 13 receives input of the locally estimated mammary gland state as described above, and locally sets the diagnostic method. For example, when the mammary gland density is classified into multiple classes for each pixel or each region, the diagnostic method setting unit 13 sets an image diagnostic method for each pixel or each region that corresponds to the mammary gland density class of each pixel or each region. Note that the setting of the image diagnostic method according to the class classification is similar to the method described above. Alternatively, when the mammary gland density is estimated by a continuous value for each pixel or each region, the diagnostic method setting unit 13 may set an image diagnostic method according to the continuous value estimated for each pixel or each region.
• the image diagnosis unit 14 diagnoses the breast image based on the image diagnosis method set by the diagnostic method setting unit 13. For example, if the diagnostic method is set for each region in the diagnostic method setting unit 13, diagnosis is performed using the image diagnosis method set for each region. The diagnostic results calculated for each region are then aggregated for the entire image. When aggregating, the lesions detected in each region may be output together, or the malignancy scores calculated for each region may be added up and one malignancy score may be output for one breast image.
• the image diagnosis unit 14 may execute the diagnostic process according to the following method.
• the image diagnosis unit 14 detects candidate lesion regions using all of the lesion detection methods set in the diagnostic method setting unit 13. At this time, for example, detection is performed by rectangular display or segmentation using a supervised trained model. After that, for example, the mammary gland condition class that occupies the largest proportion of the set of pixels contained in the lesion region detected by the image diagnosis method corresponding to class A is regarded as the mammary gland condition class X of that region. At this time, if class X does not match class A, the detected lesion region is rejected and is not output as a diagnostic result.
• the image diagnostic unit 14 may input a breast image and a breast density value to a model set for each region, as described above, to output a diagnostic result.
• the image diagnostic unit 14 may input the spatial map itself having local breast information, and obtain a diagnostic result by inputting the above-mentioned spatial map to a supervised trained model that outputs a local diagnostic result.
• the above-mentioned diagnostic process by the image diagnostic unit 14 is one example, and the diagnostic process may be performed by any other method.
• the display processing unit 15 outputs the diagnostic information etc. passed from the image diagnosis unit 14 to be displayed on the display device 30.
• the display processing unit 15 displays the breast image G, as well as the breast density and the classified class "Breast density: 9.85 (D)" D1 as the estimated breast condition, the "diagnosis mode” D2 as the selected image diagnosis method, "malignancy score: 95%” D3, and the "diagnosis result: mass” D4 as the type of suspected lesion.
• the display processing unit 15 may define a color etc. according to local breast information (breast density) and highlight each pixel on the breast image G. For example, in the example of FIG.
• the display processing unit 15 may display "diagnosis modes" D21, D22, types of lesions suspected of developing D41, D42, and lesion areas A1, A2 for each region R1, R2, as shown in FIG. 6. In the example of FIG.
• a diagnostic mode D21 specialized for dense breasts is set, and it is displayed that the detected lesion is "tumor 1" D41, and for region R2 of the low breast density class, a diagnostic mode D22 specialized for dense breasts is set, and it is displayed that the detected lesion is "calcification 1" D42.
• the display method of the various information described above by the display processing unit 15 is one example, and other display methods may be used, and other information may be displayed.
• the image acquisition unit 11 acquires breast images from the image capturing device 20, an image storage medium, etc. (step S31 in FIG. 7).
• the acquired images are output to the condition estimation unit 12, the image diagnosis unit 14, and the display processing unit 15 for use, as described below.
• the state estimation unit 12 then infers the state of the mammary gland tissue contained in the breast image in a class classification format.
• the mammary gland density value is estimated for each pixel or region in the entire breast image, and the density values are classified into classes for each region (step S32 in FIG. 7).
• the density value of the mammary gland tissue is classified into the high density class out of the two-value classes of high density and low density.
• the state estimation unit 12 then outputs the inference result to the diagnostic method setting unit 13, the image diagnosis unit 14, and the display processing unit 15.
• the diagnostic method setting unit 13 sets an image diagnostic method according to the class classification result of the mammary gland condition.
• an image diagnostic method is set according to the class classified for each pixel or each region (step S33 in FIG. 7).
• a model corresponding to the high density class is set for the first region R1 in FIG. 6, and a model corresponding to the low density class is set for the second region R2.
• the image diagnosis unit 14 receives the image diagnosis method set for each region, performs image diagnosis on the breast image using the image diagnosis method set for each region, and outputs the diagnosis result to the display processing unit 15 (step S34 in FIG. 7).
• a breast image of the first region R1 is input to a model corresponding to the high density class
• a breast image of the second region R2 is input to a model corresponding to the low density class, to obtain diagnostic information for each region R1, R2.
• the image diagnosis unit 14 also aggregates the diagnostic results for each region R1, R2 (step S35 in FIG. 7). For example, among the lesions in each region R1, R2, only lesions set to be more serious are aggregated, or only lesions with a high probability according to the output diagnostic result are aggregated.
• the display processing unit 15 outputs the diagnostic information from the image diagnosis unit 14 to the display device 30 (step S36 in FIG. 7).
• the display processing unit 15 outputs the breast image G together with each region R1, R2, the class (color coding) of the breast density for each region, the model which is the diagnostic method for each region, the diagnostic result for each region, the overall malignancy score and diagnostic result, etc.
• diagnosis is performed by setting a diagnostic method for each region in a breast image. Therefore, even in breast images in which the breast density is not uniform, highly accurate diagnosis can be performed for each region. As a result, diagnostic accuracy can be further improved in response to variations in the location where a lesion appears.
• the information processing device 10 in this embodiment has a configuration similar to that of the information processing device described in the above-mentioned embodiment. That is, as shown in Fig. 8, the information processing device 10 includes an image acquisition unit 11, a state estimation unit 12, a diagnostic method setting unit 13, an image diagnosis unit 14, a display processing unit 15, an image storage unit 16, and a model storage unit 17.
• the information processing device 10 in this embodiment further includes an image correction unit 18 as shown in Fig. 8.
• the image correction unit 18 is realized by the arithmetic unit of the information processing device 10 executing a program.
• the image correction unit 18 performs correction processing on the breast images used for image diagnosis based on the estimated mammary gland state.
• the image correction unit 18 performs conversion processing on the breast images according to the class of mammary gland density estimated by the state estimation unit 12 as described above.
• the breast image conversion process here includes, for example, brightness correction using the Contrast Limited Adaptive Histogram Equalization (CLAHE) method, and image conversion using a deep learning generation model such as a generative adversarial network (GAN).
• the deep learning generation model is, for example, CycleGAN, which is trained to convert a high-density breast image into a low-density breast image when it is input.
• the image correction unit 18 may, for example, when the state estimation unit 12 classifies the breast image into two classes in terms of breast density, perform luminance correction using the CLAHE method for the class corresponding to high density, and perform identity conversion without correction for low-density breasts.
• the image correction unit 18 may perform image conversion using a deep learning generative model such as CycleGAN that has been appropriately trained for each class.
• the image correction unit 18 may also use the breast density value as an input for the model used for image conversion.
• the model used in the image correction unit 18 is, for example, trained to convert a high-density image and a breast density value into a low-density breast image as input.
• the image correction unit 18 may perform correction processing such as image conversion by any method.
• the image diagnosis unit 14 performs image diagnosis on the breast image corrected by the image correction unit 18, using the diagnostic method set by the diagnostic method setting unit 13.
• the model used during diagnosis can be a model trained using the image corrected by the image correction unit 18.
• the display processing unit 15 displays and outputs information such as breast images, estimated breast density, the set image diagnostic method, and diagnostic results, as shown in Figures 2 and 6. Furthermore, the display processing unit 15 may display breast images corrected by the image correction unit 18, and in this case, may display the corrected breast image together with the uncorrected breast image, or may display only the corrected breast image without displaying the uncorrected breast image.
• the image correction unit 18 may perform image conversion for each region of the breast image according to the mammary gland condition of that region, as described above. For example, the image correction unit 18 performs image correction processing for each region according to each mammary gland density class estimated for each region, and generates an image I(J) for each region corresponding to class J. The image diagnosis unit 14 then performs detection of lesion area candidates for the corrected image I(J) using the diagnostic method M(J) set for the mammary gland density class J for each region in the diagnostic method setting unit 13.
• This may be performed for all classes J, and if the representative mammary gland density class for a lesion area candidate detected in the image I(J) for each region does not match the estimated class J using the mammary gland density for each region, the lesion area may be rejected.
• the image acquisition unit 11 acquires breast images from the image capturing device 20, an image storage medium, etc. (step S41 in FIG. 9 ).
• the acquired images are output to the condition estimation unit 12, the image diagnosis unit 14, the display processing unit 15, and the image correction unit 18 for use, as described below.
• the state estimation unit 12 infers the state of the mammary gland tissue contained in the breast image.
• the density values of the mammary gland tissue are classified (step S42 in FIG. 9).
• the state estimation unit 12 then outputs the inference result to the diagnostic method setting unit 13, the image diagnosis unit 14, the display processing unit 15, and the image correction unit 18.
• the image correction unit 18 performs a correction process on the breast image according to the classified class (step S43 in FIG. 9). For example, the image correction unit 18 performs a luminance correction process on the breast image.
• the diagnostic method setting unit 13 sets the image diagnostic method according to the class classification result of the mammary gland condition.
• a diagnostic model corresponding to the class is selected (step S44 in FIG. 9).
• the image diagnosis unit 14 receives the set image diagnosis method, performs image diagnosis on the breast image corrected using that image diagnosis method, and outputs the diagnosis result to the display processing unit 15 (step S45 in FIG. 9).
• the display processing unit 15 outputs the diagnostic information from the image diagnosis unit 14 to the display device 30 (step S46 in FIG. 9). At this time, the display processing unit 15 outputs the estimated mammary gland density state, the model selected as the image diagnosis method, the malignancy score and diagnostic result that are the output results of the model, etc. together with the breast image G, as described above, and may also display and output the corrected breast image.
• breast images are corrected according to the state of the mammary glands, and image diagnosis is performed using the corrected breast images.
• the image correction unit 18 can improve the detection accuracy of lesions that overlap with mammary gland tissue by performing contrast enhancement processing using the CLAHE method only on high-density breasts.
• performing contrast enhancement processing using the CLAHE method on low-density breasts in which lesions are originally clearly imaged leads to a decrease in diagnostic accuracy due to information loss caused by image pre-processing.
• information related to mammary gland density estimated in advance it is possible to avoid a decrease in diagnostic accuracy due to excessive image pre-processing.
• by appropriately correcting the input breast image according to the mammary gland density in addition to the diagnostic method it is possible to perform image diagnosis with even higher accuracy for mammary gland density.
• the information processing device 10 in this embodiment has a configuration similar to that of the information processing device described in the above-mentioned embodiment. That is, as shown in FIG. 10, the information processing device 10 includes an image acquisition unit 11, a state estimation unit 12, a diagnostic method setting unit 13, an image diagnosis unit 14, a display processing unit 15, an image storage unit 16, and a model storage unit 17.
• the information processing device 10 in this embodiment further includes an update unit 19 as shown in FIG. 10.
• the update unit 19 is realized by the arithmetic unit of the information processing device 10 executing a program.
• an input device 40 capable of inputting information, such as a keyboard or a mouse, is connected to the information processing device 10.
• the configuration different from the other embodiments described above will be mainly described in detail.
• the display processing unit 15 mainly displays the breast image G, the estimated mammary gland condition D1 (mammary gland density and class), the set imaging diagnostic method D2 (diagnostic mode), the diagnostic result D4, etc. on the display device 30. At this time, the display processing unit 15 displays the estimated mammary gland condition D1 (mammary gland density and class) and the set imaging diagnostic method D2 (diagnostic mode) so that they can be modified.
• D1 mammary gland density and class
• D2 diagnostic method
• the update unit 19 receives correction information for the estimated mammary gland condition D1 (mammary gland density and class) and the set imaging diagnostic method D2 (diagnostic mode) input by the diagnostician from the input device 40 as described above, and updates the estimated mammary gland condition D1 (mammary gland density and class) and the set imaging diagnostic method D2 (diagnostic mode) to the correction information.
• the update unit 19 passes the input correction information to the corresponding processing unit. For example, when the mammary gland condition, ie, the mammary gland density or class, is corrected, the update unit 19 passes the correction information to the condition estimation unit 12, and when the imaging diagnostic method is corrected, the update unit 19 passes the correction information to the diagnostic method setting unit 13.
• the state estimation unit 12 When the state estimation unit 12 receives correction information of the mammary gland density or class, which is the mammary gland state, from the update unit 19, it estimates the state of the breast image to the mammary gland density and class corresponding to the correction information. In other words, when correction information of the mammary gland density is received, it assumes that it has estimated the mammary gland density corresponding to the correction information and performs class classification according to the mammary gland density, or when correction information of the class is received, it assumes that it has estimated the class corresponding to the correction information. As an example, even if the low density class was estimated before correction, if the correction information is a high density class, it estimates the class of the breast image to be the high density class. The state estimation unit 12 then passes the new mammary gland density and class based on the correction information to the diagnostic technique estimation unit 13 and the display processing unit 15.
• the diagnostic method setting unit 13 sets a new image diagnostic method corresponding to the new breast density or class. For example, the diagnostic method setting unit 13 selects a model corresponding to the new breast density or class, or sets output rules by the model to correspond to the new breast density or class. As an example, even if a low-density class model was selected before correction, when a new high-density class is estimated based on the correction information, the diagnostic method setting unit 13 selects a high-density class model as the image diagnostic method for the corresponding breast image.
• the diagnostic technique setting unit 13 when the diagnostic technique setting unit 13 receives correction information for the image diagnostic technique from the update unit 19, it sets the image diagnostic technique corresponding to the correction information. As an example, even if a low-density class model was selected before correction, if the correction information has corrected it to a high-density class, the diagnostic technique setting unit 13 selects a high-density class model as the image diagnostic technique for the corresponding breast image.
• the image diagnosis unit 14 performs diagnosis on the breast image using the image diagnosis method newly set by the diagnosis method setting unit 13 based on the modification information as described above. As an example, even if image diagnosis was performed using a low-density class model before modification, when a high-density class model is set by the above-mentioned modification, image diagnosis is performed on the corresponding breast image using the high-density class model.
• the display control unit 15 displays the mammary gland condition D1 (mammary gland density and class) and image diagnosis method D2 (diagnosis mode) based on the input correction information, or the image diagnosis method D2 newly set from the mammary gland condition D1 based on the correction information, and also displays the diagnosis result D4 based on the corrected image diagnosis.
• D1 mammary gland density and class
• D2 diagnosis mode
• the mammary gland concentration value which is the mammary gland state estimated by the state estimation unit 12
• a continuous value can be used as correction information.
• the mammary gland concentration value may be input in a seek bar format.
• the state estimation unit 12 estimates a class by classifying continuous values such as mammary gland concentration values
• the threshold value of the continuous value can be used as correction information. In this case, too, the threshold value may be input in a seek bar format, for example.
• the state estimation unit 12 estimates the mammary gland state for each region of the breast image, or when the diagnostic method setting unit 13 sets an image diagnostic method for each region of the breast image
• the mammary gland state (density value or class) or image diagnostic method (model or output rule) for each region of the breast image can be used as correction information.
• the corrected information is used to estimate the mammary gland state for each region, set the image diagnostic method, and perform image diagnosis and display output.
• the information processing device 10 is equipped with the image correction unit 18 described above, the breast image is corrected using a correction method that corresponds to the mammary gland state, such as the corrected mammary gland density value.
• the update unit 19 updates the mammary gland state and the image diagnosis method with the input correction information (step S57 in FIG. 11).
• the state estimation unit 12 then newly estimates the mammary gland density value and class that represent the state of the mammary gland tissue contained in the breast image based on the content of the correction information (step S52 in FIG. 11).
• the diagnostic method setting unit 13 sets the image diagnosis method based on the newly estimated mammary gland density value and class (step S53 in FIG. 11).
• the diagnostic method setting unit 13 newly sets the image diagnosis method for the breast image based on the content of the correction information (step S53 in FIG. 11).
• the imaging diagnostic unit 14 receives the newly set imaging diagnostic method and performs imaging diagnosis on the breast image using this imaging diagnostic method (step S54 in FIG. 11).
• the display processing unit 15 then outputs the diagnosis results from the imaging diagnostic unit 14, as well as information on the mammary gland condition and imaging diagnostic method updated by the correction information, to the display device 30 (step S55 in FIG. 11).
• such information can be corrected using correction information for the mammary gland state and image diagnosis method input by the diagnostician, etc.
• an appropriate image diagnosis method can be set through correction, making it possible to further improve the final accuracy of the image diagnosis.
• the information processing device 100 is configured as a typical information processing device, and is equipped with the following hardware configuration, as an example.
• ⁇ CPU Central Processing Unit
• ROM Read Only Memory
• RAM Random Access Memory
• Program group 104 loaded into RAM 103
• a storage device 105 for storing the program group 104
• a drive device 106 that reads and writes data from and to a storage medium 110 outside the information processing device.
• a communication interface 107 that connects to a communication network 111 outside the information processing device
• Input/output interface 108 for inputting and outputting data
• a bus 109 that connects each component
• FIG. 12 shows an example of the hardware configuration of the information processing device 100, and the hardware configuration of the information processing device is not limited to the above-mentioned case.
• the information processing device may be configured with a part of the above-mentioned configuration, such as not having the drive device 106.
• the information processing device may use a GPU (Graphic Processing Unit), a DSP (Digital Signal Processor), an MPU (Micro Processing Unit), an FPU (Floating point number Processing Unit), a PPU (Physics Processing Unit), a TPU (Tensor Processing Unit), a quantum processor, a microcontroller, or a combination of these.
• the information processing device 100 can be equipped with an estimation unit 121, a setting unit 122, and a diagnosis unit 123 shown in FIG. 13 by having the CPU 101 acquire and execute the group of programs 104.
• the group of programs 104 is stored in advance in the storage device 105 or the ROM 102, for example, and is loaded into the RAM 103 and executed by the CPU 101 as necessary.
• the group of programs 104 may be supplied to the CPU 101 via the communication network 111, or may be stored in advance in the storage medium 110, and the drive device 106 may read out the programs and supply them to the CPU 101.
• the estimation unit 121, setting unit 122, and diagnosis unit 123 described above may be constructed with dedicated electronic circuits for realizing such means.
• the estimation unit 121 estimates the state of the mammary gland tissue contained in the input breast image.
• the setting unit 122 sets an image diagnosis method based on the estimated state.
• the diagnosis unit 123 performs image diagnosis on the breast image using the set image diagnosis method.
• the present disclosure sets an image diagnostic method according to the state of the mammary gland tissue contained in the breast image, and performs image diagnosis using this image diagnostic method. This makes it possible to apply an appropriate image diagnostic method to breasts with various states of mammary gland density, etc., and to perform a highly accurate diagnosis.
• At least one of the functions of the estimation unit 121, the setting unit 122, and the diagnosis unit 123 described above may be executed by an information processing device installed and connected anywhere on the network, that is, they may be executed by so-called cloud computing.
• Non-transitory computer readable medium includes various types of tangible storage medium.
• Examples of non-transitory computer readable medium include magnetic recording media (e.g., flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (e.g., magneto-optical disks), CD-ROM (Read Only Memory), CD-R, CD-R/W, and semiconductor memory (e.g., mask ROM, PROM (Programmable ROM), EPROM (Erasable PROM), flash ROM, RAM (Random Access Memory)).
• the program may also be supplied to a computer by various types of transitory computer readable medium. Examples of transitory computer readable medium include electrical signals, optical signals, and electromagnetic waves.
• the temporary computer-readable medium can provide the program to the computer via a wired communication path, such as an electric wire or optical fiber, or via a wireless communication path.
• the setting unit selects a preset diagnostic model that outputs diagnostic information in response to input of the breast image based on the estimated state; the diagnostic unit uses the selected diagnostic model to perform image diagnosis based on the diagnostic information output in response to the input of the breast image.
• Information processing device (Appendix 3) 3.
• the information processing device according to claim 2 The setting unit selects at least one of the plurality of diagnostic models based on the estimated state, the diagnostic unit performs image diagnosis based on the diagnostic information output in response to the input of the breast image, using at least one of the selected diagnostic models; Information processing device. (Appendix 4) 2.
• the setting unit sets an output rule of a preset diagnostic model that outputs diagnostic information in response to an input of the breast image, based on the estimated state; the diagnosis unit uses the diagnostic model to perform image diagnosis based on the diagnostic information output according to the set output rule in response to the input of the breast image.
• Information processing device. (Appendix 5) 2.
• Information processing device (Appendix 6) 2.
• the information processing device a correction unit that performs a correction process on the breast image based on the estimated state, the diagnostic unit performs image diagnosis on the breast image that has been subjected to the correction process using the set image diagnosis method.
• Information processing device. (Appendix 7) 2. The information processing device according to claim 1, and an output unit that outputs the breast image, the estimated state, the set imaging diagnosis method, and the result of the imaging diagnosis.
• Information processing device. (Appendix 8) 6.
• the information processing device according to claim 5, an output unit that outputs the breast image, the partial regions set in the breast image, the state estimated for each of the partial regions, the image diagnosis method set for each of the partial regions, and a result of the image diagnosis, Information processing device. (Appendix 9) 7.
• the information processing device an output unit that outputs the breast image and/or the breast image after correction processing, the estimated state, the set imaging diagnosis method, and the result of the imaging diagnosis; Information processing device. (Appendix 10) 2.
• the information processing device according to claim 1, an input unit for receiving an input of the imaging diagnostic technique, the diagnosis unit executes image diagnosis on the breast image using the image diagnosis method that has been received as input; Information processing device. (Appendix 11) 6.
• the information processing device according to claim 5 an input unit that receives an input of the image diagnosis method for each of the partial regions of the breast image, the diagnosis unit performs image diagnosis on the partial region of the breast image by the image diagnosis method that has been input, for each partial region, and performs image diagnosis on the breast image.
• Information processing device .
• (Appendix 12) 2. The information processing device according to claim 1, an input unit for receiving an input of the state, The setting unit sets the image diagnosis method based on the state of the input received. Information processing device. (Appendix 13) 6. The information processing device according to claim 5, an input unit that receives an input of the state for each of the partial regions of the breast image, the setting unit sets the image diagnosis technique for each of the partial regions based on the state in which the input is received. Information processing device. (Appendix 14) 7. The information processing device according to claim 6, an input unit for receiving an input of the state, the correction unit performs a correction process on the breast image based on the state of input received. Information processing device. (Appendix 15) 2. The information processing device according to claim 1, The condition is breast density. Information processing device.
• the setting unit sets a model generated by machine learning based on the estimated state
• the diagnosis unit executes image diagnosis by inputting the breast image into the set model, and obtains decision-making information output from the model.
• Information processing device (Appendix 17) Estimating the state of mammary gland tissue contained in the input breast image; Setting an image diagnostic technique based on the estimated state; performing image diagnosis on the breast image using the set image diagnosis technique; Information processing methods. (Appendix 18) Estimating the state of mammary gland tissue contained in the input breast image; Setting an image diagnostic technique based on the estimated state; performing image diagnosis on the breast image using the set image diagnosis technique; A computer-readable storage medium that stores a program for causing a computer to execute a process.

Landscapes

• Health & Medical Sciences (AREA)
• Life Sciences & Earth Sciences (AREA)
• Medical Informatics (AREA)
• Engineering & Computer Science (AREA)
• Radiology & Medical Imaging (AREA)
• Biomedical Technology (AREA)
• Biophysics (AREA)
• Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
• Optics & Photonics (AREA)
• Pathology (AREA)
• Physics & Mathematics (AREA)
• High Energy & Nuclear Physics (AREA)
• Heart & Thoracic Surgery (AREA)
• Molecular Biology (AREA)
• Surgery (AREA)
• Animal Behavior & Ethology (AREA)
• General Health & Medical Sciences (AREA)
• Public Health (AREA)
• Veterinary Medicine (AREA)
• Apparatus For Radiation Diagnosis (AREA)

Priority Applications (2)

Applications Claiming Priority (1)

Publications (1)

Family

ID=94215300

Family Applications (1)

Country Status (2)

Citations (8)

• 2023
  • 2023-07-11 JP JP2025532299A patent/JPWO2025013213A1/ja active Pending
  • 2023-07-11 WO PCT/JP2023/025627 patent/WO2025013213A1/ja active Pending

Patent Citations (8)

Also Published As

Similar Documents

Legal Events