JP5655327B2 - Program and information processing apparatus - Google Patents

Description

  The present invention relates to a program and an information processing apparatus.

  Techniques that support diagnosis based on medical images are known. For example, Patent Literature 1 discloses an image storage communication system that preferentially displays an image having an abnormal shadow (feature). In the technique described in Patent Document 1, when a feature of an input image is detected, information indicating the feature is added to the input image, and an image to be displayed is selected based on the added information.

  In the technique described in Patent Document 2, a feature amount such as a lesion position and a lesion type in a captured patient image is detected, the image and the feature amount are stored in a database, and the patient image and the feature amount are stored from the database. Acquire and display the specified image, or create a provisional report including information about the feature amount acquired from the database and the image of the patient.

  In Patent Document 3, a database of images having a classified severity level for a disease or medical condition is created based on the designation of severity by a person, and a comparison between a patient image and an image in the database is made. Discloses a technique for determining the severity of a disease or medical condition.

  Patent Document 4 discloses a patient to be diagnosed by detecting an image region of an abnormal shadow candidate from a medical image obtained by photographing a patient, and using detection results of the abnormal shadow candidate obtained for a plurality of patients who are ill. A technique for calculating a predicted value indicating the possibility of future disease occurrence is disclosed.

Japanese Patent Laid-Open No. 05-205018 JP 2003-33327 A JP 2007-105461 A JP 2007-135683 A

  By the way, in medical diagnosis, it may be desired not only to identify the current symptom of a patient based on an image obtained by imaging the patient, but also to predict a change in the state of the subsequent imaging target.

  An object of the present invention is to provide a program and an information processing apparatus that assist in predicting a change in the state of an imaging target in an image obtained by imaging a patient.

The invention according to claim 1 is a feature amount storage that stores a feature amount representing a feature of a target image that is an image to be processed, and a feature amount representing each feature of a patient image captured in the past. Means for identifying the patient image similar to the target image using the feature amount of the target image and the feature amount of each of the patient images, and the patient photographed in the past The information indicating the change with time of the image specified in the specifying step is acquired from the change information storage means storing the information indicating the change with time of the patient image obtained using the image of using the information, the function representing the total change in the imaging target site, and a function representing the change in nonlinear lesion area, and generating an image conversion function, in said step of generating And outputting an image obtained by converting the target image using the transform function has been made, is a program for causing a computer to execute the.

The invention according to claim 2 is the invention according to claim 1, wherein the change information storage means includes a first image obtained by photographing the patient and a second image obtained by photographing the patient after the photographing time of the first image. A parameter for determining a conversion function to be converted into the first image, the parameter calculated using the first image and the second image, and the shooting time of the second image from the shooting time of the first image The feature amount storage means stores the feature amount of each of the first images, and in the specifying step, the first amount similar to the target image is stored. image is specified, in the step of generating the conversion function of the image, the first time before Symbol course a parameter associated with the image specified in the step of the identified associated parameters Obtained from the change information storing means, for generating a transform function of the target image using the obtained parameter.

  According to a third aspect of the present invention, in the second aspect of the invention, the feature amount of each of the target image and the first image is a feature of a lesion area determined to be a region including a lesion in the image. The parameter is a parameter that defines a conversion function for converting the image of the lesion area in the first image into the image of the lesion area in the second image.

  According to a fourth aspect of the present invention, in the invention according to the second or third aspect, among the images stored in the image storage means that stores the patient, the image obtained by photographing the patient, and the photographing time of the image in association with each other. At least one of the initial state images is taken as an initial state image, an image obtained by photographing the patient of the initial state image after the photographing time of the initial state image is obtained from the image storage means, and the obtained image and the initial state image are used. The initial state image is obtained from the image storage means in a step of calculating a parameter for determining a conversion function for converting the acquired initial image into the obtained image, and the initial state image is the first image. Using the image as the second image, the initial state image as the first image, the parameter calculated in the step of calculating the parameter, and the initial state And outputting to the change information storing means the elapsed time, the in association with each other from the photographing time of the image to the photographing time of image and said second image, further the computer to execute a.

The invention according to claim 5 is the invention according to any one of claims 2 4 from capturing timing of the target image, associated associated with the parameters used in the step of generating the conversion function of the image The parameter used in the step of obtaining a result image that is an image of a patient related to the target image when the elapsed time has passed and the step of generating a conversion function of the image is evaluated based on the result image Outputting the evaluation value obtained in association with the parameter to the change information storage means, and further causing the computer to execute the step of generating a conversion function of the image. The evaluation function is further used to generate a conversion function for the image.

According to a sixth aspect of the present invention, there is provided a calculation unit that calculates a feature amount that represents a feature of a target image that is an image to be processed, and a feature amount that stores a feature amount that represents each feature of a patient image captured in the past. Referring to storage means, specifying means for specifying the patient image similar to the target image using the feature quantity of the target image and the feature quantity of each of the patient images; Obtained from the change information storage means storing the information representing the change of the patient image over time, which was obtained using the patient image, and obtained the information representing the change over time of the image specified by the specifying means A generating unit that generates a conversion function of an image including a function that expresses a change in the whole region to be imaged using information and a function that expresses a non-linear change in a lesion area; and a conversion function generated by the generating unit The And output means for outputting an image obtained by converting the target image have an information processing apparatus, characterized in that it comprises a.

  According to the invention which concerns on Claim 1 or 6, it can assist estimating the change of the state of the imaging | photography object in the image which image | photographed the patient.

  According to the invention which concerns on Claim 2, the image which estimated the state of the imaging | photography object for every elapsed time can be shown to a user.

  According to the third aspect of the present invention, it is possible to assist in predicting a change in a lesion area over time in an image obtained by imaging a patient.

  According to the invention which concerns on Claim 4, the parameter showing the change of the image by progress of a certain time can be calculated and recorded from the image which image | photographed the patient.

  According to the invention according to claim 5, the parameter used for generating the image in which the state of the photographing target at a certain elapsed time is predicted is evaluated using the image actually taken when the elapsed time has passed. An image conversion function can be generated using the obtained evaluation value.

It is a functional block diagram which shows the example of a structure of a diagnostic assistance apparatus. It is a figure which shows the example of the data content of interpretation report DB. It is a figure which shows the example of the data content of feature-value DB. It is a figure for demonstrating the example of the procedure of the process which calculates | requires a deformation coefficient. It is a figure which shows the example of the data content of deformation | transformation model DB. It is a figure which shows the example of the data content of prediction image DB. It is a flowchart which shows the example of the procedure of a deformation | transformation model production | generation process. It is a flowchart which shows the example of the procedure of a predicted image generation process. It is a figure which shows the example of the deformation | transformation vector acquired from deformation | transformation model DB in a prediction image generation process. It is a flowchart which shows the example of the procedure of the process which updates various databases. It is a flowchart which shows the example of the procedure of the process which calculates | requires the weight of a deformation coefficient. It is a figure which shows the example of the hardware constitutions of a computer.

  FIG. 1 shows an example of the configuration of a diagnosis support apparatus 10 that functions as an information processing apparatus according to an embodiment of the present invention. The diagnosis support apparatus 10 includes an input reception unit 100, a past case DB (database) 102, an interpretation report DB 104, a feature amount calculation unit 106, a feature amount DB 108, a deformation model generation unit 110, a deformation model DB 112, a similar image search unit 114, and a prediction An image generation unit 116, a predicted image DB 118, and an output processing unit 120 are provided.

  The input receiving unit 100 receives an input representing a user instruction or the like to the diagnosis support apparatus 10. Input by the user is acquired by the input receiving unit 100 via an input device (not shown) such as a mouse or a keyboard. In addition, the input receiving unit 100 may receive an input of a medical image to be processed by the diagnosis support apparatus 10. A medical image is an image obtained by imaging a patient. For example, any of various images used for medical diagnosis such as an X-ray photograph, an MRI (Magnetic Resonance Imaging) image, or a CT (Computed Tomography) image. There may be. For example, the input reception unit 100 receives a medical image from another information processing apparatus connected to the diagnosis support apparatus 10 via a communication unit, or acquires a medical image from an apparatus that forms a medical image.

  The past disease case DB 102 is a database that stores image data of medical images taken in the past. The image data may be digital image data having a value such as hue, luminance, or brightness for each pixel. In the following description, the image data may be simply referred to as “image”. In the example of the present embodiment, it is assumed that an image ID that is identification information for identifying an image is assigned to each image stored in the past disease case DB 102. For example, information including the name of the image and the storage position of the image in the past case DB 102 may be used as the image ID.

  The interpretation report DB 104 is a database that stores information related to diagnosis for each image stored in the past case DB 102. FIG. 2 shows an example of the contents of the interpretation report DB 104. In the interpretation report DB 104 in the example of FIG. 2, the contents of each item of patient ID, disease ID, treatment ID, photographing date, patient information, and findings are registered in association with the image ID of each image. The patient ID is identification information for identifying a patient to be imaged of corresponding image data. The disease ID is identification information for identifying the type of disease of the corresponding patient. The treatment ID is identification information for identifying a treatment (medicine, surgery, etc.) performed on the corresponding patient. The shooting date represents the date when the image with the corresponding image ID was shot. Instead of “shooting date”, an item of “shooting time” indicating the time when the image was shot may be provided, and the shooting time may be registered together with the date. Patient information represents information about the patient, such as the name, age, and gender of the corresponding patient. The finding represents a doctor's view of the image with the corresponding image ID. The value of each item illustrated above is input by the user together with information specifying the image ID, and is registered in the interpretation report DB 104 in association with the specified image ID. Note that the interpretation report DB 104 may further store not only the above-described information about each image stored in the past disease example DB 102 but also information related to a predicted image generated by the predicted image generation unit 116 described later.

  Referring to FIG. 1 again, the feature amount calculation unit 106 calculates a feature amount representing the feature of the medical image. The feature amount calculation unit 106 may calculate the feature amount of the image stored in the past case DB 102, or may calculate the feature amount of the image acquired by the input receiving unit 100. In the example of the present embodiment, the feature amount calculation unit 106 specifies a lesion area in the image, and calculates a value representing the position, size, and shape of the specified lesion area in the image as the feature amount of the image. . As a technique for specifying a lesion area in an image, a technique conventionally used in specifying a lesion area (abnormal area) in a medical image may be used. For example, a method of extracting isolated shadows such as tumor shadows using a morphological filter after performing processing to emphasize circular shadows existing in an image (see JP 2004-336378 A), health A method of identifying an abnormal region in the data of the processing target image by comparing the data of the image obtained by photographing a person and the data of the processing target image may be used. The feature amount calculation unit 106 registers the feature amount calculated for the image in the feature amount DB 108 in association with the image ID of the image.

  The feature amount DB 108 is a database that stores image feature amounts. FIG. 3 shows an example of the data contents of the feature amount DB 108 in the example of the present embodiment. In the table of the example of FIG. 3, the feature amount calculated by the feature amount calculation unit 106 for the image with the image ID is registered in association with the image ID of each image. The feature amount includes items of a lesion position, a lesion size, and a lesion shape. The position of the lesion represents the position of the lesion area in the image, for example, the coordinates of the center point of the lesion area in the coordinate system defined for the image. The size of the lesion represents, for example, the area of the lesion area in the image. Alternatively, for example, the ratio of the area of the lesion area to the area of the entire image may be used as the value of the lesion size. The shape of the lesion may be, for example, a value representing the shape of the lesion area in the image (such as a circle, an ellipse, or a polygon), or may be a function that approximates the shape of the lesion area. Alternatively, image data of a lesion area specified in an image may be extracted from the image, and the extracted image data itself may be registered as a lesion shape.

  Returning to the description of FIG. 1, the deformation model generation unit 110 generates a deformation model that is a model of temporal change of a medical image. The deformation model generation unit 110 according to the example of the present embodiment refers to the past case DB 102, the interpretation report DB 104, and the feature amount DB 108, and obtains a deformation coefficient that represents a non-linear change in time series of images obtained by photographing the same patient.

Hereinafter, an example of a procedure for obtaining the deformation coefficient will be described. The deformation model generation unit 110 specifies an image ID associated with a set of a specific patient ID, disease ID, and treatment ID in the interpretation report DB 104. Further, the image of the specified image ID is read from the past disease case DB 102, and the feature amount of the specified image ID is read from the feature amount DB 108. In this example, as shown in FIG. 4, images I0, I1, and I2 obtained by photographing the lungs of one lung cancer patient are read from the past disease case DB 102, and feature amounts (lesion regions (circles r0) of the images I0, I1, and I2 are read. , R1, r2)), the position, size, and shape) are read from the feature value DB 108. The images I0, I1, and I2 are taken in this order at intervals of three months. The deformation model generation unit 110 obtains the shooting order and time interval of the images I0, I1, and I2 from the shooting date of each image registered in the interpretation report DB 104. The deformation model generation unit 110 obtains, as a deformation coefficient from the image Ii to the image Ij, a parameter that defines the conversion function T for converting the image of the lesion area in the image Ii into the image of the lesion area in the image Ij (where i , J = 0, 1, 2; i <j). In this example, it is assumed that this conversion function T is a conversion matrix expressed as T = T _global + T _local by two conversion matrices T _global and T _local .

ここで、（ｘ，ｙ）は、変換前の画像Ｉｉにおける一画素の座標を表す。式（１）のθ_１１，θ_１２，θ_２１，θ_２２，θ_１３，θ_２３は、変換マトリクスＴ_{ｇｌｏｂａｌ}を定義するパラメータである。 The conversion matrix T _global expresses the entire change of the imaging target part in the image. Even images obtained by photographing the same subject to be photographed of the same patient may not be completely the same if the photographing timing is different. In order to express such a difference between images, the transformation matrix T _global of this example is a matrix representing affine transformation as shown in Expression (1).
[Equation 1]

Here, (x, y) represents the coordinates of one pixel in the image Ii before conversion. In Equation (1), θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ , θ ₁₃ , and θ ₂₃ are parameters that define the conversion matrix T _global .

ただし、
［数３］

である。また、式（２）において、関数ＢはＢ−スプライン関数である。式（２）の変換マトリクスＴ_{ｌｏｃａｌ}を定義するパラメータは、Ｎｘ，Ｎｙである。 The transformation matrix T _local represents the change of the lesion area in the image. The conversion matrix T _{local in} this example represents conversion by free form deformation (FFD). Specifically, in the lesion area in the image, a control point by an Nx × Ny mesh is generated, and the change of the lesion area is represented by the transformation matrix T _local of Expression (2) using this control point.
[Equation 2]

However,
[Equation 3]

It is. Moreover, in Formula (2), the function B is a B-spline function. The parameters that define the transformation matrix T _local in equation (2) are Nx, Ny.

式（３）において、Ｈ（Ｉ（Ｓ_ｉ）），Ｈ（Ｉ（Ｓ_ｊ））は、それぞれ、画像Ｉｉ，Ｉｊの病変領域の画像のエントロピーを表す。また、Ｈ（Ｉ（Ｓ_ｉ），Ｉ（Ｓ_ｊ））は、画像Ｉｉおよび画像Ｉｊの病変領域の画像の結合エントロピーを表す。 The deformation model generation unit 110 uses the images Ii and Ij and the feature amount (position, size, and shape of the lesion area) of each image, and the parameters θ ₁₁ and θ _{12 of the} transformation matrices T _global and T _local. , Θ ₂₁ , θ ₂₂ , θ ₁₃ , θ ₂₃ , Nx, Ny are obtained. For example, the parameters θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ , θ ₁₃ , and the like that maximize the mutual information (Mutual Information, MI) between the image of the lesion area in the image Ii and the image in the lesion area of the image Ij. The values of θ ₂₃ , Nx, Ny are obtained. The mutual information amount is expressed by the following equation (3).
[Equation 4]

In Expression (3), H (I (S _i )) and H (I (S _j )) represent the entropy of the lesion areas of the images Ii and Ij, respectively. H (I (S _i ), I (S _j )) represents the combined entropy of the image Ii and the image of the lesion area of the image Ij.

Using the parameter values obtained to maximize the mutual information amount of Expression (3), the deformation model generation unit 110 uses the deformation coefficients P = (θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ from the image Ii to the image Ij. , Θ ₁₃ , θ ₂₃ , Nx, Ny). When the deformation model generation unit 110 obtains the deformation coefficient P from the image Ii to the image Ij, the image Ii is set to the initial state S, and the time interval between the shooting time of the image Ii and the shooting time of the image Ij is set as the elapsed time t. And the obtained deformation coefficient P is registered in the deformation model DB 112 in association with the initial state S and the elapsed time t. Hereinafter, the vector (S, P, t) represented by the initial state S, the deformation coefficient P, and the elapsed time t is referred to as a “deformation vector”. In addition, a set of deformation vectors whose initial state is each of a plurality of images stored in the past disease case DB 102 is referred to as a “deformation model”.

  The deformation model DB 112 is a database that stores the deformation model generated by the deformation model generation unit 110. FIG. 5 shows an example of data contents of the deformation model DB 112. In the table of the example of FIG. 5, the items of patient ID, disease ID, treatment ID, initial state S, and (deformation coefficient P, elapsed time t) are associated with each other. One row of the table in the example of FIG. 5 is a record representing information on the deformation vector obtained with one image as an initial state. In the example table of FIG. 5, the values of the patient ID, the disease ID, and the treatment ID are values registered in the interpretation report DB 104 in association with the corresponding image in the initial state S, and the interpretation model generation unit 110 interprets the interpretation report. Obtained from the DB 104 and registered in the deformation model DB 112. In this example, it is assumed that the value of the initial state S is the image ID of the corresponding image. Further, in the table of the example of FIG. 5, each column in the item of (deformation coefficient P, elapsed time t) represents information of a deformation vector including the same elapsed time. The character string “N / A” indicates that the deformation vector of the elapsed time of the corresponding column is not generated for the image in the initial state of the corresponding row. In other words, the image is taken when the elapsed time of the column has elapsed from the date of photographing the initial state image of the row, and is associated with the same combination of the patient ID, disease ID, and treatment ID as the initial state image. The image to be displayed is not registered in the past disease case DB 102.

  The similar image search unit 114 searches for images similar to the query image received as a processing target by the input receiving unit 100 among the images registered in the past case DB 102. For example, the similar image search unit 114 obtains a similarity between the feature amount of the query image calculated by the feature amount calculation unit 106 and the feature amount of each image registered in the feature amount DB 108, and the similarity is calculated in advance. An image larger than the set threshold is specified as a similar image. The degree of similarity between two feature quantities can be obtained, for example, by representing each feature quantity in a vector format and calculating a value representing the degree of similarity between the two feature quantity vectors. As a value representing the degree of similarity between vectors, for example, a correlation coefficient or Euclidean distance may be calculated.

The predicted image generation unit 116 generates a predicted image that is an image obtained by predicting the state of the shooting target when a certain time has elapsed from the shooting time of the query image received as the processing target by the input receiving unit 100. The predicted image generation unit 116 acquires, from the deformation model DB 112, a deformation vector having the initial state S for each of the similar images of the query image searched by the similar image search unit 114. Of the acquired deformation vectors, for each set of deformation vectors including the same elapsed time, the deformation coefficients included in the deformation vectors of the set are averaged, and the value of the result is used as a deformation parameter. In the following description, the term “deformation parameter” refers to a parameter that defines an image conversion function and is obtained using a deformation coefficient included in a plurality of deformation vectors. The predicted image generation unit 116 converts the query image using a conversion function T = T _global + T _local (see formula (1) and formula (2)) defined by the deformation parameter, and uses the converted image as the predicted image. . It can be said that the predicted image generated in this way is an image obtained by predicting the state of the query image to be captured when the elapsed time included in the deformation vector used to generate the deformation parameter has elapsed.

  The predicted image DB 118 is a database that stores predicted images. The predicted image DB 118 stores an elapsed time and a predicted image related to the elapsed time in association with the original query image. In FIG. 6, an example of the data content of prediction image DB118 is shown. In the table of the example of FIG. 6, a predicted image ID that is identification information of a predicted image generated when the time of the “elapsed time” item has elapsed is registered in association with the image ID of each query image. . Instead of the predicted image ID, the storage position of the corresponding predicted image in the predicted image DB 118 may be registered.

  The output processing unit 120 performs processing for outputting a predicted image. For example, the output processing unit 120 displays the predicted image stored in the predicted image DB 118 on a display device (not shown). At this time, the query image that is the basis of the predicted image, the corresponding elapsed time, information on the patient to be imaged of the query image, and the like may be displayed on the display device together with the predicted image. Further, the output processing unit 120 may generate and output a prediction report that is a document including a predicted image in accordance with a predefined format. The prediction report includes a prediction image and various types of information related to the prediction image. The various types of information related to the predicted image include, for example, a query image that is the basis of prediction, elapsed time, a patient to be imaged, a disease, a treatment, and a test result. The prediction report may be, for example, a document in an XML (Extensible Markup Language) format. The output destination of the prediction report may be the interpretation report DB 104 or another information processing apparatus connected to the diagnosis support apparatus 10.

  FIG. 7 is a flowchart illustrating an example of a procedure of a deformation model generation process performed in the diagnosis support apparatus 10. Before the procedure of the example of FIG. 7 is started, images taken in the past have been registered in the past case DB 102, and information about each image registered in the past case DB 102 is stored in the interpretation report DB 104 (see FIG. 7). 2) is registered.

  Referring to FIG. 7, first, the feature amount calculation unit 106 of the diagnosis support apparatus 10 calculates the feature amount of each image stored in the past case DB 102 (step S10). In this example, the feature amount calculation unit 106 specifies a lesion area in each image, and obtains the position, size, and shape of the lesion area as the feature amount of the image. The feature amount calculation unit 106 registers the calculated feature amount in the feature amount DB 108 in association with the image ID of each image (step S12).

  When the calculation of the feature amount and the registration in the feature amount DB are completed, the deformation model generation unit 110 selects one of the images stored in the past disease case DB 102 as an initial state image in the deformation vector (step S14). ).

  Next, it is determined whether or not there is an image representing a change from the selected image in the initial state (step S16). In this example, with reference to the interpretation report DB 104, images taken after the shooting date of the image in the initial state among the images associated with the same combination of the patient ID, the disease ID, and the treatment ID as the image in the initial state are taken. The presence or absence of an image is determined.

If there is an image representing a change from the image in the initial state (YES in step S16), the deformation model generation unit 110 generates a deformation vector representing a change from the image in the initial state for each of the images representing the change. (Step S18). In step S18 of this example, the image in the initial state is set as the image Ii, each of the images confirmed to exist in step S16 is set as the image Ij, and the conversion function is performed according to the above procedure using the equations (1) to (3). A parameter of T (= T _global + T _local ) is obtained. Then, for each of the images whose existence has been confirmed in step S16, the obtained parameter is a deformation coefficient P, and a deformation vector having the difference between the shooting date of the image and the shooting date of the initial image as elapsed time t (S, P, t) is generated (S is the image ID of the image in the initial state). As a specific example, if the image in the initial state is the image I0 in the example of FIG. 4, the presence of the image I1 and the image I2 is confirmed as an image representing a change from the image I0 in step S16. In this case, in step S18, the parameters (P _{I0 → I1} ) of the conversion function T from the image I0 to the image I1 and the conversion function T from the image I0 to the image I2 are calculated using the equations (1) to (3). The parameter (P _{I0 → I2} ) is obtained. Furthermore, a deformation vector (I0, P _{I0 → I1} , 3 months) representing a change from the image I0 to the image I1, and a deformation vector (I0, P _{I0 → I2} , 6 months) representing a change from the image I0 to the image I2. Is generated.

  In step S18, the image Ii and each image Ij used in the calculation for obtaining the deformation coefficient are acquired from the past disease case DB 102, and the feature amounts of these images are acquired from the feature amount DB 108.

  When a deformation vector representing a change from the image in the initial state to each of the images confirmed to exist in step S16 is generated, the deformation model generation unit 110 registers the generated deformation vector in the deformation model DB 112 (step S20). In this example, the deformation model generation unit 110 associates the image ID of the initial state image with the combination of the patient ID, the disease ID, and the treatment ID related to the initial state image, and the deformation coefficient P of each generated deformation vector. And a set of elapsed time t are registered in the deformation model DB 112 (see FIG. 5).

  After step S20 or when there is no image representing a change from the image in the initial state selected in step S14 (NO in step S16), each of all images in the past disease case DB 102 has been processed as the initial state. It is determined whether or not there is (step S22). If the processing has been completed (YES in step S22), the processing of the procedure in the example of FIG. 7 ends. If not (NO in step S22), the processing in steps S14 to S20 is performed on the unprocessed image. Done.

  7, the deformation model DB 112 stores a set of deformation vectors (deformation model) representing changes with time of each image stored in the past disease case DB 102. Using this deformation model, the diagnosis support apparatus 10 generates a predicted image representing the state of the imaging target of an image that has not been registered in the past case DB 102 when a certain time has elapsed.

  FIG. 8 is a flowchart illustrating an example of a procedure of predicted image generation processing performed in the diagnosis support apparatus 10.

  Referring to FIG. 8, the input receiving unit 100 acquires a query image that is a processing target image (step S30). The query image is a medical image obtained by photographing a patient, and is an image that is not registered in the past disease example DB 102. In step S30, at least one of the disease ID and treatment ID of the patient related to the query image may be acquired together with the query image.

  Next, the feature amount calculation unit 106 calculates the feature amount of the query image acquired in step S30 (step S32). In this example, the lesion area in the query image is specified, and the position, size, and shape of the lesion area are calculated as the feature amount.

  When the feature amount of the query image is calculated, the similar image search unit 114 searches for an image similar to the query image among the images in the past disease case DB 102 (step S34). In this example, the similar image search unit 114 obtains a similarity between the feature amount of the query image and the feature amount of each image stored in the feature amount DB 108, and the similarity is larger than a preset threshold value. The image ID of the image is specified. The image with the specified image ID is an image stored in the past case DB 102. As a specific example for the following description, it is assumed that similar images Ia, Ib, Ic,. If at least one of the disease ID and the treatment ID of the patient related to the query image is acquired in step S30, only the image associated with the disease ID and the treatment ID (at least one) is searched in step S34. It should be the target.

  The predicted image generation unit 116 acquires a deformation vector having each of the similar images searched in step S34 as an initial state from the deformation model DB 112 (step S36). FIG. 9 shows an example of the deformation vector acquired in the case of the above specific example. A row of the table in the example of FIG. 9 represents information on deformation vectors having the similar images Ia, Ib, Ic,. In the table of the example of FIG. 9, each column in the item of (deformation coefficient P, elapsed time t) represents information of a deformation vector including the same elapsed time ti (i = 1, 2,..., M). In FIG. 9, the character string “N / A” represents that the deformation vector of the elapsed time of the corresponding column is not generated for the image in the initial state of the corresponding row.

Referring again to FIG. 8, after step S <b> 36, the predicted image generation unit 116 uses the acquired modified vectors to query for the elapsed time t for each elapsed time t included in these modified vectors. A deformation parameter representing a change in the image is generated (step S38). In the case of the specific example of FIG. 9, the average of the value of the deformation coefficient P is obtained for each column of the item (deformation coefficient P, elapsed time t), and the obtained average value is used as the deformation parameter Pt for the elapsed time t of the column. To do. That is, the deformation parameter Pti regarding the elapsed time ti (i = 1, 2,..., M) is an average of the deformation coefficients Pai, Pbi,. Further, in the example of the present embodiment, each deformation coefficient P in the deformation vector is expressed by the parameters θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ , θ ₁₃ , θ ₂₃ of the transformation matrix T _global of equation (1) and the equation ( 2) including parameters Nx and Ny of the transformation matrix T _local, the deformation parameters Pti relating to the elapsed time ti are θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ , θ ₁₃ , θ ₂₃ , Nx, and Ny. Includes the average of each value.

When the deformation parameter related to each elapsed time is generated, the predicted image generation unit 116 generates an image as a result of converting the query image using the conversion function T defined by the generated deformation parameter, and the generated image is converted to each elapsed time. (Step S40). In the specific example described above with reference to FIG. 9, θ ₁₁ , θ ₁₂ , θ ₂₁ , θ ₂₂ , θ ₁₃ included in the deformation parameters Pt 1, Pt 2,. , Θ ₂₃ , Nx, Ny are used to convert the query image using each conversion function T = T _global + T _local to generate a predicted image for each elapsed time t1, t2,.

  The predicted image generation unit 116 registers each elapsed time and the predicted image generated for the elapsed time in association with each other in the predicted image DB 118.

  Further, the predicted image generated in step S40 is output by the output processing unit 120 (step S42). In step S42, for example, the output processing unit 120 displays the generated predicted image and the elapsed time associated with the predicted image on a display device (not shown) in association with each other. At this time, the query image may be displayed together with the predicted image. Alternatively, for example, a prediction report of each prediction image may be generated according to a predefined format, and the generated prediction report may be registered in the interpretation report DB 104. Further, for example, the generated prediction report may be displayed on the display device or transmitted to another information processing device connected to the diagnosis support device 10. After step S42, the process of the procedure in the example of FIG. 8 ends.

  In the predicted image generation process in the example of FIG. 8 described above, all of the deformation vectors having the initial states of the similar images searched in step S34 are acquired from the deformation model DB 112 and included in these deformation vectors. A predicted image is obtained for each of the elapsed times. In another example of the predicted image generation process, prior to the process of acquiring the deformation vector (step S36), the input reception unit 100 receives designation of the elapsed time, and in step S36, the deformation vector having each similar image as an initial state. Of these, only the deformation vector including the specified elapsed time may be acquired from the deformation model DB 112. In this example, if the processing after step S38 is performed in the same manner as described above, a predicted image representing the state of the query target of the query image when the specified elapsed time has elapsed is generated and output.

  Further, in still another example of the predicted image generation process, various types of predicted image generation processes, such as the generated predicted image, the deformation parameter used to generate the predicted image, and the deformation vector used to generate the deformation parameter, are used. This information may be stored in a cache storage device (not shown). In this example, if the acquisition target or generation target information has already been stored in the cache storage device in each of the processes in steps S36 to S40, the information may be acquired from the cache storage device. For example, if the modified vector having the similar image of the search result in step S34 as the initial state has already been stored in the cache storage device, the modified vector is read from the cache storage device and a modified parameter is generated (step S38). Also, for example, if a deformation parameter generated from the same deformation vector as that acquired in step S36 has already been stored in the cache storage device, the deformation parameter generation process is omitted, and the deformation parameter read from the cache storage device is The predicted image is generated (step S40). Further, when the predicted image generated using the same deformation parameter as the deformation parameter to be generated in step S38 is already stored in the cache storage device, the predicted image is output as the predicted image of the query image in step S42. Steps S38 and S40 may be omitted. When the cache storage device is used, the processing speed is increased as compared with the case where the cache storage device is not used.

  The past disease example DB 102, the interpretation report DB 104, the feature amount DB 108, and the modified model DB 112 may be updated using the query image that is the target of the predicted image generation processing according to the procedure of the example of FIG. FIG. 10 is a flowchart illustrating an example of a processing procedure for updating each database. The process of the procedure in the example of FIG. 10 is started after the process of the procedure in the example of FIG.

  Referring to FIG. 10, first, a new image ID is given to the query image, and the query image is registered in the past disease case DB 102 together with the new image ID (step S50).

  Next, the feature amount of the query image calculated in step S32 of FIG. 8 is registered in the feature amount DB 108 in association with the new image ID given to the query image in step S50 (step S52).

  After step S52, information on each item registered in the image interpretation report DB 104 in association with the query image is acquired (step S54). Information on each item is acquired, for example, by receiving user input in the input receiving unit 100. In the case of the interpretation report DB 104 in the example of FIG. 2, the patient ID of the patient to be imaged of the query image, the disease ID of the disease of the patient, the treatment ID of the treatment performed on the patient, the date of imaging of the query image , And the contents of each item of the doctor's findings regarding the query image.

  The information acquired in step S54 is registered in the image interpretation report DB 104 in association with the new image ID assigned to the query image in step S50 (step S56).

  The past disease case DB 102, the interpretation report DB 104, and the feature amount DB 108 are updated by the processing in steps S50 to S56 described above. Hereinafter, the process of step S58 to step S62 for updating the deformation model DB 112 will be described.

  First, the deformation model generation unit 110 acquires an image captured before the query image capturing time, which is an image of the same patient as the query image, from the past case DB 102 (step S58). In the example of the present embodiment, in step S58, the deformation model generation unit 110 refers to the interpretation report DB 104, and among the image IDs associated with the shooting date before the shooting date of the query image, the same patient as the query image. An image ID associated with a combination of ID, disease ID, and treatment ID is acquired. And the image of acquired image ID is acquired from past disease example DB102.

  Next, the deformation model generation unit 110 sets each of the images acquired in step S58 as an initial state image, and generates a deformation vector representing a change from the initial state image to the query image (step S60). The details of the deformation vector generation process may be the same as described with reference to step S18 of FIG.

  After step S60, the deformation model generation unit 110 registers the generated deformation vector in the deformation model DB 112 (step S62). That is, in the deformation model DB 112, a set of the deformation coefficient P and the elapsed time t obtained in step S60 is additionally registered in a record in which each of the images acquired in step S58 is in an initial state.

  The various databases are updated using the query image as a past image by the procedure of the example in FIG. 10, and the contents of the various databases after the update are used to generate predicted images for other query images thereafter.

  After the predicted image is generated, when the elapsed time related to the predicted image has elapsed, an image of the same patient may actually be captured. The accuracy of prediction of the predicted image may be evaluated using the image taken at this time, and the evaluation result may be reflected in the subsequent predicted image generation processing. For example, in step S38 in FIG. 8, instead of simply averaging the deformation coefficients in the deformation vector to generate the deformation parameters, the deformation coefficients in the deformation vectors are weighted based on the above-described evaluation results, A weighted average may be used as a deformation parameter. The weight of the deformation coefficient is obtained, for example, by the procedure shown in FIG.

  Referring to FIG. 11, the input receiving unit 100 acquires an image and information related to the image (step S <b> 70). In this example, a patient ID, a disease ID, a treatment ID, and an imaging date are acquired as information about the image.

  Next, a predicted image corresponding to the image acquired in step S70 is specified (step S72). The process of step S72 is executed by the following procedure, for example. First, the image ID associated with the combination of the patient ID, the disease ID, and the treatment ID acquired in step S70 and the photographing date are acquired from the interpretation report DB 104. Then, the prediction image ID registered in the prediction image DB 118 in association with the set of the acquired image ID and the elapsed time from the shooting date to the shooting date acquired in step S70 is specified. The predicted image with the specified predicted image ID is a predicted image corresponding to the image acquired in step S70. In addition, when the user grasps the corresponding predicted image, the input of the predicted image ID by the user may be accepted instead of the process of step S72 described above.

  When the corresponding predicted image is specified, the deformation vector used to generate the predicted image specified in step S72 is evaluated using the image acquired in step S70 (step S74). An example of the processing procedure of step S74 will be described below. First, the deformation vector used to generate the predicted image is specified. The deformation vector used for generating the predicted image is obtained by, for example, obtaining a similar image of a query image (image ID registered in the predicted image DB 118 in association with the predicted image) from the past disease case DB 102 when the predicted image is generated. What is necessary is just to specify it by searching, making the searched similar image into an initial state, and searching the deformation | transformation model DB112 for the deformation | transformation vector containing the elapsed time linked | related with the said prediction image. Alternatively, in the predicted image generation process (FIG. 8), if the deformation vector used to generate this predicted image is stored in the predicted image DB 118 in association with the generated predicted image, in step S74, the predicted image DB 118 corresponds. It is only necessary to acquire the deformation vector used to generate the predicted image to be generated. When the deformation vector used to generate the predicted image is specified, an image in the initial state is read from the past disease case DB 102 for each deformation vector. The read image is converted using a conversion function defined by the deformation coefficient in the deformation vector, and the feature amount of the image after the conversion is obtained. Also, the feature amount of the image acquired in step S70 is obtained, and the similarity between the feature amount of the image obtained in step S70 and the feature amount of the image after the conversion described above is obtained. This similarity is used as the evaluation value of the deformation vector related to the converted image.

  The weight based on the evaluation of each deformation vector obtained in step S74 is registered in the deformation model DB 112 in association with each deformation vector used for generating the predicted image (step S76). The weight value may be determined so as to increase as the evaluation obtained in step S74 increases. For example, in the case of the example of the processing procedure in step S74 described above, the greater the similarity between the feature amount of the image acquired in step S70 and the feature amount of the image after conversion for each deformation vector, the greater the deformation. What is necessary is just to enlarge the value of the weight linked | related with a vector. The higher the similarity is, the more the deformation coefficient of the deformation vector is converted to a query function that converts the query image into an image that is more similar to the actually captured image when the elapsed time of the deformation vector has elapsed from the capturing time of the query image. It can be said that it is a parameter that defines After step S76, the process of the procedure in the example of FIG. 11 ends.

  Embodiments of the present invention are not limited to the various embodiments and modifications described above. For example, specific modes of various databases included in the diagnosis support apparatus 10 may be different from the above-described example. For example, registration of an image ID in each database may be omitted, and a pair of a patient ID and an imaging date may be used as image identification information. Further, for example, in the interpretation report DB 104, in addition to the items illustrated in FIG. 2, the types of examinations performed on the patient and the results thereof may be registered. Further, two or more databases among the past case DB 102, the interpretation report DB 104, and the feature DB 108 may be collected as one database.

In the above-described embodiment, as the conversion function T, the conversion function T = T _global + T _local represented by T _{global in} Expression (1) and T _{local in} Expression (2) is used. The conversion function T may be used. A function for converting one image into another image, and a function defined by one or more parameters, using a conversion function T of a format different from that described above, a modified model by the same processing procedure as described above Generation and prediction image generation may be performed. In the above-described embodiment, the conversion function T is a function for converting image data of a lesion area in an image obtained by imaging a patient, but a function for converting the entire image obtained by imaging a patient is used as the conversion function T. Also good.

  In the above-described embodiment, the deformation vector is generated for the image associated with the same combination of the patient ID, the disease ID, and the treatment ID as the image in the initial state, but at least the patient ID is the same as the image in the initial state. For example, a deformation vector may be generated for images having different disease IDs or treatment IDs.

  The diagnosis support apparatus 10 exemplified above is realized, for example, by executing a program describing the function or processing content of each part of the diagnosis support apparatus 10 described above on a computer having a hardware configuration as shown in FIG. The 12 includes a CPU (Central Processing Unit) 80, a GPU (Graphics Processing Unit) 81 that performs operations related to image processing, a memory (primary storage) 82, various I / O (input / output) interfaces 84, and the like. 86. The circuit configuration is connected via 86. Further, a hard disk drive (HDD) 88, a disk drive 90 for reading portable nonvolatile recording media of various standards such as a CD, a DVD, and a flash memory is connected to the bus 86 via, for example, an I / O interface 84. Is done. Such a drive 88 or 90 functions as an external storage device for the memory. A program in which the processing content of the embodiment is described is stored in a fixed storage device such as the HDD 88 via a recording medium such as a CD or DVD or via a network, and is installed in a computer. The program stored in the fixed storage device is read into the memory and executed by the CPU or GPU, thereby realizing the processing of the embodiment. The GPU is in charge of operations related to image processing such as retrieval of similar images and calculation of image feature amounts. Operations related to image processing may be performed in parallel by the CPU and GPU.

  In the above description, the embodiment of the example in which the diagnosis support apparatus 10 is realized by one computer has been described. However, the various functions of the above-described example of the diagnosis support apparatus 10 are realized by being distributed to a plurality of computers. May be.

  DESCRIPTION OF SYMBOLS 10 Diagnosis support apparatus, 80 CPU, 81 GPU, 82 Memory, 84 I / O interface, 86 bus, 88 HDD, 90 Disk drive, 100 Input reception part, 102 Past disease example DB, 104 Interpretation report DB, 106 Feature-value calculation part , 108 feature amount DB, 110 deformation model generation unit, 112 deformation model DB, 114 similar image search unit, 116 prediction image generation unit, 118 prediction image DB, 120 output processing unit.

Claims (6)

Calculating a feature amount representing a feature of a target image that is an image to be processed;
With reference to a feature amount storage unit that stores feature amounts representing features of patient images taken in the past, the feature amounts of the target image and the feature amounts of the patient images are used. Identifying an image of the patient similar to the target image;
The change information storage means that stores the information indicating the change of the patient's image with time, which is obtained using the patient image taken in the past, represents the change with time of the image specified in the specifying step. Generating information, and using the acquired information, generating a conversion function of the image including a function expressing the entire change of the imaging target region, and a function expressing a non-linear change of the lesion area ;
Outputting an image obtained by converting the target image using the conversion function generated in the generating step;
A program that causes a computer to execute. The change information storage means is a parameter for determining a first image obtained by photographing a patient and a conversion function for converting the first image after the photographing time of the first image into a second image obtained by photographing the patient. And the parameter calculated using the second image and the second image and the elapsed time from the photographing time of the first image to the photographing time of the second image are stored in association with each other,
The feature amount storage means stores the feature amount of each of the first images,
In the specifying step, the first image similar to the target image is specified;
In the step of generating the conversion function of the image, a parameter associated with the first image identified in the identifying step and associated with the elapsed time is acquired from the change information storage unit, and acquired. A conversion function of the target image is generated using the obtained parameters.
The program according to claim 1. The feature amount of each of the target image and the first image represents a feature of a lesion area determined to be an area including a lesion in the image,
The parameter is a parameter that defines a conversion function for converting an image of the lesion area in the first image into an image of the lesion area in the second image.
The program according to claim 2, wherein: At least one of the images stored in the image storage means that stores the patient, the image obtained by photographing the patient, and the photographing time of the image in association with each other is set as the initial state image, and the initial state image is taken after the photographing time. A parameter for determining a conversion function for acquiring an image obtained by photographing the patient of the initial state image from the image storage means, and converting the initial state image into the acquired image using the acquired image and the initial state image. Calculating steps,
The initial state image as the first image, the image acquired from the image storage means in the step of calculating the parameter as the second image, the initial state image as the first image, and the parameter The parameter calculated in the step of calculating and the elapsed time from the shooting time of the initial state image to the shooting time of the image as the second image are output to the change information storage means in association with each other When,
Further executing the computer,
The program according to claim 2 or 3, characterized in that. A result image that is an image of a patient related to the target image is acquired when the elapsed time associated with the parameter used in the step of generating the conversion function of the image has elapsed from the shooting time of the target image. Steps,
Outputting the evaluation value obtained by evaluating the parameter used in the step of generating the conversion function of the image based on the result image to the change information storage unit in association with the parameter;
Is further executed by the computer,
In the step of generating the image conversion function, the image conversion function is generated using the evaluation value of the acquired parameter.
The program according to any one of claims 2 to 4, wherein: Calculation means for calculating a feature amount representing a feature of a target image that is an image to be processed;
With reference to a feature amount storage unit that stores feature amounts representing features of patient images taken in the past, the feature amounts of the target image and the feature amounts of the patient images are used. Identifying means for identifying an image of the patient similar to a target image;
Information representing a change with time of the image specified by the specifying means is obtained from a change information storing means storing information showing a change with time of the patient's image obtained using the patient image taken in the past. Generating means for generating an image conversion function including a function that expresses a change in the whole region to be imaged and a function that expresses a non-linear change in a lesion area using the acquired information;
An output unit that outputs an image obtained by converting the target image using the conversion function generated by the generation unit;
An information processing apparatus comprising:

Images