JP2017162124A - Image processing system, image processing method and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To accurately calculate similarity between a case of a query and other case.SOLUTION: When an image set similar to a search object image set which is a search object, is searched in a case image set group acquired by photographing by multi times, with a time interval for each patient, an image processing device 100 performs following processing of: identification of one or more sets including a set of images having a feature amount similar to a set of a feature amount of the first image and a feature amount of the second image, in a case image set group, for the set of the first image which is photographed at a first time and a second image photographed at a second time different from the first time, in the search object image set; and determination of similarity between the search object image set and one or more sets in the case image set group, using a feature amount of one or more images photographed at a time between a photographing time of a first similar image similar to the feature amount of the first image, and a photographing time of a second similar image similar to the feature amount of the second image, included in the identified one or more sets.SELECTED DRAWING: Figure 12

Description

  The present invention relates to an image processing system and the like.

  In the current medical field, images of a diseased part taken at different times for the same patient are compared, and the patient's medical condition is diagnosed from the difference in the compared images. Diagnosis by comparing images related to a diseased part of a patient is called comparative interpretation.

  In the task of comparative interpretation, a similar case search is performed for a case being diagnosed by searching for a similar case in the past and referring to the search result. For example, past cases with similar image characteristics of lesions are retrieved from the past case database and displayed. As a prior art regarding these operations, there is, for example, the prior art 1.

JP 2004-5364 A

  However, the above-described conventional technique has a problem that the similarity between the query case and other cases cannot be calculated with high accuracy.

  FIG. 14 is a diagram for explaining the problem. For example, it is assumed that a case image, which is an image related to a diseased part, is acquired by photographing a diseased part of one patient at a plurality of timings. For example, since a plurality of images are often collected at one timing for one diseased part in a CT image or MRI image, FIG. 14 shows a state in which a plurality of images are overlapped as an image. . As a result, a set of images taken for each patient at a plurality of timings is formed. In FIG. 14, for example, a surrounding line 21 indicates a case image set captured at each of five times. Similarly, it is assumed that case image sets 22 and 23 that are time-series image groups of a plurality of past imaging results are accumulated for other patients or other diseased sites. If each case image set 21 to 23 is referred to as a case image set group, a change in the diseased part of the patient to be diagnosed this time is stored in the database 20 in order to perform comparative interpretation. Which case image set in the group is similar is identified, and compared with past cases that are similar to each other, the doctor determines how the patient's current state is.

  Here, if the imaging intervals of the images included in each case image set 21 to 23 and the search target image set 25 to be searched, that is, each image included in the query case, are the same, the comparison will be easy. However, in general, the timing at which a diseased part is imaged differs depending on the patient, and the number of timings for imaging varies. In the example of FIG. 14, the search target image set 25 shows an image in which images taken at three times are present. Therefore, in order to be able to determine the similarity, a mechanism that absorbs the difference in imaging time for each patient is required.

  FIG. 15 is a diagram for explaining that the progression of the lesion, that is, the method of changing the lesion portion is non-linear. Assuming that each lesion image is expressed as a feature vector in the feature space, the trajectory of the feature vector is generally nonlinear. Therefore, the trajectory of each feature vector 31 in the linear space 30 is non-linear. For example, cancer tends to increase non-linearly so that the larger the cancer, the faster the growth rate.

  FIG. 16 is a diagram for explaining the problem. For example, when the feature vectors at times t1 to t3 are obtained from the feature vectors at time t0 and time t4 by linear interpolation, each feature vector moves along the line segment 32. However, since the actual feature vector moves nonlinearly along the curved surface 33, a divergence occurs between each feature vector obtained by linear interpolation and each actual feature vector.

  Since the tumor changes nonlinearly while taking various shapes, the high-dimensional curved surface formed by the feature vector is extremely complicated, and it is difficult to estimate the trajectory curve related to the feature vector of the image by learning.

  In one aspect, an object of the present invention is to provide an image processing system, an image processing method, and an image processing program that can accurately calculate the similarity between a query case and another case.

  In the first plan, the image processing system includes a specifying unit, a generation unit, and a determination unit. The specific unit is a search target among case image set groups including individual case image sets that can be expressed by a plurality of types of feature amounts, which are acquired by shooting multiple times for each patient. When searching for an image set similar to the search target image set, the first image shot at the first time in the search target image set and the second image different from the first time For the set with the second image, identify one or more sets in the case image set group that include sets of images each having a feature amount similar to the feature amount of the first image and the feature amount of the second image. To do. The generation unit has a shooting time of a first similar image having a feature amount similar to the feature amount of the first image and a feature amount similar to the feature amount of the second image included in the specified one or more sets. An interpolated image corresponding to an image photographed at a time between the first image and the second image is obtained using a feature amount of one or more images photographed at a time between the photographing times of the second similar images. One or more feature quantities estimated to have are generated. The determination unit determines the degree of similarity between the search target image set and one or more sets in the case image set group using the feature amount corresponding to the generated interpolation image.

  The similarity between the query case and other cases can be calculated with high accuracy.

FIG. 1 is a functional block diagram illustrating the configuration of the image processing apparatus according to the present embodiment. FIG. 2 is a diagram illustrating an example of the data structure of the image table. FIG. 3 is a diagram for explaining the processing of the control unit. FIG. 4 is a diagram for explaining feature vector extraction for a query case. FIG. 5 is a diagram for explaining extraction of a time-series image group. FIG. 6 is a diagram illustrating an example of a time-series feature vector group similar to a query case. FIG. 7 is a diagram for explaining mapping by kernel principal component analysis. FIG. 8 is a diagram for explaining the processing of the determination unit. FIG. 9 is a diagram for explaining processing for determining a time for interpolation. FIG. 10 is a diagram illustrating an example of a correspondence table between times. FIG. 11 is a diagram illustrating an example of a correspondence result between times by DP. FIG. 12 is a flowchart illustrating the processing procedure of the image processing apparatus according to the present embodiment. FIG. 13 is a diagram illustrating an example of a computer that executes an image processing program. FIG. 14 is a diagram for explaining the problem. FIG. 15 is a diagram for explaining the problem. FIG. 16 is a diagram for explaining the problem.

  Embodiments of an image processing system, an image processing method, and an image processing program disclosed in the present application will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments.

  FIG. 1 is a functional block diagram illustrating the configuration of the image processing apparatus according to the present embodiment. As illustrated in FIG. 1, the image processing apparatus 100 includes a communication unit 110, an input unit 120, a display unit 130, a storage unit 140, and a control unit 150.

  The communication unit 110 is a processing unit that performs data communication with other devices via a network or the like. For example, the communication unit 110 corresponds to a communication device.

  The input unit 120 is an input device that inputs various types of information to the image processing apparatus 100. For example, the input unit 120 corresponds to a keyboard, a mouse, a touch panel, or the like. The user operates the input unit 120 to select a query case or the like.

  The display unit 130 is a display device that displays the processing result of the control unit 150. The display unit 130 corresponds to a liquid crystal display, a touch panel, or the like. For example, the display unit 130 displays images of other cases similar to the query case searched for by the control unit 150.

  The storage unit 140 has an image table 141. The storage unit 140 corresponds to a semiconductor memory element such as a random access memory (RAM), a read only memory (ROM), and a flash memory, and a storage device such as a hard disk drive (HDD).

  The image table 141 is a table that stores information on time-series images regarding various cases. FIG. 2 is a diagram illustrating an example of the data structure of the image table. As shown in FIG. 2, the number of cases is recorded in the image table 141, and information in which a patient ID (Identification), the number of imaging times, and a data pointer are set is recorded in order for the number of cases. Has been. At the tip of the data pointer, the set time of the imaging time, image data, tumor position information, and feature vector is recorded in order for the number of imaging times.

  For example, each piece of image data associated with the same patient ID corresponds to the case image sets 21 to 23 described with reference to FIG. A group of case image sets for each patient ID corresponds to a group of case image sets. Further, as will be described later, a search target image set to be searched is selected from each case image set.

  The feature vector is a vector that represents, for example, a feature of a tumor as a lesion site included in the image data. For example, the feature vector is information obtained by digitizing the size of a rectangle surrounding the tumor, the coordinates of the rectangle, the concentration of the tumor, the color, and the like.

  The control unit 150 includes a specifying unit 151, a generation unit 152, and a determination unit 153. The control unit 150 corresponds to an integrated device such as an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA). The control unit 150 corresponds to an electronic circuit such as a CPU (Central Processing Unit) or an MPU (Micro Processing Unit).

  FIG. 3 is a diagram for explaining the processing of the control unit. As illustrated in FIG. 3, the control unit 150 receives time-series image feature vectors 40 and 50 corresponding to the symptom of the query (step S10). For example, the feature vectors 40 and 50 exist on the feature space 30a.

  The control unit 150 compares the feature vectors 40 and 50 with each feature vector in the image table 141 to obtain a similar feature vector (step S11). For example, feature vectors similar to the feature vector 40 are assumed to be feature vectors 41, 42, 43, 44. Feature vectors similar to the feature vector 50 are referred to as feature vectors 51, 52, 53, and 54.

  The control unit 150 extracts a time-series feature vector of a time-series image group to which each similar feature vector belongs (step S12). For example, the control unit 150 identifies patient IDs to which similar feature vectors belong based on the image table 141, and extracts all feature vectors belonging to the identified patient IDs as a time-series vector group. In the example shown in step S12 of FIG. 3, the control unit 150 extracts the feature vector groups 61a, 62a, 63a, and 64a using the patient IDs to which the feature vectors 41 to 44 and 51 to 54 belong as keys. In this example, four patient IDs are specified. By step S12, the vicinity part 32 regarding the trajectory curve of the query can be obtained on the high-dimensional curved surface 31 formed by the feature vector.

  The control unit 150 captures the nonlinear structure of the neighboring portion 32 related to the query trajectory curve as a linear space 30b by kernel principal component analysis. Actually, by the kernel principal component analysis, the time series feature vector groups 61a to 64a are regarded as the vector groups 61b to 64b of the linear space 30b through the high-dimensional space.

  The control unit 150 can estimate the approximate straight line 65b for the query case based on the time-series feature vector groups 61b to 64b (step S13). The approximate line 65b becomes an approximate curve 65a on the feature space 30a. The similarity between this approximate curve 65a and the other time series feature vector groups 61a to 64a is calculated based on the calculation of the distance between the approximate straight line 65b on the linear space 30b and the time series feature vector groups 61b to 64b. It becomes possible. In the example of FIG. 3, for simplification of explanation, an example in which the feature vectors 40 and 50 of two images are received among a plurality of time-series images for one patient is shown. The number may be any number as long as it is 2 or more.

  Next, each process of the specifying unit 151, the generating unit 152, and the determining unit 153 included in the control unit 150 will be described in order.

  When receiving a patient ID as a query case, the specifying unit 151 compares the received patient ID with the image table 141, and calculates two feature vectors of each time-series image indicated by the patient ID pointer. Extract above. The number of feature vectors extracted by the specifying unit 151 is not limited to two, and may be any number as long as it is two or more. For example, the feature vector extracted by the specifying unit 151 includes a feature vector of an image taken at the earliest time and a feature vector of an image taken at the last time among the time series images.

FIG. 4 is a diagram for explaining feature vector extraction for a query case. Assume that the query case has n time-series images. As described above, when receiving the patient ID, the specifying unit 151 extracts feature vectors of n time-series images from the image table 141. As a result, the obtained sequence of feature vectors is defined as “y _τ1 ,..., Y _τn ”. Here, the suffix τ _i of the feature vector indicates the observed time. As shown in FIG. 4, when n = 2, the time series image for the query case is a time series image of τ _{1 and} a time series image of τ ₂ , and the feature vectors are y _τ1 and y _τ2. It becomes. As described above, n may be 2 or more.

  The specifying unit 151 extracts a time series image group similar to the time series image of the query case by calculating the distance between the feature vector related to the case of the query and all the feature vectors on the image table 141.

FIG. 5 is a diagram for explaining extraction of a time-series image group. For each of the n feature vectors y _τ1 ,..., Y _τn of the query case, the specifying unit 151 includes the top N closest feature vectors among the feature vectors related to the individual images of the cases included in the image table 141. The feature vectors up to are calculated. In the example illustrated in FIG. 5, the specifying unit 151 extracts the top N-rank feature vectors x ^{τ1,1... Τ1, N} that are closest to the feature vector y _τ1 . The identifying unit 151 extracts the top N-th feature vector x ^{τ2,1... Τ2, N} that is closest to the feature vector y _τ2 .

  The closest top N feature vectors are defined as in Expression (1) with respect to the vector related to the query case.

The specifying unit 151 extracts all feature vector groups of time-series images of cases to which each feature vector of Expression (1) belongs. A feature vector group of the extracted time-series images is defined as in Expression (2). In Expression (2), it is ^{assumed that} the time series image of the case to which x ^{τi, j} belongs is a time series image composed of p _k ^{i, j} .

The specifying unit 151 selects a feature vector of a time-series image similar to the feature vector of the query case from the feature vector group defined by Expression (2). For example, the identifying unit 151 ^selects at least two vectors x _k ^{τi, J} defined by Expression (3) that are similar to the time series feature vector of the query case.

  FIG. 6 is a diagram illustrating an example of a time-series feature vector group similar to a query case. In the example illustrated in FIG. 6, feature vector groups 61a to 64a similar to the query case are extracted. For example, at least two feature vectors in the feature vector group 61a are similar to any feature vector of a query case. Similarly to the feature vector group 61a, the feature vector groups 62a to 64a are similar to any one of the feature vectors of the query cases.

  In the following description, as an example, it is assumed that the specifying unit 151 has extracted m time-series feature vector groups, and each feature vector group is defined as in Expression (4).

  Subsequently, the description proceeds to the generation unit 152. The generation unit 152 captures a nonlinear structure formed by a time-series feature vector group similar to a query case as a linear space through a high-dimensional space by kernel principal component analysis. As a result, the time series feature vector group is regarded as a vector group in a linear space. The distance between the approximate curve for the query case and the time-series feature vector group can be estimated based on the vector group in the linear space.

The generation unit 152 calculates a kernel matrix K using the kernel function k. Here, for convenience of explanation, a time-series feature vector group similar to a query case is represented as x ₁ ,..., X _q . Here, q is the number of time-series feature vector groups similar to the query. Further, as the kernel function, for example, a Gaussian kernel represented by Expression (5) is used. T included in Equation (5) is a certain constant.

  The kernel matrix K calculated by Expression (5) by the generation unit 152 is calculated as a q × q square matrix K as shown in Expression (6).

The generation unit 152 calculates, for the kernel matrix K, an upper eigenvalue having a large r value and an eigenvector corresponding to the upper eigenvalue. For example, the generation unit 152 obtains the eigenvalue λ by solving the eigen equation (7) of the kernel matrix K as an equation of the unknown λ. Further, each eigenvalue λ is substituted into the simultaneous equations (8) to obtain an eigenvector v corresponding to the eigenvalue λ. For example, the eigenvalues of the upper λ _1, ···, and λ _r, the corresponding eigenvectors _v 1, ···, and _{v r.}

  FIG. 7 is a diagram for explaining mapping by kernel principal component analysis. The generation unit 152 obtains the kernel matrix K of the feature vector groups 61a to 64a and calculates the eigenvalue λ and the eigenvector v based on the kernel matrix K, so that the feature vector groups 61a to 64a are vectors in the linear space 30b. It can be grasped as groups 61b to 64b.

  Subsequently, the description proceeds to the description of the determination unit 153. The determination unit 153 estimates an approximate curve obtained by interpolating the feature vector for the time series vector group of the query case, and calculates the similarity between the approximate curve and another case based on the distance calculation on the linear section 30b. A processing unit to calculate.

  FIG. 8 is a diagram for explaining the processing of the determination unit. The determination unit 153 estimates the distance between the query case and another time-series feature vector group based on the feature vector groups 61b to 64b. Specifically, the determination unit 153 determines the similarity between the approximate curve 65a and the other time-series feature vector groups 61a to 64a, and the distance between the approximate straight line 65b on the linear space 30b and the feature vector groups 61b to 64b. Based on the calculation of

  Hereinafter, the process of the determination unit 153 will be described in order. The determination unit 153 determines the time for interpolating the feature vector for the query case. For each time corresponding to the feature vector, the length between the times is set to a certain value σ or less. When the length between adjacent times is larger than σ, the determination unit 153 divides the time into s equal parts so as to be smaller than σ.

FIG. 9 is a diagram for explaining processing for determining a time for interpolation. For example, it is assumed that imaging times τ ₁ , τ ₂ , τ ₃ , τ ₄ , τ ₅ , τ ₆ and a predetermined time width σ of the query case are given as shown in FIG. The determination unit 153 determines that the interval between τ ₁ and τ _{2 and the} interval between τ ₄ and τ ₅ are larger than σ. The determination unit 153 divides τ ₁ and τ ₂ into s equal parts, and determines the time to interpolate based on the length of s equal parts when it becomes equal to or less than σ for the first time. In the example shown in FIG. 9, interpolated times τ ₁₁ and τ ₁₂ are determined between τ ₁ and τ ₂ . The determination unit 153 adds times τ ₁₁ and τ ₁₂ between τ ₁ and τ ₂ .

Similarly, the determination unit 153 divides τ ₄ and τ ₅ into s equal parts, and determines the interpolation time based on the length of s equal parts when it becomes equal to or less than σ for the first time. In the example shown in FIG. 9, a time τ ₄₁ to be interpolated is determined between τ ₄ and τ ₅ . The determination unit 153 adds a time τ ₄₁ between τ ₄ and τ ₅ .

The determination unit 153 calculates the distance between the feature vector related to the time-series image of the case in the image table 141 and the feature vector of the case of the query by DP (Dynamic Programming). Let x _t1 ,..., X _{t1 be} feature vectors related to time series images of cases in the image table 141. The determination unit 153 generates a correspondence table between times, and at each lattice point (τ _i , t _j ) or (τ _ik , t _j ), the _distance between y _τi and x _tj or y _τik and x _tj is _expressed as follows: Calculate by formula. Note that the feature vector of the query case time τ _i is y _τi .

When τ _i is not the interpolation time, the determination unit 153 calculates a distance D (y _τi , x _tj ) between the feature vector y _τi and the feature vector at the time t _j based on Expression (9).

When τ _ik is an interpolation time, the determination unit 153 calculates a distance D (y _τik , x _tj ) between the feature vector y _τik and the feature vector at the time t _j based on Expression (10). Here, τ _ik is a point that internally divides τ _i and τ _{i + 1} into m: n.

With respect to the above formulas (9) and (10), the determination unit 153 calculates the value of v (z) according to the formula (11). z corresponds to the _y τi, y τik or _{x tj.} V ₁ ,..., V _r included in Equation (11) correspond to the eigenvectors of the kernel matrix K described above.

FIG. 10 is a diagram illustrating an example of a correspondence table between times. The vertical axis of the correspondence table 60 indicates the imaging time of the query case and the interpolated time. The horizontal axis of the correspondence table 60 indicates the imaging time of the case in the image table 141. The determination unit 153 calculates the distance D (y _τi , x _tj ) or the distance D (y _τi , x _tj ) based on the above formulas (9) and (10). For example, the determination unit 153 creates a pair of τ _i and t _j or a pair of τ _ik and t _j and calculates a cumulative value of the distance D of each pair. The determination unit 153 comprehensively changes the pair of τ _i and t _j or the pair of τ _ik and t _j, and the pair of τ _i and t _j or τ _ik and t _j that minimizes the cumulative value. And a pair is searched based on DP.

  The concrete calculation method of DP is described in the literature (Yoshimura et al., “Offline verification method of Japanese signature by sequential application of DP matching method,” IEICE theory (D-II), Vol. J81-D-II No. 10, pp.2259-2266, “2.2.2 DP matching process” in October 1998). The determination unit 153 calculates DP based on the above calculation method.

FIG. 11 is a diagram illustrating an example of a correspondence result between times by DP. FIG. 11 shows a DP calculation result of a feature vector of a query case and a feature vector of a time-series image associated with a patient ID in the image table 141. In the example shown in FIG. 11, the DP, the feature vector y _.tau.1 time tau ₁ which case the query is captured, the time-series images of cases of image table 141 and a feature vector x _t3 of time t ₃ when taken It is associated. The feature vector y _τ2 at the time τ _{2 when} the query case is photographed is associated with the feature vector x _t6 at the time t ₆ when the time series image of the case in the image table 141 is photographed. The feature vector y _τ3 at the time τ _{3 when} the query case is photographed is associated with the feature vector x _t6 at the time t ₆ when the time series image of the case in the image table 141 is photographed. The feature vector y _τ4 at the time τ _{4 when} the query case is photographed is associated with the feature vector x _t7 at the time t ₇ when the time series image of the case in the image table 141 is photographed. The feature vector y _τ5 at the time τ _{5 when} the query case is photographed is associated with the feature vector x _t9 at the time t ₉ when the time series image of the case in the image table 141 is photographed. The feature vector y _τ6 at the time τ _{6 when} the query case is photographed is associated with the feature vector x _t9 at the time t ₉ when the time series image of the case in the image table 141 is photographed.

Further, the feature vector y _Tau11 time tau ₁₁ interpolated, the image sequence of the case of the image table 141 are associated with the feature vector x _t4 of time t ₄ when being photographed. The interpolated feature vector y _τ12 at the time τ ₁₂ is associated with the feature vector x _t5 at the time t ₅ when the time series image of the case in the image table 141 is captured. A feature vector y _Tau41 of interpolated time tau _41, time-series images of cases of image table 141 are associated with the feature vector x _t8 of the captured time t ₈ is.

  Similarly, the determination unit 153 repeatedly executes the above process for the feature vectors of the time-series images associated with other patient IDs. The determination unit 153 determines the similarity based on the accumulated value of the distance D related to the feature vector for each patient ID. For example, the determination unit 153 determines that the similarity is higher as the cumulative value is smaller.

  For example, the determination unit 153 uses the top N number of cases where the degree of similarity is large based on the degree of similarity between the feature vector of the query case and the feature vector of the time-series image associated with a patient ID in the image table 141. The series image is displayed on the display unit 130.

  Next, a processing procedure of the image processing apparatus 100 according to the present embodiment will be described. FIG. 12 is a flowchart illustrating the processing procedure of the image processing apparatus according to the present embodiment. As illustrated in FIG. 12, the specifying unit 151 of the image processing apparatus 100 receives a patient ID corresponding to a query case (step S101). The identifying unit 151 extracts a feature vector for the query case (step S102). The identifying unit 151 extracts a feature vector of a time-series image group similar to the query case (step S103).

  The generation unit 152 of the image processing apparatus 100 calculates an eigenvector by performing kernel principal component analysis on a time-series feature vector group similar to a query case (step S104). The determination unit 153 of the image processing apparatus 100 calculates the similarity between the query case and the case of the image table 141 (step S105). The determination unit 153 outputs a similar case (step S106).

  Next, effects of the image processing apparatus 100 according to the present embodiment will be described. The image processing apparatus 100 extracts a time-series image group similar to the time-series image of the query case, performs a kernel principal component analysis on the feature vector of the extracted time-series image group, and performs a stretch using the resulting eigenvector. The query case image is interpolated on the obtained linear space, and the similarity between the time series image of the query case and another time series image is obtained. Therefore, it is possible to accurately identify a time series image similar to the time series image of the query case by absorbing the difference between the shooting time of the time series image of the query case and the shooting time of other time series images. Can do.

  Depending on the patient, the time interval at which the lesion portion is imaged varies, and the time-series images for one case differ in the time interval between image captures, making it difficult to determine the similarity of lesion changes. However, as described above, the difference between the photographing time intervals is interpolated using similar time-series image groups, so that the degree of similarity between the query case and other cases can be determined.

  For example, when a time-series image of a query case is given with a set of time-series images relating to a tumor part of a patient being given as the image table 141, the time-series images are stored in the image table 141 regardless of the photographing time. The degree of similarity with the series image can be calculated with high accuracy. In particular, even when the progression of the lesion part is nonlinear, since the nonlinear structure is estimated and collated, the collation can be performed with high accuracy.

  The image processing apparatus 100 captures a nonlinear structure formed by the time-series feature vector group as a linear space by mapping a time-series feature vector group similar to the query case on the linear space based on the kernel principal component analysis. As a result, it is possible to estimate an approximate curve for a query case based on a linear space based on kernel principal component analysis, and thus it is possible to suppress the occurrence of deviation from an actual feature vector.

  The image processing apparatus 100 specifies the similarity based on the distance between the feature vector related to the time series image of the case in the image table 141 and the feature vector of the query case mapped in the linear space. For this reason, the similarity between the query case and other cases can be calculated with high accuracy.

  In the above-described embodiment, an example in which the storage unit 140, the specifying unit 151, the generating unit 152, and the determining unit 153 are included in the image processing apparatus 100 has been described. However, the present invention is not limited to this. For example, a first server having the storage unit 140, a second server having the specifying unit 151, a third server having the generating unit 152, and a fourth server having the determining unit 153 may be arranged on the network to form an image processing system. . Each of the first to fourth servers included in the image processing system performs data communication with each other, and executes the same processing as that of the image processing apparatus 100.

  Next, an example of a computer that executes an image processing program that realizes the same function as the image processing apparatus 100 described in the above embodiment will be described. FIG. 13 is a diagram illustrating an example of a computer that executes an image processing program.

  As illustrated in FIG. 13, the computer 200 includes a CPU 201 that executes various arithmetic processes, an input device 202 that receives data input from a user, and a display 203. The computer 200 also includes a reading device 204 that reads programs and the like from a storage medium, and an interface device 205 that exchanges data with other computers via a network. The computer 200 also includes a RAM 206 that temporarily stores various information and a hard disk device 207. The devices 201 to 207 are connected to the bus 208.

  The hard disk device 207 has a specific program 207a, a generation program 207b, and a determination program 207c. The CPU 201 reads the specific program 207 a, the generation program 207 b, and the determination program 207 c and develops them in the RAM 206.

  The specific program 207a functions as a specific process 206a. The generation program 207b functions as a generation process 206b. The determination program 207c functions as a determination process 206c.

  The process of the identification process 206a corresponds to the process of the identification unit 151. The process of the generation process 206 b corresponds to the process of the generation unit 152. The process of the determination process 206c corresponds to the process of the determination unit 153.

  The specific program 207a, the generation program 207b, and the determination program 207c are not necessarily stored in the hard disk device 207 from the beginning. For example, each program is stored in a “portable physical medium” such as a flexible disk (FD), a CD-ROM, a DVD disk, a magneto-optical disk, and an IC card inserted into the computer 200. Then, the computer 200 may read and execute each of the programs 207a to 207c.

  The following supplementary notes are further disclosed with respect to the embodiments including the above examples.

(Additional remark 1) It is set as a search object among the case image set groups including individual case image sets that can be expressed by a plurality of types of feature values acquired by taking a plurality of times for each patient. When searching for an image set similar to the search target image set, the first image captured at the first time in the search target image set and the second image different from the first time are captured. A set including a set of images each having a feature amount similar to the set of the feature amount of the first image and the feature amount of the second image in the case image set group. A specific part that specifies one or more of
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times A generating unit that generates one or more feature quantities that are estimated;
An image processing system comprising: a determination unit that determines a degree of similarity between the search target image set and one or more sets in the case image set group using a feature amount corresponding to the generated interpolation image.

(Additional remark 2) The said production | generation part performs the said kernel resemblance | analogue by performing a kernel principal component analysis with respect to the feature-value respectively similar to the combination of the feature-value of said 1st image, and the feature-value of said 2nd image The feature amount is mapped to a linear space, and one or more feature amounts estimated to be included in the interpolated image are generated based on the similar feature amount on the linear space. Image processing system.

(Additional remark 3) The said determination part is the distance of the said similar feature-value on the said linear space and the feature-value of the 1st image on the said linear space, the said similar feature-value on the said linear space, and the said 2nd image The image processing system according to claim 2, wherein the degree of similarity is determined based on a distance from a feature amount of the image and a distance between the similar feature amount in the linear space and the feature amount of the interpolated image. .

(Additional remark 4) About the said search object image set, the 3rd image and 4th image which were image | photographed before and after the time corresponding to the said interpolation image exist, and the time corresponding to the said interpolation image is the said 3rd image. A value obtained by multiplying the feature vector of the third image by n when the time is located at a point dividing the shooting time and the shooting time of the fourth image into m to n And subtracting the similar feature amount vector from a vector obtained by adding a feature value vector of the fourth image multiplied by m to the similar feature amount in the linear space and the interpolation 4. The image processing system according to appendix 2 or 3, wherein a distance from the feature amount of the image is calculated.

(Supplementary Note 5) An image processing method executed by a computer,
A search target image set to be searched between case image set groups including individual case image sets that can be expressed by a plurality of types of feature values, acquired by shooting multiple times for each patient. When searching for an image set similar to, the first image taken at the first time in the search target image set and the second image taken at a second time different from the first time Identify one or more sets in the case image set group that include sets of images each having a feature amount similar to the feature amount of the first image and the feature amount of the second image. And
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times Generate one or more features that are estimated to be
An image processing method comprising: executing a process of determining a degree of similarity between the search target image set and one or more sets in the case image set group using a feature amount corresponding to the generated interpolation image.

(Additional remark 6) The said process to generate | occur | produce performs said principal component analysis by performing a kernel principal component analysis with respect to the feature amount respectively similar to the feature amount of said 1st image, and the feature amount of said 2nd image Appendix 5 characterized in that one or more feature quantities estimated to be included in the interpolated image are generated on the basis of the similar feature quantities on the linear space. The image processing method as described.

(Supplementary Note 7) The process of determining the degree of similarity includes the distance between the similar feature amount on the linear space and the feature amount of the first image on the linear space, the similar feature amount on the linear space, and The additional degree is characterized in that the degree of similarity is determined based on a distance from the feature amount of the second image and a distance between the similar feature amount in the linear space and the feature amount of the interpolated image. Image processing method.

(Additional remark 8) About the said search object image set, the 3rd image and 4th image image | photographed before and after the time corresponding to the said interpolation image exist, and the time corresponding to the said interpolation image is the said 3rd image. When the time is located at a point where the shooting time and the shooting time of the fourth image are divided internally into m to n, the process of determining the degree of similarity is performed by adding n to the feature vector of the third image. The similar feature in the linear space is subtracted from a vector obtained by adding the value obtained by multiplying the vector of the feature amount of the fourth image and the value obtained by multiplying the vector of the feature amount of the fourth image by m. The image processing method according to appendix 6 or 7, wherein a distance between the amount and the feature amount of the interpolated image is calculated.

(Supplementary note 9) An image processing program executed by a computer,
A search target image set to be searched between case image set groups including individual case image sets that can be expressed by a plurality of types of feature values, acquired by shooting multiple times for each patient. When searching for an image set similar to, the first image taken at the first time in the search target image set and the second image taken at a second time different from the first time In the case image set group, at least one set including a set of images each having a feature amount similar to the set of the feature amount of the first image and the feature amount of the second image in the group of case images Identify,
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times Generate one or more features that are estimated to be
An image processing program for executing a process of determining a degree of similarity between the search target image set and one or more sets in a case image set group using a feature amount corresponding to the generated interpolation image.

(Additional remark 10) The said process to produce | generate is performed by performing a kernel principal component analysis with respect to the feature-value respectively similar to the feature-value group of said 1st image, and the feature-value of said 2nd image, and the said similarity Appendix 9 characterized in that one or more feature quantities estimated to be included in the interpolation image are generated based on the similar feature quantities on the linear space. The image processing program described.

(Supplementary Note 11) The process of determining the degree of similarity includes the distance between the similar feature amount on the linear space and the feature amount of the first image on the linear space, the similar feature amount on the linear space, and The supplementary note 10 is characterized in that the degree of similarity is determined based on a distance between the feature amount of the second image and a distance between the similar feature amount on the linear space and the feature amount of the interpolated image. Image processing program.

(Additional remark 12) About the said search object image set, the 3rd image and 4th image image | photographed before and after the time corresponding to the said interpolation image exist, and the time corresponding to the said interpolation image is the said 3rd image. When the time is located at a point where the shooting time and the shooting time of the fourth image are divided internally into m to n, the process of determining the degree of similarity is performed by adding n to the feature vector of the third image. The similar feature in the linear space is subtracted from a vector obtained by adding the value obtained by multiplying the vector of the feature amount of the fourth image and the value obtained by multiplying the vector of the feature amount of the fourth image by m. 12. The image processing program according to appendix 10 or 11, wherein a distance between the amount and the feature amount of the interpolated image is calculated.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 140 Storage part 141 Image table 150 Control part 151 Specification part 152 Generation part 153 Determination part

Claims (6)

A search target image set to be searched between case image set groups including individual case image sets that can be expressed by a plurality of types of feature values, acquired by shooting multiple times for each patient. When searching for an image set similar to, the first image taken at the first time in the search target image set and the second image taken at a second time different from the first time Identify one or more sets in the case image set group that include sets of images each having a feature amount similar to the feature amount of the first image and the feature amount of the second image. Specific part to do,
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times A generating unit that generates one or more feature quantities that are estimated;
An image processing system comprising: a determination unit that determines a degree of similarity between the search target image set and one or more sets in the case image set group using a feature amount corresponding to the generated interpolation image.   The generating unit performs kernel principal component analysis on a feature amount similar to each of the feature amount of the first image and the feature amount of the second image, thereby linearly converting the similar feature amount. 2. The image processing according to claim 1, wherein one or more feature amounts that are mapped to a space and estimated to be included in the interpolated image are generated based on the similar feature amounts on the linear space. system.   The determination unit includes a distance between the similar feature amount on the linear space and the feature amount of the first image on the linear space, the similar feature amount on the linear space, and a feature amount of the second image. The image processing system according to claim 2, wherein the degree of similarity is determined based on a distance between the feature amount and the distance between the similar feature amount in the linear space and the feature amount of the interpolated image.   For the search target image set, there are a third image and a fourth image captured before and after the time corresponding to the interpolation image, and the time corresponding to the interpolation image is the imaging time of the third image, When the time is located at a point that internally divides the shooting time of the fourth image into m to n, the determination unit multiplies the vector of feature values of the third image by n, Subtracting the similar feature amount vector from a vector obtained by adding the feature value vector of four images to a value obtained by multiplying m, thereby obtaining the similar feature amount in the linear space and the feature amount of the interpolated image. 4. The image processing system according to claim 2, wherein a distance between the first and second images is calculated. An image processing method executed by a computer,
A search target image set to be searched between case image set groups including individual case image sets that can be expressed by a plurality of types of feature values, acquired by shooting multiple times for each patient. When searching for an image set similar to, the first image taken at the first time in the search target image set and the second image taken at a second time different from the first time Identify one or more sets in the case image set group that include sets of images each having a feature amount similar to the feature amount of the first image and the feature amount of the second image. And
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times Generate one or more features that are estimated to be
An image processing method comprising: executing a process of determining a degree of similarity between the search target image set and one or more sets in the case image set group using a feature amount corresponding to the generated interpolation image. An image processing program executed by a computer,
A search target image set to be searched between case image set groups including individual case image sets that can be expressed by a plurality of types of feature values, acquired by shooting multiple times for each patient. When searching for an image set similar to, the first image taken at the first time in the search target image set and the second image taken at a second time different from the first time In the case image set group, at least one set including a set of images each having a feature amount similar to the set of the feature amount of the first image and the feature amount of the second image in the group of case images Identify,
The shooting time of the first similar image having a feature amount similar to the feature amount of the first image and the second similarity having a feature amount similar to the feature amount of the second image included in the specified one or more sets An interpolation image corresponding to an image photographed at a time between the first image and the second image using a feature amount of one or more images photographed at a time between the image photographing times Generate one or more features that are estimated to be
An image processing program for executing a process of determining a degree of similarity between the search target image set and one or more sets in a case image set group using a feature amount corresponding to the generated interpolation image.

Images