JPH10234674A - Method for judging presence or absence of change of fundus oculi image with time - Google Patents

Abstract

PROBLEM TO BE SOLVED: To support the judgment of normal example dismissal by means of a doctor by automatically judging presence or absence of change with time by means of comparing the fundus oculi images of a same patient. SOLUTION: Presence or absence of change with time in the fundus oculi image is judged in this judging method. In this case, the method is provided with process (S101-S107) where the shapes of an eyeball and a fundus part in two kinds of eye fundus oculi images to be a comparison object are made to be appropriate by a parabolic surface so as to be aligned, process (S108) where respective values are corrected in RGB space between fundus oculi images after aligning and process (S109) where presence or absence of change with time is judged through the use of Euclidean distance between respectively corrected eye fundus oculi images.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

[0001] 1. Field of the Invention [0002] The present invention relates to an image processing method for judging the occurrence of a lesion when performing an individual comparison of a fundus image.

[0002]

2. Description of the Related Art Conventionally, in computer-aided fundus image interpretation support, detection of individual lesions such as crossover phenomena, bleeding / white spots, and abnormalities of the retinal nerve fiber layer have been performed.
A method of extracting a blood vessel intersection from a blood vessel group obtained by filtering from a G plane of a color fundus image,
Template matching based on GB values, a method of extracting a lesion based on the concentration of a straight line group in an image, and the like have been used.

In order to detect a lesion based on a change over time, a simple linear transformation is used at the time of registration between images as a pre-process of a difference process. Has been used.

[0004]

SUMMARY OF THE INVENTION In health examinations,
90% of the total number of patients under the reference range in "screening" where specialists determine from experience whether or not a detailed examination is necessary because cost is an important factor
It is considered important to promptly reject the normal cases occupying the above.

The characteristics of lesions of important diseases that can be found from fundus images have various colors and shapes, as represented by circular / linear / point-like red spots and vitiligo, etc. In addition, since the object is a living body, individual differences in characteristics are large. Therefore, the above-described method for finding individual lesions has a first problem that a doctor must re-read a fundus photograph for finding other lesions.

In the method for detecting a lesion based on a change with time, there is a second problem that a simple linear transformation for alignment causes a large residual.

Further, the third problem is that the method of using a simple difference in luminance value as a criterion for a change with time during the difference processing is susceptible to a change in the photographing environment.

[0008] The object of the present invention is to achieve the object of rejecting normal cases, the first problem is that, in the method aimed at finding individual lesions, a doctor must re-read the image to find other lesions. The second problem is that the residual is large in a simple linear transformation for alignment, and the third method is that the method is susceptible to a change in a shooting environment in a method in which a simple difference in luminance value is used as a criterion of a change with time in a difference process. It is an object of the present invention to provide a method for judging the presence or absence of a temporal change which automatically determines the presence or absence of a temporal change by comparing the fundus images of the same examinee, and assists a doctor in determining the rejection of a normal case.

[0009]

According to the present invention, there is provided a method for judging whether or not there is a temporal change in a fundus image in a method for judging whether or not there is a temporal change in a fundus image. Process of approximating the shape of a parabola with a paraboloid, and RG
There is a step of correcting each value in the B space, and a step of determining whether there is a temporal change using the Euclidean distance between the fundus images corrected respectively.

In the process of performing positioning by approximating the shape of the eyeball and the fundus of the fundus image with a paraboloid, the intersection of the blood vessel and the intersection of the blood vessel and the nipple of the two fundus images to be compared are determined. It may be extracted as feature points, each feature point may be projected onto a paraboloid, and geometric transformation may be performed between corresponding feature points to perform alignment.In the alignment process after parabolic surface approximation, Matching may be performed by shifting each block of the fundus image divided into a predetermined size for each local region, and in the process of correcting the respective values in the RGB space, the luminance values between the fundus images are matched. Alternatively, interpolation may be performed.

In addition, the presence or absence of a temporal change using the Euclidean distance between the corrected fundus images is determined by three-dimensional RG.
The determination of the presence or absence of a temporal change may be based on the Euclidean distance between images in the B space.

In the present invention, by approximating the shapes of the eyeball and the fundus with a paraboloid, it is possible to suppress the displacement even when the photographing place of the fundus image is different. Further, by correcting the positions between the images after the alignment in the RGB space, it is possible to ignore the difference in brightness at the time of shooting. Further, by determining the Euclidean distance between the corrected images, it is possible to determine whether there is a change with time from the magnitude of the obtained value, and the doctor can objectively reject the normal case based on the determination result. .

[0013]

Next, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a flowchart of a method for determining the presence or absence of a change with time according to the embodiment of the present invention.
1 is input of a measurement image and a reference image, S102 is extraction of a corresponding candidate point, S103 is an initial corresponding point search and estimation of a relative shift amount between images, S104 is a corresponding candidate point group determination based on the relative shift amount, and S105 is Project corresponding point cloud on paraboloid, S10
6 is the estimation of the geometric transformation coefficient between the projected corresponding points, S1
07 is block position shift correction by shift matching,
S108 indicates each processing step of color correction by RGB value conversion, and S109 indicates each processing step of determining whether or not there is a change with time based on the Euclidean distance.

FIGS. 2A and 2B are photographs of a halftone image displayed on a fundus image display showing an example of extraction processing of a corresponding candidate point and an initial corresponding point, wherein FIG. 2A is a reference image, and FIG. In the figure, reference numeral 201 denotes a reference image, 202 denotes a measurement image, 203 denotes an initial corresponding point, 204 denotes a block having the maximum number of corresponding candidate points, and 205 denotes a corresponding candidate point.

FIG. 3 is a photograph of a halftone image displayed on a fundus image display showing an example of determining corresponding points.
(A) is a reference image, (b) is a measurement image, and reference numeral 301 in the figure denotes a reference image, 302 denotes a measurement image, and 306 denotes a corresponding point.

FIG. 4 is a schematic diagram showing an example of a process of projecting a corresponding point on a paraboloid. In FIG.
The axis, 403 indicates an image plane, 404 indicates a paraboloid, 405 indicates a corresponding point, and 406 indicates a corresponding point after projection.

FIG. 5 is a photograph of a halftone image displayed on a fundus image display showing a reference image after parabolic approximation conversion, and reference numeral 501 in the figure denotes a reference image after parabolic approximation conversion. .

FIG. 6 is a photograph of a halftone image displayed on the display of a fundus image showing a positional shift amount distribution for each block. In the figure, reference numeral 601 denotes a positional shift amount distribution of a reference image, and 606 denotes a corresponding point. , 607 are blocks.

FIG. 7 is a photograph of a halftone image displayed on the display of the fundus image showing the reference image after the shift matching, and reference numeral 701 in the figure is a reference image after the shift matching. FIG. 5 to FIG. 7 show examples of position adjustment by parabolic surface approximate conversion and shift matching.

FIG. 8 is a photograph of a halftone image displayed on the display of the fundus image of the reference image in the determination of the presence or absence of change with time, and reference numeral 801 in the figure denotes a reference image.

FIG. 9 is a photograph of a halftone image displayed on the display of the fundus image of the measured image in the determination of the presence or absence of a change with time, in which reference numeral 902 denotes a measured image, and 908 denotes a time-varying portion.

FIG. 10 is a photograph of a halftone image displayed on the display of the fundus image showing the distribution of the Euclidean distance value between the reference image and the measurement image, and reference numeral 1008 in the figure denotes a temporal change portion. FIG. 8 to FIG. 10 are diagrams illustrating an example of a temporal change presence / absence determination process for an image with temporal change.

First, in a fundus image of the same examinee, an image for which the presence or absence of a temporal change is to be determined is a measurement image 202, a comparison image to be referred to at the time of determination is a reference image 201, and each image is input to the computer through input processing S101. Corresponding points between images are determined by a corresponding candidate point extraction process S102, an initial corresponding point search and a relative displacement amount estimation process S103, and a corresponding point group determination process S104 based on the relative displacement amount.

In the process S102 for extracting the corresponding candidate points, the Gre image of the color image of the measurement image 202 and the reference image 201 is obtained.
Filter processing (smoothing, binarization,
An edge portion (a blood vessel portion or an edge of a nipple) that appears conspicuously in the image is extracted by minute area removal and thinning.

FIG. 2 shows the intersections of the edges (the blood vessel bifurcation points and the intersections between the nipples and the blood vessels) by template matching using a template such as a cross or a Y-shape for the extracted edge image. Like candidate point 2
Calculate as 05.

Next, in the initial corresponding point search and relative displacement estimation processing 103, the measured image is first divided into blocks of the same size, and the corresponding candidate points 205 included in the area of each block are divided. Count.

In each block, the block 204 having the largest number of corresponding candidate points including the largest number of corresponding candidate points 205 is selected, and the corresponding candidate point 205 is arbitrarily selected from within the block area.

The normalized correlation coefficients of the block centered on the candidate point and the block centered on the candidate point similarly obtained from the reference image are obtained for all the candidate points, and one set that satisfies the threshold or more is obtained. Repeat until you find one.

The set of found candidate points is the first corresponding point between the measurement image and the reference image. However, the size of the block when dividing the measurement image and the size of the block for obtaining the normalized correlation coefficient need not be the same.

The corresponding points found are used as the initial corresponding points 203 of the respective images, and the relative displacement between the corresponding points is estimated.
This relative shift amount corresponds to the amount of movement between images.

Determination processing S of the corresponding point group based on the relative shift amount
At 104, blocks that do not include the initial corresponding point are sequentially selected from the measurement image, and one candidate point in the area is arbitrarily selected.

Starting from the coordinates of the reference image obtained by adding the relative shift amount to the coordinates, shift matching based on the normalized correlation coefficient is performed, and if the coefficient value is equal to or larger than the threshold value, it is added as a corresponding point.

However, the size of the block at the time of the initial corresponding point search and the size of the block at the time of the shift matching based on the normalized correlation coefficient are the same.

Each block has a set of corresponding points, and the corresponding points are repeatedly assigned.

The finally obtained corresponding point group becomes a reference point 306 on the reference image 301 and the measurement image 302 in FIG.

Next, the process of projecting the corresponding point onto the radiation surface S10
5 and a geometric transformation coefficient estimation process S between the corresponding point group projected
At 106, using the determined corresponding point group, the reference image 3
01 and the geometric transformation coefficient of the measurement image 302 are estimated. In the present invention, the shape of the eyeball is approximated to the paraboloid 404 schematically shown in FIG. 4, and the geometric transformation coefficient is estimated.

Therefore, first, a paraboloid 404 in a three-dimensional space to be approximated is defined by the following equation.

[0038]

(Equation 1) However, the image plane 403 is the xy plane, the vertical direction of the plane is the z-axis 401, and the origin is the center of the fundus image. α and β are coefficients that define the parabolic surface shape. From Expression (1), each corresponding point 405 on the reference image / measurement image is projected onto the paraboloid 404, and each corresponding point 4 after projection on the reference image and the measurement image is projected.
The geometric transformation coefficient between 06 is obtained by the least square method as in the following equation.

[0039]

(Equation 2) However, P _K: coordinates P'the k-th corresponding points 406 on paraboloid 404 of the reference image _K: coordinates of the k-th corresponding points 406 on paraboloid 404 of the reference image after the geometric transformation Q _K: The k-th corresponding point 406 on the parabola 404 of the measurement image Coordinate R: rotation component of geometric transformation T: translation component of geometric transformation J: error function for estimating geometric transformation coefficient

As a result of transforming the reference image using the obtained geometric transformation coefficients, a reference image 501 after parabolic approximation transformation shown in FIG. 5 is obtained.

The paraboloid used in the present invention is represented by two variables, and there is a limit in approximating the eyeball shape. Therefore, in the block position shift correction processing S107 by the shift matching,
The reference image 501 shown in FIG. 5 after the parabolic approximation conversion is divided into blocks 607 shown in FIG.
On the other hand, as a result of calculating the position shift amount for each block 607 by the shift matching based on the normalized correlation with the measurement image, the position shift amount distribution shown in FIG. 6 is obtained. The size of the block used when determining the corresponding point group (S102, S103, S104) and the size of the block 607 need not be the same. In the positional deviation amount distribution shown in FIG. 6, a + mark indicates a corresponding point 606, and the closer the block 607 is to white, the larger the positional deviation amount is. As a result of shifting for each block using this positional shift amount, a reference image 701 after shift matching shown in FIG. 7 is obtained.

Next, a description will be given of the color correction processing S108 based on the RGB value conversion after the alignment between the images. Since shading of the fundus image is not uniform, in the present invention, color correction by RGB value conversion is performed for each block 607 used at the time of positioning. RGB for each block alone
Since the continuity after the conversion is not maintained when the value conversion is performed, the RGB value conversion coefficients are obtained by using the adjacent eight blocks in the vicinity. The conversion equation is expressed by the following equation.

[0043]

(Equation 3) Here, S _{i, k} : the i-th coordinate point in the k-th block on the reference image after alignment Q _{i, k} : the i-th coordinate point in the k-th block on the measurement image C [S _{i, k} ]: RGB value at S _{i, k} C [Q _{i, k} ]: RGB value at Q _{i, k} C ′ [S _{i, k} ]: RG at S _{i, k} after RGB value conversion
B value CG _{S, k} : RG of the territory including the k-th block on the reference image after alignment and the eight blocks in the vicinity thereof
Center of gravity of B value (average value) CG _{Q, k} : Center of gravity of RGB value (average value) of region including k-th block and 8 blocks in the vicinity of the measurement image I: Brightness value (0.299R + 0.587G)
+ 0.114B)

Next, a description will be given of a time-dependent change presence / absence determination process S109 based on the Euclidean distance. As a result of performing the alignment and the RGB conversion in the combination of the reference image 801 with the change with time in FIG. 8 and the measurement image 902 with the change with time in FIG.
The difference in GB value is seen. Therefore, in the present invention, the presence or absence of a change with time is determined based on the Euclidean distance in the RGB space. This Euclidean distance E _{i, k} is represented by the following equation.

[0045]

(Equation 4)

In the distribution of the Euclidean distance E _{i, k} values shown in FIG. 10 obtained from the reference image 801 and the measurement image 902, it is shown that the Euclid distance amount E _{i, k} value is larger as approaching white. It can be confirmed that the value has increased in the aging portion 1008. Therefore, it is possible to determine whether there is a change with time by performing threshold processing on the Euclidean distance E _{i, k} value.

[0047]

As described above, by using the method for judging whether or not there is a change with time according to the present invention, the change with time between the past normal diagnosis image of the same examinee and the fundus image to be judged can be clarified. Therefore, it is possible for a doctor to promptly reject normal cases, which occupy a large part of the medical examination, and this has the effect of leading to the spread of fundus examination.

In addition, by individually determining whether or not there is a change with time with respect to not only two images of the same examinee but also a plurality of images taken at different times, the amount of change with time is obtained, and The effect is that the degree of progress or healing can be known.

[Brief description of the drawings]

FIG. 1 is a flowchart of a method for determining the presence / absence of a temporal change according to an embodiment of the present invention.

FIG. 2 is a photograph of a halftone image displayed on a display of a fundus image in the form of a living being, showing an example of extraction processing of a corresponding candidate point and an initial corresponding point. (A) is a reference image. (B) is a measurement image.

FIG. 3 is a photograph of a halftone image displayed on a fundus image display showing a corresponding point determination example. (A) is a reference image. (B) is a measurement image.

FIG. 4 is a schematic diagram showing an example of projection processing of a corresponding point on a paraboloid.

FIG. 5 is a photograph of a halftone image displayed on a fundus image display showing a reference image after parabolic approximation conversion.

FIG. 6 is a photograph of a halftone image displayed on a fundus image display showing a positional shift amount distribution for each block.

FIG. 7 is a photograph of a halftone image displayed on a fundus image display showing a reference image after a shift matching.

FIG. 8 is a photograph of a halftone image displayed on a display of a fundus image of a reference image in a temporal change determination.

FIG. 9 is a photograph of a halftone image displayed on a display of a fundus image of a measurement image in the determination as to whether or not there is a temporal change.

FIG. 10 is a photograph of a halftone image displayed on a fundus image display showing the distribution of Euclidean distance values between a reference image and a measurement image.

[Explanation of symbols]

Reference numerals 201, 301, 801 Reference images 202, 302, 902 Measurement images 203 Initial corresponding points 204 Blocks with the largest number of corresponding candidate points 205 Corresponding candidate points 306, 606 Corresponding points 401 Z-axis 402 x-axis 403 Image plane 404 Parabolic surface 405 Corresponding points 406 Corresponding point after projection 501 Reference image after parabolic approximation conversion 601 Distribution of displacement amount of reference image 607 Block 701 Reference image after shift matching 908, 1008 Time-varying portion S101 Input of measurement image and reference image S102 Correspondence candidate Point extraction S103 Initial correspondence point search and estimation of relative displacement between image impressions S104 Determination of correspondence candidate point group based on relative displacement S105 Projection of corresponding point group on paraboloid S106 Between corresponding point groups projected Estimation of geometric transformation coefficient S107 Compensation of block position shift by shift matching S108 aging existence determination based on the color correction S109 Euclidean distance by the RGB value conversion

Claims (5)

[Claims] 1. A method for determining whether or not there is a temporal change in a fundus image, a process of performing positioning by approximating a shape of an eyeball and a fundus of two types of fundus images to be compared with a paraboloid. Correcting the respective values in the RGB space between the fundus images after the alignment, and determining whether there is a temporal change using the Euclidean distance between the corrected fundus images. A method for determining the presence or absence of temporal changes in a fundus image. 2. In the process of performing positioning by approximating the shape of the eyeball and the fundus of the fundus image with a paraboloid, the blood vessel intersection, the blood vessel, and the nipple of the two fundus images to be compared. 2. The method according to claim 1, further comprising: extracting a feature point as a feature point, projecting each feature point on a paraboloid, performing geometric transformation between corresponding feature points, and performing alignment. A method for determining whether or not a fundus image has changed with time. 3. The step of performing the positioning after the parabolic surface approximation, wherein shifting matching is performed for each block of the fundus image divided into a predetermined size for each local region. The method according to claim 1 or 2, wherein the fundus image has a temporal change. 4. The method according to claim 1, wherein in the step of correcting the respective values in the RGB space, interpolation is performed by matching luminance values between the fundus images. 3. The method for judging the presence or absence of a temporal change in a fundus image according to the item. 5. A method of determining whether there is a temporal change using the Euclidean distance between the corrected fundus images, the method comprising the steps of:
The method according to any one of claims 1 to 4, wherein the determination is made based on the Euclidean distance between the images in the B space.