JP2010259629A - Image processor, control method for the same, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a mechanism for observing the progress depending on the past image, and setting an imaging area effective for diagnosis. <P>SOLUTION: A tomographic image acquiring section 101 acquires a first tomographic image of an eye portion imaged in the past. A first imaging area calculating section 104 calculates a first imaging area in a first tomographic image using the first tomographic image and an eyeground image imaged by an eyeground image imaging section 120. A position extracting section 103 for a significant region extracts the position of a significant section of the eye section from the eyeground image, and an imaging size acquiring section 105 acquires an imaging size to be imaged by a tomographic image imaging section 130. A second imaging area calculating section 108 calculates a second imaging area to image the eye portion by the tomographic image imaging section 130 based on the information on the first imaging area, the information on the position of the significant region and the information on the imaging size. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus that processes an image obtained by photographing a subject, a control method thereof, a program for causing a computer to execute the control method, and a computer-readable storage that stores the program It relates to the medium. In particular, it is suitable for use in processing an image obtained by photographing an eye part as a subject.

  For example, in diagnostic imaging in ophthalmology, in addition to fundus images that have been widely used in the past, in recent years tomographic images have been used to three-dimensionally observe the internal state of the retina layer in the eye of the subject. Is possible. Therefore, diagnosis using tomographic images is expected to be useful for more accurately diagnosing diseases.

  The above-described fundus image is captured using, for example, a fundus camera, a scanning laser ophthalmoscope (SLO), IR, or the like. Further, the above-described tomographic image is taken using an optical coherence tomography (Optical_Coherence_Tomography: OCT) or the like.

FIG. 6 is a schematic diagram illustrating an example of a tomographic image 600 of the macular region in the retina of the eye, which is captured using OCT.
A tomographic image 600 of the eye by OCT is obtained three-dimensionally. In FIG. 6, cross-sectional images T _{1 to} T _n are two-dimensional cross-sectional images (B-scan images) of the macular portion.

FIG. 7 is a schematic diagram showing an example of an integrated image 700 created by integrating the cross-sectional images T _{1 to} T _n shown in FIG. 6 in the depth direction.
Here, the depth direction is the z direction in FIG. 6, and the process of integrating in the depth direction is a process of adding the light intensity (luminance value) at each depth position in the z direction of FIG. 6.

FIG. 8 is a schematic diagram illustrating an example of a fundus image 800.
The tomographic image projection area 803 of the macular portion is an area corresponding to the tomographic image 600 in the fundus image 800 by projecting the tomographic image 600 shown in FIG.

  As described above, in the tomographic image, only a narrow region in the fundus portion can be imaged compared to the fundus image, and it is impossible to image both the optic disc 801 and the macula 802 shown in FIG. It is common.

  Conventionally, for example, the following Patent Document 1 proposes a technique for photographing a specific position of the fundus in a tomographic imaging apparatus. Further, for example, in Patent Document 2 below, there is a technique for setting a lesion portion captured in the past and setting the position, size, exposure, etc. of a region of interest at the next imaging in the fundus image capturing apparatus. Proposed.

JP 2007-185244 A Japanese Patent Laid-Open No. 2001-2751979

  However, the conventional techniques described above have the following problems.

  In Patent Document 1, only the use of tomographic images captured by the same tomographic imaging apparatus is considered. Therefore, for example, when the performance of the tomographic imaging apparatus is improved and the imageable area is enlarged, the imaging area cannot be determined and the operator needs to specify the imaging area. In this case, the previous imaging area and the subsequent imaging area are shifted from each other, making it difficult to perform follow-up observation. In addition, for example, an imaging region cannot be determined in consideration of important parts of the eye, such as the optic nerve head and the macula.

  In addition, as described above, Patent Document 2 discloses the position, size, and size of a region of interest at the next shooting according to the setting of the region of interest of an image captured in the past (hereinafter referred to as “past image”). It has been proposed to set shooting conditions such as exposure. However, in Patent Document 2, since the photographing target is a fundus image, the imageable region is wide in the first place, and it is not considered that the imageable region is enlarged. In Patent Document 2, when the performance of the photographing apparatus is improved and a wider area can be photographed, setting an effective photographing area is not considered.

  The present invention has been made in view of such problems, and provides a mechanism that enables follow-up observation by comparison with a past image and enables setting of an imaging region effective for diagnosis. Objective.

  An image processing apparatus according to the present invention includes a tomographic image acquisition unit configured to acquire a first tomographic image of a subject photographed in the past, and a first photographing region that is a photographing region of the first tomographic image. Storage means for storing a wide-area image relating to a wide imaging area in an image storage unit; and first imaging area calculation means for calculating the first imaging area using the first tomographic image and the wide-area image; A position extracting unit for extracting an important part of the subject from the wide image, a tomographic image taking unit for taking a tomographic image of the subject in a second imaging region, and the tomographic imaging Based on imaging size acquisition means for acquiring imaging size information that can be imaged by the means, information on the first imaging area, information on the position of the important part, and information on the imaging size, the second Calculate shooting area And a second imaging area calculating unit that.

  An image processing apparatus control method according to the present invention is an image processing apparatus control method for processing an image, wherein a tomographic image acquisition step of acquiring a first tomographic image of a subject photographed in the past; A storage step of storing, in an image storage unit, a wide range image related to a wide range of shooting areas of the subject including a first shooting area that is a shooting area of the tomographic image, and using the first tomographic image and the wide range image A first imaging area calculating step for calculating the first imaging area; an important part position extracting step for extracting an important part position of the subject from the wide image; and a second of the subject. An imaging size acquisition step for acquiring imaging size information that can be imaged by the tomographic image imaging means from a tomographic imaging means for imaging a tomographic image in the imaging area; information on the first imaging area; Information-position, and, based on information of the image size, and a second imaging area calculating step of calculating the second imaging area.

The program of the present invention is for causing a computer to function as each means of the image processing apparatus.
Another aspect of the program of the present invention is for causing a computer to execute each step of the control method of the image processing apparatus.

  The computer-readable storage medium of the present invention stores the program.

  ADVANTAGE OF THE INVENTION According to this invention, the mechanism which enables the follow-up observation by the comparison with a past image and the setting of the imaging region effective for a diagnosis can be provided.

1 is a schematic diagram illustrating an example of a schematic configuration of an eye image capturing system (image processing system) according to an embodiment of the present invention. It is a schematic diagram which shows an example of an internal structure of the 1st imaging | photography area | region calculation part shown in FIG. It is a flowchart which shows an example of the process sequence in the control method of the eye part imaging device (image processing device) concerning the embodiment of the present invention. FIG. 4 is a flowchart illustrating an example of a detailed processing procedure of a first imaging region position calculation process in step S <b> 103 of FIG. 3. FIG. It is a schematic diagram which shows an example of the hardware constitutions applicable to the image photography control part of FIG. It is a schematic diagram which shows an example of the tomographic image of the macular part in the retina of an eye part image | photographed using OCT. FIG. 7 is a schematic diagram showing an example of an accumulated image created by integrating the cross-sectional images T _{1 to} T _n shown in FIG. 6 in the depth direction. It is a schematic diagram which shows an example of a fundus image.

  Hereinafter, embodiments (embodiments) for carrying out the present invention will be described with reference to the drawings. In the embodiment described below, an example in which an eye image capturing device that captures an image in the eye of a subject is applied as the image processing device according to the present invention will be described. In the embodiment described below, an example in which an eye part is applied as a subject will be described. However, the present invention is not limited to this example. For example, any object can be used as long as it is a tomographic image capturing target. A subject can also be applied. Further, the embodiment described below is merely an example, and the present invention is not limited to the illustrated configuration.

FIG. 1 is a schematic diagram illustrating an example of a schematic configuration of an eye image capturing system (image processing system) 10 according to an embodiment of the present invention.
As shown in FIG. 1, the eye image capturing system 10 includes an eye image capturing device (image processing device) 100, a LAN 200, and an image database 300.

  The eye image capturing apparatus 100 is connected via the LAN 200 to an image database 300 that stores past images. That is, the eye image capturing apparatus 100 is configured to acquire a past image from the image database 300 and to store an image captured in the image database 300 as a past image.

  As shown in FIG. 1, the eye image capturing apparatus 100 includes an image capturing control unit 110, a fundus image capturing unit 120, and a tomographic image capturing unit 130.

  In addition, as shown in FIG. 1, the image capturing control unit 110 includes a tomographic image acquisition unit 101, an image storage unit 102, a first imaging region calculation unit 104, an imaging size acquisition unit 105, an imaging error acquisition unit 106, and A third imaging area calculation unit 107 is included. Further, as shown in FIG. 1, the image capturing control unit 110 includes an important part position extracting unit 103, a second imaging region calculating unit 108, an imaging instruction unit 109, an image information output unit 111, and an image display unit 112. It is comprised.

  The fundus image photographing unit 120 photographs a fundus image in the eye part of the subject using, for example, a scanning laser ophthalmoscope (Scanning_Laser_Ophthalmoscope: SLO). In this example, the fundus image capturing unit 120 captures a fundus image using SLO. However, for example, the fundus image capture unit 120 may capture a fundus image using IR or the like.

  The tomographic image capturing unit 130 captures a tomographic image of the subject's eye. In this example, the tomographic image capturing unit 130 captures a tomographic image using the spectral domain method of OCT. However, the tomographic image may be captured using another method.

  The fundus image capturing unit 120 and the tomographic image capturing unit 130 are connected to each other and, for example, capture a tomographic image while always capturing a fundus image by a known method disclosed in Patent Document 1 or the like. It is configured to be able to. Specifically, it is configured such that a tomographic image at an arbitrary position on the fundus image can be captured by controlling the positional relationship of lasers used for capturing each image.

  The tomographic image acquisition unit 101 acquires past images from the image database 300 and stores them in the image storage unit 102. Here, the tomographic image acquisition unit 101 sets the past tomographic image acquired from the image database 300 as the first tomographic image. Note that, as the first tomographic image, for example, an image captured by a tomographic image capturing unit other than the tomographic image capturing unit 130 can also be applied.

  The image storage unit 102 stores the past image acquired by the tomographic image acquisition unit 101, and stores and stores the fundus image captured by the fundus image capturing unit 120 and the tomographic image captured by the tomographic image capturing unit 130. To do.

  The important part position extraction unit 103 takes in the image stored in the image storage unit 102, extracts the positions of important parts such as the optic nerve head and the macula, and uses this information as the second imaging region calculation unit 108. Output to.

  The first shooting area calculation unit 104 calculates the position of the shooting area (first shooting area) of the past image from the image stored in the image storage unit 102, and uses this information as the third shooting area calculation unit 107. Output to.

  The imaging size acquisition unit 105 acquires information on the maximum imaging size that can be imaged by the tomographic image imaging unit 130 from the tomographic image imaging unit 130, and uses this information as a third imaging area calculation unit 107 and a second imaging area calculation. Output to the unit 108.

  The imaging error acquisition unit 106 acquires an error of the imaging position that occurs when the tomographic image imaging unit 130 performs imaging, and outputs this information to the third imaging area calculation unit 107.

  The third imaging area calculation unit 107 uses the information from the imaging size acquisition unit 105 and the imaging error acquisition unit 106 to use the information about the surrounding area (first area) output from the first imaging area calculation unit 104. 3) is calculated. Then, the third imaging area calculation unit 107 outputs information related to the position of the third imaging area to the second imaging area calculation unit 108.

  The second imaging region calculation unit 108 includes the position information of the important part from the position extraction unit 103 of the important part, the information on the maximum imaging size from the imaging size acquisition unit 105, and the third information from the third imaging region calculation unit 107. The position of the second shooting area is calculated using the position information of the shooting area. Here, the second imaging area is an imaging area where the tomographic image imaging unit 130 actually performs imaging.

  The imaging instruction unit 109 transmits the position information of the second imaging area calculated by the second imaging area calculation unit 108 to the tomographic image imaging unit 130, and transmits the second imaging area to the tomographic image imaging unit 130. An instruction to take a tomographic image is issued. Thereby, the tomographic image capturing unit 130 captures a tomographic image (second tomographic image) for the second imaging region, and the second tomographic image is stored in the image storage unit 102.

  The image information output unit 111 acquires the second tomographic image stored in the image storage unit 102 and outputs the second tomographic image to the image database 300 via the LAN 200.

  The image display unit 112 displays a fundus image, a second tomographic image, and the like stored in the image storage unit 102.

Next, the internal configuration of the first imaging area calculation unit 104 shown in FIG. 1 will be described.
FIG. 2 is a schematic diagram illustrating an example of an internal configuration of the first imaging region calculation unit 104 illustrated in FIG. In addition to the internal configuration of the first imaging area calculation unit 104, FIG. 2 also illustrates an image storage unit 102 and a third imaging area calculation unit 107 connected to the first imaging area calculation unit 104. .

  As shown in FIG. 2, the first imaging region calculation unit 104 includes an integrated image creation unit 1041, a tomographic image feature extraction unit 1042, a fundus image feature extraction unit 1043, and an alignment unit 1044. Yes. Each of these components will be described together in the description using the flowchart of FIG.

  Next, the flow of processing in the control method of the eye image capturing apparatus 100 will be described.

  FIG. 3 is a flowchart showing an example of a processing procedure in the control method of the eye image capturing device (image processing device) 100 according to the embodiment of the present invention.

<Step S101>
First, in step S <b> 101, the tomographic image acquisition unit 101 performs processing for acquiring a first tomographic image stored in the image database 300. Specifically, the tomographic image acquisition unit 101 acquires a first tomographic image of the subject to be imaged from the first tomographic images stored in the image database 300, and this is acquired as an image storage unit. Save to 102. At this time, the tomographic image acquisition unit 101 acquires information about the eye to be examined in the first tomographic image held in the image database 300 and stores this information in the image storage unit 102. Here, the information on the eye to be examined is information such as the name, age, and sex of the subject (patient) and whether the examination target is the right eye or the left eye.

<Step S102>
Subsequently, in step S <b> 102, the image capturing control unit 110 performs processing for acquiring the fundus image captured by the fundus image capturing unit 120 from the fundus image capturing unit 120 and storing the fundus image in the image storage unit 102. Here, the fundus image photographed by the fundus image photographing unit 120 relates to a wide photographing region of the eye part including the first photographing region which is the photographing region of the first tomographic image obtained by the tomographic image obtaining unit 101. It corresponds to a wide image. The image capturing control unit 110 that performs processing for storing the fundus image in the image storage unit 102 constitutes a storage unit.

<Step S103>
Subsequently, in step S103, the first imaging region calculation unit 104 uses the fundus image and the first tomographic image stored in the image storage unit 102 to capture the first tomographic imaging region (the first tomographic image). The processing for calculating the position of the shooting area is performed. Specifically, for example, an area corresponding to the first tomographic image is calculated as the first imaging area from the imaging area of the fundus image. Then, the first shooting area calculation unit 104 outputs the position information of the first shooting area to the third shooting area calculation unit 107.

  FIG. 8 is a schematic diagram of a fundus image as described above, and the first imaging region 803 is a corresponding region in the fundus image of the fundus image. That is, in the example of FIG. 8, the first imaging region calculation unit 104 calculates the position of the first imaging region 803 in the fundus image. In the present embodiment, the position of the tomographic image is defined by the position of the first tomographic image in the fundus image. In this manner, by using the fundus image, it is possible to accurately perform processing related to an important part that is not included in the first tomographic image. Details of the processing in step S103 will be described later with reference to FIG.

<Step S104>
Subsequently, in step S104, the position extraction unit 103 of the important part extracts the position of the important part from the fundus image stored in the image storage unit 102, and uses this information as the second imaging region calculation unit 108. Output to.

  In this embodiment, an example of extracting (detecting) the position of the macular portion of the eye and the position of the optic nerve head as an important part will be described. However, various important parts of the eye can be considered as targets to be detected. It is not limited to the extraction of the macular portion and the optic papilla shown in FIG.

Here, details of the processing in step S104 will be described.
First, the position extraction unit 103 of the important part extracts blood vessels from the fundus image. At this time, since the blood vessel has a thin linear structure, the position extraction unit 103 of the important part extracts the blood vessel using a filter that emphasizes the linear structure. Here, as a filter for emphasizing a linear structure, for example, when a line segment is a structural element, a difference between an average value of image density values in the structural element and an average value in a local region surrounding the structural element is calculated. Use a filter to calculate. However, the present invention is not limited to this filter, and a differential filter such as a Sobel filter may be used. Alternatively, the eigenvalue of the Hessian matrix may be calculated for each pixel of the density value image, and a line segment area (that is, blood vessel) may be extracted from the combination of the two eigenvalues obtained as a result. Furthermore, a top hat calculation may be performed simply using a line segment as a structural element.

  In the present embodiment, the optic nerve head, which is an important part, has blood vessels, and the brightness of the image is higher than that of the peripheral part. Therefore, when extracting the optic nerve head, the position extraction unit 103 of an important part pays attention to such image features and performs extraction.

  In addition, the macular portion as an important site in the present embodiment is in the center of the fundus, and the optic nerve head as an important site in the present embodiment is closer to the center of the face than the macula, that is, closer to the nose, Blood vessels gathered in the optic nerve head run so as to surround the macula. Therefore, the position extraction unit 103 of the important part performs the extraction while paying attention to such an anatomical feature when extracting the macula and the optic nerve head.

  In the present embodiment, the position extraction unit 103 of the important part extracts the positions of the optic nerve head and the macula, and outputs the position information to the second imaging region calculation unit 108. In the images of FIGS. 7 and 8, the first tomographic image includes the macular portion 802 but does not include the optic papilla 801, and has the optic papilla 801 in the -x direction. . As described above, information on whether or not an important part is included in the first tomographic image and in which direction the important part not included is relatively included from the part obtained by capturing the tomographic image. Also, the position is extracted from the position extraction unit 103 of the important part to the second imaging region calculation unit 108.

<Step S105>
Subsequently, in step S105, the imaging size acquisition unit 105 acquires size information of an area where the tomographic image (second tomographic image) can be imaged from the tomographic image imaging unit 130, and calculates the third imaging area calculation. Output to the unit 107 and the second imaging region calculation unit 108.

  Here, the imageable area of the tomographic image imaging unit 130 can be expressed as the sizes in the x and y directions in FIG. In the present embodiment, it is assumed that the photographic area of the tomographic image photographing unit 130 is larger than the photographic area in the first tomographic image. In addition, regarding the position and size in the z direction, that is, the thickness direction of the retina, it is generally possible to measure a sufficient thickness with the current tomographic imaging unit. Therefore, the entire retinal layer can be imaged by adjusting the position of the retina in the depth direction by automatic adjustment of a coherence gate generally performed in a conventional tomographic imaging unit. At this time, the coherence gate may be adjusted manually.

<Step S106>
Subsequently, in step S106, the third shooting area calculation unit 107 detects the position information of the first shooting area from the first shooting area calculation unit 104, the shootable size information from the shooting size acquisition unit 105, and the shooting error. Based on the photographing error information from the acquisition unit 106, the position of the third photographing region, which is a surrounding region in the first photographing region, is calculated. Here, the third imaging area is calculated so as to have an area having a width equal to or larger than the imaging error around the first imaging area. Then, the third shooting area calculation unit 107 outputs position information of the third shooting area in consideration of shooting errors to the second shooting area calculation unit 108.

Here, a specific example of the process of step S106 will be described.
For example, the position of the first imaging area is (x1, y1), and the size of the first imaging area is (x1w, y1w).
Assume that the imaging error is (xd, yd) from the position specified in the tomographic imaging unit 130, the position of the third imaging region 804 considering the imaging error is (xr, yr), and the size is (xrw, yrw). ). Then, the position (xr, yr) and the size (xrw, yrw) of the third imaging region 804 are expressed by the following equations (1), (2), (3), and (4). Can be calculated.
xr = x1-xd (1)
yr = y1-yd (2)
xrw = x1w + xd * 2 (3)
yrw = y1w + yd * 2 (4)

  That is, an imaging region (third imaging region) for absorbing an error of xd in the x direction and yd in the y direction is secured around the first imaging region. If the third imaging region 804 considering the imaging error is indicated as an imaging region to the tomographic image imaging unit 130, a tomographic image including the same region as the past image is obtained even if there is an error in imaging. Is possible. Note that if the imaging error due to the positional shift in the tomographic imaging unit 130 can be ignored in the image diagnosis, the third imaging region calculation unit 107 uses the positional information input from the first imaging region calculation unit 104. The data is output to the second imaging area calculation unit 108 as it is.

<Step S107>
Subsequently, in step S <b> 107, the second imaging area calculation unit 108 first acquires position information of the third imaging area that should be included from the third imaging area calculation unit 107. Similarly, the second imaging area calculation unit 108 acquires the size information (maximum imaging size information) that can be imaged from the imaging size acquisition unit 105, and adds the important area to the actual imaging area from the important part position extraction unit 103. Acquire location information of the part. Then, using the acquired information, the second imaging area calculation unit 108 calculates the position of the second imaging area where the tomographic image imaging unit 130 actually performs imaging, and uses this information as the imaging instruction unit 109. Output to.

Here, a specific example of the process of step S107 will be described.
The second imaging region calculation unit 108 enlarges the third imaging region (804 in FIG. 8) from the third imaging region calculation unit 107 in the direction of the important part not included in the first tomographic image. Then, the position of the second imaging region is calculated. For example, when the size that can be captured by the tomographic image capturing unit 130 acquired by the capturing size acquisition unit 105 is (x2w, y2w), the size in the x direction and the y direction (xe, ye) that can be expanded is as follows. It can be calculated by the equations (5) and (6).
xe = x2w−xrw (5)
ye = y2w−yrw (6)

In the example of the first tomographic image in the present embodiment, among the important parts, the macular portion is included but the optic papilla is not included. Therefore, it is detected here that the optic papilla is in the −x direction. Has been. Therefore, the second imaging area calculation unit 108 extends the imaging area in the positive and negative directions in the y direction and extends the imaging area in the negative direction in the x direction with respect to the third imaging area. Then, the position of the second imaging region is calculated. Specifically, when the second imaging region is (x2, y2), it can be calculated by the following equations (7) and (8).
x2 = xr−xe (7)
y2 = yr− (ye / 2) (8)

  Here, in the fundus image shown in FIG. 8, the position of the imaging region corresponding to the portion of the second imaging region 805 is calculated.

<Step S108>
Subsequently, in step S108, the imaging instruction unit 109 controls the tomographic image imaging unit 130 and the fundus image imaging unit 120 to correspond to the eye corresponding to the second imaging area calculated by the second imaging area calculation unit 108. The image of the area is taken. By the processing in step S108, the tomographic image captured by the tomographic image capturing unit 130 forms a second tomographic image.

  In the present embodiment, the tomographic image capturing unit 130 is linked to the fundus image capturing unit 120 and captures a tomographic image at a specific position in the fundus image, for example, by the method disclosed in Patent Document 1. It is configured to be able to. Further, the tomographic image capturing unit 130 may perform tomographic image positioning by any method as long as a tomographic image at an arbitrary position in the fundus image can be captured.

  Next, detailed processing of the position calculation processing of the first imaging region in step S103 of FIG. 3 will be described.

  FIG. 4 is a flowchart illustrating an example of a detailed processing procedure of the position calculation process of the first imaging region in step S103 of FIG. Hereinafter, the flowchart of FIG. 4 will be described together with the description of each component of the first imaging region calculation unit 104 shown in FIG.

<Step S201>
The accumulated image creation unit 1041 performs the cross-sectional image (for example, B-scan image) in the depth direction in order to align the fundus image stored in the image storage unit 102 and the tomographic image (first tomographic image). A process of creating an integrated image integrated in the above is performed. Then, the accumulated image creation unit 1041 outputs the created accumulated image to the tomographic image feature extraction unit 1042.

As described above, FIG. 6 shows cross-sectional images T _{1 to} T _n of the macula in the retina of the eye, and FIG. 7 shows cross-sectional images T _{1 to} T of FIG. 6 as described above. _The accumulated image created from _n is shown. Here, the depth direction is the z direction in FIG. 6, and the process of integrating in the depth direction is a process of adding the light intensity (luminance value) at each depth position in the z direction of FIG. If the entire retinal layer can be detected in advance, only an arbitrary layer within the retinal layer may be added in step S201 to generate a clearer integrated image.

  Here, the curve in the integrated image in FIG. 7 indicates a blood vessel, and the circle at the center of the image indicates a macular portion. In a fundus tomographic image, it is known that the reflected light intensity of light at a position deeper than the blood vessel tends to be weak in a place where there is a blood vessel, and the value integrated in the z direction is smaller than the place where there is no blood vessel. Yes. Therefore, by creating an integrated image, it is possible to obtain an image having a contrast between the blood vessel and the rest.

<Step S202>
Subsequently, in step S202, the tomographic image feature extraction unit 1042 and the fundus image feature extraction unit 1043 perform processing for extracting features of the first tomographic image and fundus image, respectively.

  The tomographic image feature extraction unit 1042 performs feature extraction on the accumulated image of the first tomographic image created by the accumulated image creation unit 1041 and outputs this information to the alignment unit 1044.

  Specifically, the tomographic image feature extraction unit 1042 performs blood vessel extraction as feature extraction. At this time, since the blood vessel has a thin linear structure, the tomographic image feature extraction unit 1042 extracts the blood vessel using a filter that emphasizes the linear structure. Here, as a filter for emphasizing a linear structure, for example, when a line segment is a structural element, a difference between an average value of image density values in the structural element and an average value in a local region surrounding the structural element is calculated. Use a filter to calculate. However, the present invention is not limited to this filter, and a differential filter such as a Sobel filter may be used. Alternatively, the eigenvalue of the Hessian matrix may be calculated for each pixel of the density value image, and a line segment area (that is, blood vessel) may be extracted from the combination of the two eigenvalues obtained as a result. Furthermore, a top-hat operation using a line segment as a structural element may be used.

  On the other hand, the fundus image feature extraction unit 1043 performs feature extraction on the fundus image input from the image storage unit 102, and outputs this information to the alignment unit 1044.

Specifically, the fundus image feature extraction unit 1043 performs blood vessel extraction as feature extraction in the same manner as the tomographic image feature extraction unit 1042.
At this time, since the blood vessel has a thin linear structure, the fundus image feature extraction unit 1043 extracts the blood vessel using a filter that emphasizes the linear structure. Here, as a filter for emphasizing a linear structure, for example, when a line segment is a structural element, a difference between an average value of image density values in the structural element and an average value in a local region surrounding the structural element is calculated. Use a filter to calculate. However, the present invention is not limited to this filter, and a differential filter such as a Sobel filter may be used. Alternatively, the eigenvalue of the Hessian matrix may be calculated for each pixel of the density value image, and a line segment area (that is, blood vessel) may be extracted from the combination of the two eigenvalues obtained as a result. Furthermore, a top-hat operation using a line segment as a structural element may be used.

<Step S203>
Subsequently, in step S203, the registration unit 1044 performs registration based on the feature of the tomographic image extracted by the tomographic image feature extraction unit 1042 and the feature of the fundus image extracted by the fundus image feature extraction unit 1043. Perform the process. As a result, the position of the first imaging region is determined.

The result of the alignment process by the alignment unit 1044 is a first image including parameters of scale (S _x , S _y ), position coordinates (x, y), and rotation (rot) of the accumulated image using the fundus image as a reference image. The position information of one shooting area is output to the third shooting area calculation unit 107.

  In the alignment unit 1044, in order to align the positions of the images, in this embodiment, blood vessels are used as anatomically characteristic regions. Specifically, the registration unit 1044 performs registration between images using blood vessels extracted from the accumulated image and the fundus image. When this alignment is performed, an evaluation value representing the similarity between two images is defined in advance, and the image is deformed so that the evaluation value becomes the best. As the evaluation value, attention is paid to a value indicating the degree of overlap between the blood vessel region of the accumulated image and the blood vessel region of the fundus image obtained in the previous stage, or a region having a characteristic geometric shape such as a blood vessel bifurcation. The distance between corresponding landmarks can be used. In this embodiment, blood vessels are used as anatomically characteristic regions. However, other anatomical features such as the optic nerve head region, vitiligo and bleeding regions caused by diseases may be used. . Further, not only attention is paid to anatomical features such as blood vessels, but evaluation values calculated from the entire image, for example, mean square error of luminance values, correlation coefficient, mutual information amount, and the like can also be used.

Next, a hardware configuration applicable to the image capturing control unit 110 in FIG. 1 will be described.
FIG. 5 is a schematic diagram illustrating an example of a hardware configuration applicable to the image capturing control unit 110 in FIG.

  For example, as shown in FIG. 5, the image capturing control unit 110 has a hardware configuration of a CPU 501, a RAM 502, a ROM 503, an external storage device 504, a monitor 505, a keyboard 506, a mouse 507, an interface 508, and a bus 509. Configured.

  Here, the tomographic image acquisition unit 101, the imaging size acquisition unit 105, the imaging error acquisition unit 106, the imaging instruction unit 109, and the image information output unit 111 in FIG. 1 are, for example, programs of the CPU 501 and the external storage device 504 in FIG. The interface 508 is configured. The image storage unit 102 in FIG. 1 includes, for example, the RAM 502, the ROM 503, or the external storage device 504 in FIG. Further, the position extraction unit 103, the first imaging region calculation unit 104, the third imaging region calculation unit 107, and the second imaging region calculation unit 108 in FIG. 1 are, for example, the CPU 501 and the external storage in FIG. The program of the device 504 is configured. Further, the image display unit 112 in FIG. 1 includes, for example, the monitor 505 in FIG.

  The CPU 501 controls the entire image capturing control unit 110 using programs and data stored in the ROM 503 or the external storage device 504, for example.

  The RAM 502 includes an area for temporarily storing programs and data loaded from the ROM 503 or the external storage device 504, and a work area necessary for the CPU 501 to perform various processes.

  The ROM 503 generally stores a computer BIOS and setting data. The external storage device 504 is a device that functions as a large-capacity information storage device such as a hard disk drive, and stores an operating system (OS), a program executed by the CPU 501, and the like. Information known in the description of the present embodiment is stored in the external storage device 504 and loaded into the RAM 502 as necessary.

  The monitor 505 is composed of a liquid crystal display or the like, and displays various information and images based on the control of the CPU 501.

  A keyboard 506 and a mouse 507 are input devices, and an operator can give various instructions to the image capturing control unit 110 using these devices.

  The interface 508 transmits and receives various types of information and data between the image capturing control unit 110 and external devices (for example, the fundus image capturing unit 120, the tomographic image capturing unit 130, and the image database 300 via the LAN 200). Is for. For example, the interface 508 is configured by IEEE 1394, USB, Ethernet port (registered trademark), or the like. Various information and data acquired through the interface 508 are stored in the external storage device 504 and loaded into the RAM 502 as necessary. The bus 509 connects the above-described components (501 to 508) so that they can communicate with each other.

  According to the eye image capturing apparatus 100 of the present embodiment, when a past image exists and a wider area can be captured, follow-up observation by comparison with the past image is possible and effective for diagnosis. The shooting area can be set. When setting an imaging region effective for diagnosis, since a positional deviation error at the time of imaging is taken into consideration, the imaging region in the past image can be reliably captured.

(Other embodiments of the present invention)
1 constituting the eye image capturing apparatus 100 according to the embodiment of the present invention described above can be realized by the CPU 501 of the computer executing a program stored in the external storage device 504. Similarly, each step of FIG. 3 and FIG. 4 showing the control method of the eye image capturing device 100 can be realized by the CPU 501 of the computer executing a program stored in the external storage device 504. This program and a computer-readable recording medium storing the program are included in the present invention.

  In addition, the present invention can be implemented as, for example, a system, apparatus, method, program, storage medium, or the like. Specifically, the present invention may be applied to a system including a plurality of devices. You may apply to the apparatus which consists of one apparatus.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowcharts shown in FIGS. 3 and 4) for supplying the functions of the above-described embodiment is supplied directly or remotely to a system or apparatus. Including. The present invention also includes a case where the system or the computer of the apparatus is achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a flexible disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, and CD-RW. In addition, there are magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R), and the like.

  As another program supply method, a browser on a client computer is used to connect to an Internet home page. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, the present invention includes a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let The downloaded key information can be used to execute the encrypted program and install it on the computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  Further, the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instructions of the program, and the functions of the above-described embodiments are realized by the processing.

  The above-described embodiments are merely examples of implementation in carrying out the present invention, and the technical scope of the present invention should not be construed as being limited thereto. That is, the present invention can be implemented in various forms without departing from the technical idea or the main features thereof.

10: eye image capturing system (image processing system), 100: eye image capturing device (image processing device), 101: tomographic image acquisition unit, 102: image storage unit, 103: position extraction unit of important part, 104: First imaging area calculation unit, 105: imaging size acquisition unit, 106: imaging error acquisition unit, 107: third imaging area calculation unit, 108: second imaging area calculation unit, 109: imaging instruction unit, 110: Image capturing control unit 111: Image information output unit 112: Image display unit 120: Fundus image capturing unit 130: Tomographic image capturing unit 200: LAN 300: Image database

Claims (8)

A tomographic image acquisition means for acquiring a first tomographic image of a subject photographed in the past;
Storage means for storing, in an image storage unit, a wide-area image relating to a wide imaging area of the subject including the first imaging area that is the imaging area of the first tomographic image;
First imaging area calculation means for calculating the first imaging area using the first tomographic image and the broad image;
An important part position extracting means for extracting an important part position of the subject from the wide image;
Tomographic image photographing means for photographing a tomographic image in a second photographing region of the subject;
Imaging size acquisition means for acquiring imaging size information that can be captured by the tomographic image imaging means;
And second imaging area calculation means for calculating the second imaging area based on the information on the first imaging area, the information on the position of the important part, and the information on the imaging size. An image processing apparatus. The subject is an eye;
The image processing apparatus according to claim 1, wherein the position extraction unit of the important part calculates at least a position of the eye in the macula or optic papilla as the important part.   The second imaging region calculation means includes the first imaging region and an imaging region that is an enlarged imaging region in the direction of the important part not included in the first imaging region. The image processing apparatus according to claim 1, wherein the image processing apparatus calculates the area. An imaging error acquisition means for acquiring an imaging error of the tomographic image imaging means;
A third imaging area calculating means for calculating a third imaging area including the first imaging area and providing an area based on the imaging error around the first imaging area;
The second imaging region calculation means calculates the second imaging region using the information of the third imaging region, the information of the position of the important part, and the information of the imaging size. The image processing apparatus according to any one of claims 1 to 3. A control method for an image processing apparatus that performs image processing,
A tomographic image acquisition step of acquiring a first tomographic image of a subject photographed in the past;
A storage step of storing, in an image storage unit, a wide-area image relating to a wide imaging area of the subject including a first imaging area that is an imaging area of the first tomographic image;
A first imaging region calculation step of calculating the first imaging region using the first tomographic image and the wide-range image;
An important part position extracting step for extracting an important part position of the subject from the wide image; and
An imaging size acquisition step of acquiring information on imaging size that can be imaged by the tomographic image imaging means from the tomographic image imaging means for imaging the tomographic image in the second imaging area of the subject;
And a second imaging region calculation step of calculating the second imaging region based on the information on the first imaging region, the information on the position of the important part, and the information on the imaging size. A control method of the image processing apparatus.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 4.   A program for causing a computer to execute each step of the control method of the image processing apparatus according to claim 5.   A computer-readable storage medium storing the program according to claim 6 or 7.

Images