JP6732093B2 - Information processing apparatus, information processing apparatus control method, and computer program - Google Patents

Description

The present invention relates to an information processing device used for ophthalmological examination, a control method of the information processing device, and a computer program.

BACKGROUND ART Eye examinations are widely performed for the purpose of early diagnosis of lifestyle-related diseases and diseases that are the leading causes of blindness. A scanning laser ophthalmoscope (SLO) is known as an ophthalmologic apparatus utilizing the principle of a confocal laser microscope. The scanning laser ophthalmoscope is a device that raster-scans a laser that is measurement light on the fundus and obtains a plane image with high resolution and high speed from the intensity of the returned light. Hereinafter, an apparatus that captures such a planar image will be referred to as an SLO apparatus, and the captured planar image will be referred to as an SLO image.

In recent years, it has become possible to acquire an SLO image of the retina having a lateral resolution improved as compared with the conventional technique by increasing the beam diameter of the measurement light in the SLO device. However, with the increase in the beam diameter of the measurement light, in obtaining the SLO image of the retina, the S/N ratio and the resolution of the SLO image are deteriorated due to the aberration in the eye to be inspected. In order to solve the problem, an adaptive optics SLO device having an adaptive optics system in which the wavefront sensor measures the aberration in the eye to be examined in real time, and the aberration of the measurement light or the return light generated in the subject eye is corrected by the wavefront correction device Is being developed. This makes it possible to acquire an SLO image with high lateral resolution.

Such a high lateral resolution SLO image can be acquired as a moving image, and various biological information can be measured using this moving image. For example, in order to observe blood flow dynamics non-invasively, retinal blood vessels are extracted from each frame, and then the moving speed of blood cells in capillaries is measured. Further, in order to evaluate the relationship with the visual function using the SLO image, the density distribution and arrangement of the photoreceptor cells P are measured after detecting the photoreceptor cells P.

However, since the angle of view of one high lateral resolution SLO image that can be captured by the adaptive optics SLO device is generally small, when the imaging target area is larger than the angle of view of the high lateral resolution SLO image, the imaging area in the imaging target area is large. How to set is a problem. This will be described with reference to FIG. FIG. 7A is a diagram schematically showing a cross-sectional view of the subject's eye. 7B to 7G are diagrams showing examples of the SLO image or the imaging target area.

FIG. 7B is a diagram showing an example of a high lateral resolution SLO image. In FIG. 7B, a low-luminance region Q corresponding to the positions of photoreceptor cells P and capillaries and a high-luminance region W corresponding to the positions of white blood cells are observed. When observing the photoreceptor cells P or measuring the distribution of photoreceptor cells P, the focus position is set near the outer layer of the retina (B5 in FIG. 7A) and an SLO image as shown in FIG. 7B is captured. To do.

On the other hand, retinal blood vessels and branched capillaries run in the inner layer of the retina (B2 to B4 in FIG. 7A). In particular, in the affected eye, the imaging target area is often larger than the angle of view of one SLO image that can be imaged by the SLO device. FIG. 7C and FIG. 7D show an example in which the imaging target area is larger than the angle of view of the SLO image. FIG. 7(c) shows an example of a common site of a capillary lesion (an annular region surrounded by a broken line), and FIG. 7(d) shows an example of a wide photoreceptor cell loss region (black closed region). There is. In the case of FIG. 7C and FIG. 7D, if all the imaging target areas are acquired at high magnification, the setting of the imaging conditions for a large number of SLO images is complicated, or the imaging time becomes long and There was a problem that the burden increased. The region to be imaged includes a region in which it is highly necessary to acquire a high-magnification image for medical treatment and a region in which it is not necessary to acquire a high-magnification image, and thus a high-magnification image is acquired in an examination time that does not burden the subject. It is necessary to appropriately set the image pickup area so that all the necessary areas can be picked up.

In connection with this, Patent Document 1 discloses a technique related to parameter setting for acquiring a plurality of high-magnification images, in which a plurality of adaptive optics SLO images are captured at different imaging positions and displayed as a panoramic image. Have been described.

JP, 2012-213513, A

However, when the cell group, tissue, or lesion area to be observed or measured in detail is distributed more widely than the area covered by the image with high lateral resolution (high-magnification image D _H ), the area such as the cell group can be efficiently distributed. When capturing an image, the conventional configuration has the following problems. That is,
i) It is necessary for the operator to individually specify the values of the acquisition parameters (for example, the acquisition position, the angle of view, the pixel size, the number of frames, the frame rate, etc.) of the multiple high-magnification images D _Hj , and the efficiency of acquiring multiple images is high. It was an obstacle to the change.
ii) When an observation target area wider than the high-magnification image _DH is imaged with the same high-magnification image acquisition parameter, the number of high-magnification images to be acquired (total number of frames) becomes enormous (thousands to tens of thousands) and the efficiency is increased. It was difficult to obtain a specific image.

In the configuration of Patent Document 1, too, the acquisition parameters of a large number of high-magnification images are manually determined for each image, and the operator is forced to perform complicated work for setting the acquisition parameters.

The present invention has been made in view of the above problems, and it is possible to generate an image of blood vessel information of an imaging target area in which blood vessel information of each imaging area acquired based on the characteristics of an image regarding an area where blood cells have moved is combined. The purpose is to provide technology that can.

In order to achieve the above object, the information processing apparatus according to the present invention has the following configuration. That is,
Each of the plurality of imaging regions divided so as to partially imaging target area of the eye overlaps an image acquisition means for acquiring a plurality of images of each imaging area captured a plurality of times,
Using a plurality of images of each of the imaging area, and the information obtaining means for obtaining blood vessel information on the blood vessel of each of the imaging area,
And a generating unit that generates an image of the blood vessel information of the imaging target area based on the acquired blood vessel information of each imaging area .

According to the present invention, an image of blood vessel information of an imaging target area can be easily generated by synthesizing blood vessel information for each imaging area acquired based on the characteristics of the image regarding the area where blood cells have moved .

FIG. 3 is a block diagram showing a configuration example of a system including the ophthalmologic apparatus 10. FIG. 3 is a block diagram showing a hardware configuration example of the ophthalmologic apparatus 10. FIG. 3 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. The figure explaining the whole structure of the SLO image pick-up device 20. The flowchart of the process which the ophthalmologic apparatus 10 performs. The figure explaining an image acquisition pattern. The figure explaining the image processing content. The flowchart which shows the detail of a high-magnification image acquisition process. The flowchart which shows the detail of an image display process. FIG. 3 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. The flowchart which shows the detail of a high-magnification image acquisition process. The figure explaining the exceptional frame contained in an image acquisition pattern and a high-magnification moving image. FIG. 3 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. The figure explaining the whole structure of the tomographic imaging device 60. The figure explaining an image acquisition pattern. The flowchart which shows the detail of an image display process.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<<Embodiment 1>>
When acquiring a plurality of high-magnification adaptive optical SLO images, the ophthalmologic apparatus as the information processing apparatus according to the present embodiment operates a basic pattern of parameters that are prepared in advance and related to imaging conditions for acquiring a plurality of high-magnification images. It is presented to the operator (user) so that the operator can select it. Next, if necessary, the operator is made to adjust the image acquisition parameters according to the lesion shape, and the acquisition parameter values for the plurality of high-magnification images are determined according to the contents of the adjustment. Hereinafter, the operator selects a basic pattern for acquiring a plurality of images in a disk shape for a wide range of photoreceptor cell loss regions of the macula, and the acquisition position, the angle of view, the pixel size, the number of frames, and the frame of each high-magnification image are selected. The case of determining the rate and the focus position will be described as an example.

(overall structure)
FIG. 1A is a configuration diagram of a system including an ophthalmologic apparatus 10 according to this embodiment. As shown in FIG. 1A, the ophthalmologic apparatus 10 includes an SLO image pickup device 20 as an image pickup device, a data server 40, and a local area network (LAN) 30 including an optical fiber, USB, IEEE1394, or the like. Connected through. The connection form of these devices is not limited to the example of FIG. For example, these devices may be connected via an external network such as the Internet, or the ophthalmologic apparatus 10 may be directly connected to the SLO image capturing apparatus 20.

The SLO image capturing device 20 is a device that captures (captures) a wide-angle image D _L and a high-magnification image D _H of the fundus. SLO image capturing apparatus 20, wide field of view image D _L and the high magnification image D _H, and fixation target position F _L used in the time of imaging, information F _H, and transmits to the ophthalmic device 10 and data server 40.

In addition, when acquiring the image of each magnification in a different imaging position, the acquired image is represented as D _Li and D _Hj . That is, i and j are variables indicating the imaging position number, and i=1, 2,. ． ． , Imax, j=1, 2,. ． ． , Jmax. Further, when the high-magnification images are acquired at a plurality of different magnifications, D _1j , D _{2k ,.} ． ． And the highest magnification D _1j is represented by a high magnification image, D _2k,. ． ． Is referred to as an intermediate magnification image.

The data server 40 holds imaging condition data, eye image features, normal values regarding the distribution of eye image features, and the like. As the imaging condition data, output by the SLO image capturing apparatus 20, a fixation target position F _L, F _H, ophthalmologic apparatus 10 outputs using wide-angle image D _L of the eye, the high magnification image D _H, the time of imaging The image features of the eye are stored in the data server 40. In the present embodiment, image features relating to photoreceptor cells P, capillaries Q, blood cells W, retinal vessels and retinal layer boundaries are handled as image features of the eye part. Further, in response to a request from the ophthalmologic apparatus 10, the data server 40 transmits the wide-angle image D _L , the high-magnification image D _H , the image feature of the eye part, and the normal value data of the image feature to the ophthalmologic device 10.

(Ophthalmology device)
The ophthalmologic apparatus 10 is realized by an information processing apparatus such as an embedded system, a personal computer (PC), or a tablet terminal. The hardware configuration of the ophthalmologic apparatus 10 will be described with reference to FIG. In FIG. 2, a CPU 301 is a central processing unit, which cooperates with other components based on a computer program such as an operating system (OS) and application programs to control the operation of the entire ophthalmologic apparatus. The RAM 302 is a writable memory and functions as a work area or the like of the CPU 301. The ROM 303 is a read-only memory and stores programs such as a basic I/O program and data used for basic processing. The external storage device 304 is a device that functions as a large capacity memory, and is realized by a hard disk device and a semiconductor memory. The monitor 305 is a display device as a display unit that displays a command input from the keyboard 306 or the pointing device 307, a response output of the ophthalmologic apparatus 10 in response to the command, and the like. The keyboard 306 and the pointing device 307 are devices that receive instructions and command inputs from the operator. The interface 308 is a device that relays data exchange with an external device.

A control program for realizing the image processing function according to the present embodiment and data used when the control program is executed are stored in the external storage device 304. Under the control of the CPU 301, these control programs and data are appropriately loaded into the RAM 302 via the bus 309, executed by the CPU 301, and function as each unit described below.

Next, the functional configuration of the ophthalmologic apparatus 10 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram showing a functional configuration of the ophthalmologic apparatus 10. As illustrated in FIG. 3, the ophthalmologic apparatus 10 includes a data acquisition unit 110, a storage unit 120, an image processing unit 130, and an instruction acquisition unit 140.

The data acquisition unit 110 is a functional block that acquires data such as image data and imaging condition data. The data acquisition unit 110 includes a wide-angle image acquisition unit 111 that acquires a wide-angle image and a high-magnification image acquisition unit 112 that acquires a high-magnification image. The storage unit 120 is a functional block that holds the data acquired by the data acquisition unit 110 and the image acquisition pattern set 121. The image acquisition pattern set 121 is a set of basic setting patterns (hereinafter referred to as “image acquisition patterns”) regarding parameters when acquiring a plurality of high-magnification images.

The image processing unit 130 is a functional block that performs processing such as determination of imaging conditions, setting of the imaging conditions, and display of captured images. The image processing unit 130 includes a display control unit 131 that controls display of captured images, a determination unit 132 that determines imaging conditions, and a positioning unit 133 that positions imaging regions based on the imaging conditions. The display control unit 131 includes an image acquisition pattern presentation unit 1311 that displays the image acquisition pattern on a monitor and presents it to the operator. The determination unit 132 includes a magnification determination unit 1321 that determines the imaging magnification, a position determination unit 1322 that determines the imaging position, a time determination unit 1323 that determines the imaging timing, and an order determination unit 1324 that determines the imaging order. ..

(SLO image pickup device)
Next, a configuration example of the SLO image pickup device 20 including the adaptive optics will be described with reference to FIG. The configuration of the SLO image pickup device described below is an example, and any image pickup device can be used as long as a high-magnification image can be obtained.

Reference numeral 201 denotes a light source, and FIG. 4 shows an example realized by an SLD light source (Super Luminescent Diode). In the present embodiment, both the light source for fundus imaging and the light source for wavefront measurement are realized by the light source 201, but they may be separate light sources and combined in the middle of the optical path.

The light emitted from the light source 201 passes through the single mode optical fiber 202 and is emitted by the collimator 203 as parallel measurement light 205. The irradiated measurement light 205 passes through the light splitting unit 204 including a beam splitter and is guided to the adaptive optics optical system.

The adaptive optics system includes a light splitting unit 206, a wavefront sensor 215, a wavefront correction device 208, and reflection mirrors 207-1 to 207-4 for guiding them. Here, the reflection mirrors 207-1 to 207-4 are installed such that at least the pupil of the eye and the wavefront sensor 215 and the wavefront correction device 208 are in an optically conjugate relationship. In this embodiment, a beam splitter is used as the light splitting unit 206. In this embodiment, a spatial phase modulator using a liquid crystal element is used as the wavefront correction device 208. A variable shape mirror may be used as the wavefront correction device. The light passing through the adaptive optics is one-dimensionally or two-dimensionally scanned by the scanning optical system 209.

As the scanning optical system 209, in this embodiment, two galvano scanners are used for main scanning (horizontal fundus direction) and sub-scanning (fundus fundus vertical direction). However, for higher speed imaging, a resonant scanner may be used on the main scanning side of the scanning optical system 209.

The measurement light 205 scanned by the scanning optical system 209 is applied to the eyeball 211 through the eyepiece lenses 210-1 and 210-2. The measurement light 205 applied to the eyeball 211 is reflected or scattered by the fundus. By adjusting the positions of the eyepieces 210-1 and 210-2, it becomes possible to perform optimum irradiation in accordance with the diopter of the eyeball 211. Although the lens is used for the eyepiece here, it may be formed of a spherical mirror or the like.

The reflected scattered light (return light) reflected or scattered from the retina of the eyeball 211 travels in the opposite direction on the same path as the path of the incident light, and is partially reflected by the light splitting unit 206 to the wavefront sensor 215. It is used to measure the wavefront of. The wavefront sensor 215 is connected to the adaptive optics control unit 216 and transmits the received wavefront to the adaptive optics control unit 216. The wavefront correction device 208 is also connected to the adaptive optics control unit 216 and performs the modulation instructed by the adaptive optics control unit 216. Based on the wavefront measured by the wavefront sensor 215, the adaptive optics control unit 216 calculates a modulation amount (correction amount) for correcting the wavefront reaching the wavefront correction device 208 into a wavefront having no aberration, and the wavefront correction is performed. Instruct the device 208 to do so. The measurement of the wavefront and the instruction to the wavefront correction device 208 are repeatedly processed, and feedback control is performed so that the wavefront is always optimal.

A part of the reflected and scattered light transmitted through the light splitting unit 206 is reflected by the light splitting unit 204, and is guided to the light intensity sensor 214 through the collimator 212 and the optical fiber 213. The light intensity sensor 214 converts the light into an electric signal, and the controller 217 forms an image as an eye image and displays it on the display 218. By increasing the swing angle of the scanning optical system in the configuration of FIG. 4 and instructing the adaptive optics control unit 216 not to perform aberration correction, the SLO image pickup device 20 operates also as a normal SLO device, and the wide range is achieved. An SLO image with a wide field of view (wide field of view image D _L ) can be captured.

(Processing procedure)
Specific processing contents executed by the ophthalmologic apparatus 10 will be described in detail in association with the role of each functional block. FIG. 5 is a flowchart showing a processing procedure executed by the ophthalmologic apparatus 10. The following steps are executed under the control of the CPU 301.

<Step S510>
The wide-angle image acquisition unit 111 requests the SLO image capturing device 20 to acquire the wide-angle image D _L and the fixation target position F _L. In the present embodiment, an example in the case of obtaining a wide angle image D _L by setting the fixation target position F _L fovea of the macula portion. The method for setting the imaging position is not limited to this, and may be set at any other position.

SLO image capturing device 20, a wide angle of view wide angle image D _L in response to the acquisition request from the image acquisition unit 111 acquires the fixation target position F _L, and transmits to the wide field of view image obtaining unit 111. The wide view angle image acquisition unit 111 receives the wide view angle image D _L and the fixation target position F _L from the SLO image capturing device 20 via the LAN 30. The wide view angle image acquisition unit 111 stores the received wide view angle image D _L and the fixation target position F _L in the storage unit 120. Note that in the present embodiment example, the wide-angle image D _L is a moving image that has undergone inter-frame alignment.

<Step S520>
The image acquisition pattern presentation unit 1311 acquires at least one type of image acquisition pattern (basic setting pattern regarding parameters when acquiring a plurality of high-magnification images) from the storage unit 120 and displays it on the monitor 305 in a selectable manner. Although any pattern can be presented as the image acquisition pattern, in the present embodiment, a case will be described in which a basic pattern as shown in FIGS. 6A to 6F is presented. 6A is linear, FIG. 6B is cross-shaped, FIG. 6C is radial, FIG. 6D is rectangular, FIG. 6E is disk-shaped, and FIG. 6F. Shows an example of a circular pattern.

Next, the instruction acquisition unit 140 externally acquires an instruction related to the selection of the image acquisition pattern desired by the operator. This instruction is input by the operator via the keyboard 306 or the pointing device 307, for example. Alternatively, when the monitor 305 includes a touch panel liquid crystal, the input may be performed via the touch panel. In the present embodiment, since the observation target is the disc-shaped photoreceptor cell loss region as shown in FIG. 7D, the disc-shaped image acquisition pattern as shown in FIG. 6E is selected.

It should be noted that the image acquisition pattern is not limited to one composed of only one high-magnification image of one kind of magnification as shown in FIGS. 6A to 6F, but a combination of images of a plurality of magnifications. Can be presented as an image acquisition pattern. For example, as shown in FIG. 6G, not only the high-magnification image D _1j but also the intermediate-magnification image D _2k may be included in the acquisition pattern and defined. Hereinafter, such an acquisition pattern including not only the high-magnification image but also the intermediate-magnification image is referred to as a “multi-magnification image acquisition pattern”. Such an image acquisition pattern is suitable for reducing the number of acquired images or for more accurate alignment with the wide-angle image D _L. In the multi-magnification image acquisition pattern, the shape of the image acquisition pattern formed by the high-magnification image or the intermediate-magnification image of each magnification may be different or the same for each magnification. For example, the intermediate-magnification image D _2k having a lower magnification may be acquired in a rectangular shape and the high-magnification image D _1j may be acquired in a disk shape. When the image acquisition patterns are different between different magnifications, the type information of the image acquisition pattern at each magnification is also acquired in this step. Note that FIG. 6G shows a case where the shape of the image acquisition pattern formed by the high-magnification image D _1j and the shape of the image acquisition pattern formed by the intermediate-magnification image D _2k are both cross-shaped and the same. ..

Further, as shown in FIG. 6(h), a pattern in which a plurality of patterns are arranged by changing the position of the basic pattern (hereinafter, referred to as "multi-arrangement type image acquisition pattern") may be presented. FIG. 6H shows a plurality of rectangular image acquisition patterns arranged. It is suitable when there are multiple lesions and when comparing the morphology and dynamics of the observation target for each site. Note that the multi-arrangement image acquisition pattern includes a case where images are acquired by changing the focus positions of the basic patterns. Further, as shown in FIG. 6I, an image acquisition pattern defined by a combination of basic patterns (hereinafter, referred to as “composite-type image acquisition pattern”) may be presented. The composite image acquisition pattern is suitable for efficiently acquiring images for different purposes in one inspection. For example, FIG. 6(i) shows a case of measuring the shape of the avascular region (white line closed region in FIG. 6(i)) of the fovea centralis (the center of gravity of the avascular region) shown by the black dots in the same figure (rectangular type). ) And the case where the photoreceptor cell density is measured at a constant distance from the fovea (cross shape).

<Step S530>
In the determining unit 132, the acquisition parameters of the plurality of images included in the image acquisition pattern selected in S520 are set as initial values, and the operator determines the acquisition parameters of the plurality of high-magnification images by adjusting the image acquisition parameters as necessary. And acquire an image. The process of this step (hereinafter referred to as “high-magnification image acquisition process”) will be described later in detail with reference to the flowchart shown in FIG.

<Step S540>
The alignment unit 133 aligns the wide-angle image D _L and the high-magnification image D _H to obtain the relative position of the high-magnification image D _H on the wide-angle image D _L. The alignment means automatically determining the positional relationship between the wide-angle image D _L and the high-magnification image D _H and setting the position of D _{H at} the position corresponding to D _L. The wide-angle image D _L is an image with a lower magnification than the high-magnification image showing the entire imaging region, and is a given image given in advance. When there is an overlapping area between the high-magnification images D _Hj , first, the inter-image similarity is calculated for the overlapping area, and the positions of the high-magnification images D _Hj are adjusted to the positions where the inter-image similarity is maximized. Next, when high-magnification images with different resolutions have been acquired in S530, alignment is performed in order from the lower-magnification image. For example, when the high-magnification image D _1j and the intermediate-magnification image D _2k are acquired as the high-magnification image D _H , first, the wide-angle image D _L and the intermediate-magnification image D _2k are aligned, and then the alignment is performed. Positioning is performed between the intermediate-magnification image D _2k and the high-magnification image D _1j . It goes without saying that when the resolution of the high-magnification image is one, only the alignment of the wide-angle image D _L and the high-magnification image D _H is performed.

Incidentally, the positioning unit 133 acquires the fixation target position F _H used during imaging from the storage unit 120 the high magnification image D _H, position in alignment with the wide field of view image D _L and the high magnification image D _H It is set as the initial point for searching the matching parameter. Any known method can be used as the inter-image similarity or the coordinate conversion method. In the present embodiment, the alignment is performed using the correlation coefficient as the inter-image similarity and the Affine conversion as the coordinate conversion method.

<Step S550>
The display control unit 131 displays the high-magnification image D _H on the wide-angle image D _L based on the value of the alignment parameter obtained in S540. The process of this step (hereinafter, referred to as “image display process”) will be described later in detail with reference to the flowchart shown in FIG.

<Step S560>
Instruction acquisition unit 140, wide field of view image D _L, the high magnification image D _H, fixation target position F _L, instruction whether to save the position alignment parameter values obtained in F _H, S540 to the data server 40 Is obtained from the outside. This instruction is input by the operator via the keyboard 306 or the pointing device 307, for example. If the save is instructed (YES in S560), the process proceeds to S570, and if the save is not instructed, the process proceeds to S580 (NO in S560).

<Step S570>
The image processing unit 130 associates the information identifying examination date, the eye to be examined, wide field of view image D _L, the high magnification image D _H and fixation target position F _L, F _H, the alignment parameter values, the data server 40 Send to.

<Step S580>
The instruction acquisition unit 140 externally acquires an instruction as to whether or not to terminate the processing of the wide-angle image D _L and the high-magnification image D _H by the ophthalmologic apparatus 10. This instruction is input by the operator via the keyboard 306 or the pointing device 307. When the instruction to end the process is acquired (YES in S580), the process ends. On the other hand, when the instruction to continue the process is acquired (NO in S580), the process is returned to S510, and the process for the next optometry eye or the reprocessing for the same optometry eye is performed.

(High-magnification image acquisition process)
Next, the details of the high-magnification image acquisition process executed in S530 will be described with reference to the flowchart shown in FIG.

<Step S810>
The determination unit 132 acquires from the storage unit 120 the type of the image acquisition pattern selected through the instruction acquisition unit 140 and the acquisition parameter value of each high-magnification image D _H that this pattern has. Specifically, among the acquisition parameters of each high-magnification image held by the selected image acquisition pattern, the following parameter values are input as initial values. That is, the number of magnifications and the values of each angle of view and pixel size are acquired by the magnification determination unit 1321, the acquisition position and the focus position are acquired by the position determination unit 1322, the number of frames and the frame rate, and the number of repeated acquisitions are acquired by the time determination unit 1323. The order is input to the order determining unit 1324 as an initial value.

<Step S820>
The determination unit 132 acquires, through the instruction acquisition unit 140, the constraint condition regarding the setting value of the acquisition parameter of each high-magnification image D _Hj forming the selected image acquisition pattern. The constraint condition is a condition that defines a range that the imaging condition can take. The constraint condition can be designated and set by the operator regarding arbitrary image acquisition parameters. In this embodiment, the case where the following four constraint conditions can be set will be described.
a) Sum of image acquisition times b) Type of magnification (number of magnifications, angle of view and pixel size)
c) Focusing position d) Overlap area between adjacent high-magnification images
a) is the permissible time per eye that the eye can withstand,
b) is the size of the image feature expected to be obtained at the imaging position,
c) is the position in the depth (z axis) direction where the observation target exists,
d) is a constraint condition regarding an allowable fixation disparity amount of the eye. In this embodiment, a) 15 minutes b) 1 and 300 [μm]×300 [μm] and 1 [μm/pixel]×1 [μm/pixel], respectively.
c) photoreceptor layer d) 20% of high magnification image area
An example of such a case will be described.

<Step S830>
The magnification determination unit 1321 determines the type of magnification (magnification number, angle of view, and pixel size) of the high-magnification image D _H. Further, the position determining unit 1322 determines the acquisition position and the focus position of each high-magnification image D _Hj .

In the present embodiment, the angle of view, the pixel size, and the focus position are fixed values due to the constraint conditions acquired in S820, but the acquisition position of each high-magnification image D _Hj is a variable parameter. Therefore, the operator first specifies the position on the fundus of a point (representative point) forming the image acquisition pattern. In the present embodiment, the representative point is the center point C in FIG. 6E, which is set in the fovea centralis of the eye to be inspected. Next, the operator enlarges or reduces the size of the entire image acquisition pattern, thereby increasing or decreasing the number of acquisition positions of the high-magnification image while maintaining the overlapping area size between the adjacent high-magnification images. The acquisition position of the image D _Hj is determined. In the present embodiment, the size of the entire image acquisition pattern is enlarged by the operator moving the position of (a certain one) high-magnification image located at the end of the image acquisition pattern to the outside of the disk, as shown in FIG. The acquisition position of the high-magnification image D _Hj as shown by the white line rectangular area in g) is determined.

The representative point is not limited to the center point of the image acquisition pattern. For example, it may be the position of a specific high-magnification image forming the image acquisition pattern. Further, in the case of a multi-magnification type image acquisition pattern as shown in FIG. 6G, the sizes of the image acquisition patterns of all the magnifications may be collectively enlarged or reduced, or the image acquisition pattern may be obtained for each magnification. The size of the pattern may be changed.

In the case of the multi-arrangement image acquisition pattern shown in FIG. 6H, the image acquisition pattern sizes or the arrangement intervals of all the arrangements may be changed together. Further, the size or arrangement interval of the image acquisition pattern may be different for each arrangement of the basic patterns. Further, even in the case of the composite type image acquisition pattern as shown in FIG. 6I, the sizes of the patterns of all types may be changed collectively, or the size of the image acquisition pattern may be changed for each type of image acquisition pattern. May be changed.

<Step S840>
The time determination unit 1323 determines the number of frames of each high-magnification image, the frame rate, and the number of repeated acquisitions. In the example of the present embodiment, the frame rate and the number of repeated acquisitions are each 32 [frame/sec] and a fixed value of once, and the number of frames is a variable parameter. The method of changing the variable parameter value can be specified using any known user interface (hereinafter abbreviated as “UI”). In the present embodiment, the operator efficiently changes the parameter value (weight) changing UI as shown in FIG. 6(j). This is a UI for adjusting the weight in the radial direction regarding the number of frames of each high-magnification image D _Hj . In FIG. 6(j), the center side of the disk-shaped image acquisition pattern in which Wc is arranged (FIG. 7(g)), and Wo represents the outer weight. By lowering the outer weight Wo, the number of frames of each high-magnification image D _Hj is automatically determined so as to gradually decrease from the pattern center toward the outer side.

Further, in the case of the multi-magnification pattern as shown in FIG. 6G, a parameter value (or weight) adjustment UI as shown in FIG. 6K can be used. The adjustment of the parameter value is performed by the following steps i) to iv).
i) A variable parameter to be adjusted is selected from the variable parameter list V.
ii) A target magnification and a target image for adjusting the parameter value are selected from the adjustment map.
iii) A parameter value changing (weighting) method R between a plurality of images at the selected magnification is selected.
iv) The parameter value (weight) for the selected image is determined on the parameter value changing UI (B).
Note that in i), the adjustment map of FIG. 6(k) is displayed for each selected variable parameter type. Regarding ii), FIG. _6K shows a case where the target magnification is D ₁ and the D ₁ image D _1c at the center is selected as the adjustment target image.
iii), in FIG. 6(k), as a parameter value setting (weighting) method R between images,
・When setting the same value for multiple images with the same magnification (uniform)
・When changing in stages (gradual)
-When changing the parameter value for a specified individual image (individual)
The case where it selects from among is illustrated. In the example of FIG. 6(k), the case (gradual) of changing in steps is selected.
6(k), by specifying the maximum value (white) on the color bar B, the parameter value of the image having the same magnification as the selected target image D _1c is changed to the selected image D _1c. It shows a case where it is automatically changed so as to gradually increase toward.

In FIG. 6(k), the parameter value changing UI is the color bar B and the parameter values are expressed in gray scale, but the present invention is not limited to this. For example, the parameter value changing UI may be a slider or a (numerical) list box. The parameter value may be displayed as a numerical value (parameter value itself or weight) or may be displayed in color. Alternatively, the parameter value may be displayed using both a numerical value and a gray scale (color).

Further, in the case of the multi-arrangement pattern as shown in FIG. 6(h), the weight within each pattern (for example, Wm1dhi in FIG. 6(l)) and the weight between patterns as shown in FIG. 6(l). Adjust both (Wm1 and Wm2). Further, in the case of the composite type image acquisition pattern as shown in FIG. 6(m), the parameter value (Wai or Wci) in each pattern and the parameter value (Wbi) in the image common to both patterns are set. To do. The adjustment procedure is almost the same as in the case of the multi-magnification image acquisition pattern of FIG. That is, the above procedure iii) is only replaced with the procedure of selecting the parameter value changing (weighting) method R between a plurality of images in the selected pattern or common area, instead of the selected magnification.

<Step S850>
The order determination unit 1324 determines the acquisition order of the high-magnification images D _Hj . In the present embodiment, among the following i) to iv), i) is the innermost (highest priority), ii) is the second innermost, iii) is the third innermost, and iv) is the outermost (lowest priority). Iterative processing is performed so that it becomes a loop of (degree). Specifically, the following procedure is executed after setting the acquisition start position to the position of highest importance in observation (fovea in the present embodiment) and the acquisition magnification to the minimum magnification.
i) Images at the same arrangement pattern, at the same acquisition magnification, and at the same image acquisition position are repeatedly acquired the number of times.
ii) Images having the same arrangement pattern and the same acquisition magnification are moved to the adjacent image acquisition positions and then acquired again in the same manner as i).
When iii) ii) is completed, the value of the acquisition magnification is increased and the operation of ii) is executed again, and the same operation is repeated for the number of magnifications.
iv) When iii) is completed, the operation of iii) is executed in another arrangement, and the process is repeated until images are acquired in all the arrangements.

In this embodiment, i) is not repeatedly acquired (the number of acquisitions is only once), and the image acquisition pattern is not of the multi-arrangement type shown in FIG. The process is omitted. In addition, the moving direction of ii) to the adjacent image may be moved in any direction, but in the present embodiment, the closer to the fovea, the stronger the influence on the visual function and the higher the importance in observation. Move from the fossa in a vortex.

By the processing of S830 to S850, the operator can easily change the image acquisition parameter indicating the imaging condition of the high-magnification image.

<Step S860>
The high-magnification image acquisition unit 112 requests the SLO image capturing device 20 to acquire a plurality of high-magnification images D _Hj and the fixation target position F _Hj using the image acquisition parameters instructed by the determination unit 132. .. Since the SLO image capturing device 20 acquires and transmits the high-magnification image D _Hj and the fixation target position F _Hj in response to the acquisition request, the high-magnification image acquiring unit 112 causes the SLO image capturing device 20 to transmit the high-magnification image D _Hj and the fixation target position F _Hj via the LAN 30. The high-magnification image D _Hj and the fixation target position F _Hj are received. The high-magnification image acquisition unit 112 stores the received high-magnification image D _Hj and fixation target position F _Hj in the storage unit 120. It should be noted that in the present embodiment, the high-magnification image D _Hj is a moving image that has _undergone inter-frame alignment.

(Image display processing)
Next, the details of the image display processing executed in S550 will be described with reference to the flowchart shown in FIG.

<Step S910>
A representative image is generated from each moving image acquired by the wide-angle-of-view image acquisition unit 111 and the high-magnification image acquisition unit 112. In the present embodiment, a superimposed image for each moving image is generated, and this superimposed image is used as a representative image. The method of generating the representative image is not limited to this, and for example, the reference frame set at the time of frame-to-frame alignment of moving images may be set as the representative image. Any known setting method can be used as the reference frame setting method. For example, the frame with the first number can be set as the reference frame.

<Step S920>
The display control unit 131 corrects the density difference between the high-magnification images when a plurality of high-magnification images D _Hj are acquired. Any known luminance correction method can be applied to the correction of the density difference. As an example, in the present embodiment, a histogram is generated Hj in each high magnification image D _Hj, the luminance of each high magnification image D _Hj as mean and variance of the histogram Hj is common values between the high magnification image D _Hj The density difference is corrected by linearly converting the value.

<Step S930>
The display control unit 131, when displaying on the wide angle image D _L high magnification image D _H as a moving image, and sets the reproduction speed of the high magnification image D _H. The reproduction speed is adjusted by arranging a reproduction speed adjustment slider and a frame advance button in the image display area, and the operator specifying the reproduction speed through the instruction acquisition unit 140.

It should be noted that in the present embodiment example, since the still images (superimposed images) generated in S910 are displayed in a pasted manner, this processing is omitted.

<Step S940>
The display control unit 131 controls display/non-display and display magnification of each high-magnification image D _Hj . The display/non-display of images is performed by displaying a list of acquired images on the monitor 305, arranging a UI (check box in this embodiment) near the image name of the acquired image list, and displaying the UI through the instruction acquisition unit 140. It is set by the operator specifying ON/OFF. A UI (check box) for collectively designating all images and a batch designation UI (check box) for each acquisition magnification are prepared to facilitate switching between display/non-display of a large number of images.

In this step, not only the display/non-display of images but also the setting of the overlapping order when there is an overlapping region between adjacent high-magnification images D _Hj or when a plurality of images are taken at the same fixation target position are set. .. As for the method of setting the overlapping order of moving images, any setting method can be used, including manual setting. In the present embodiment, the image quality index and the fixation disparity amount of the image are calculated, and the image having the highest evaluation value is set and displayed in the frontmost layer using the linear sum of the image quality index value and the fixation disparity amount as the evaluation function. .. Here, any known index can be used as the image quality index value, and in this embodiment, the average brightness value of the image histogram is used. As the fixation disparity amount, a value obtained by adding the absolute value of the translational movement distance between adjacent frames over all frames is used. It should be noted that any index may be used as long as it can evaluate fixation disparity. Regarding the display magnification, the high-magnification image designated by the operator is enlarged through the instruction acquisition unit 140 and displayed on the monitor 305.

Although the wide-angle image D _L is a single wide-angle SLO image in the above example, the present invention is not limited to this. For example, a composite image obtained by aligning the wide angle image D _L i between the different acquisition position may be wide field of view image D _L.

As described above, in the ophthalmologic apparatus 10 according to the present embodiment, the operator selects from a plurality of basic patterns relating to the high-magnification image acquisition parameter (imaging condition), adjusts the parameter value according to the lesion shape, and A high-magnification image is acquired based on the adjusted parameter values. That is, the ophthalmologic apparatus 10 presents a plurality of basic patterns indicating distributions of a plurality of positions for capturing a high-magnification image to the operator in a selectable manner. In accordance with the operator's selection, the imaging condition relating to the imaging of the high-magnification image, which is associated with the basic pattern selected from the plurality of basic patterns in advance, is adjusted according to the operator's instruction. Further, according to the adjusted image pickup condition, the image pickup apparatus is caused to pick up a plurality of high-magnification images in the image pickup area. Therefore, according to the present embodiment, it is possible to easily set an appropriate imaging condition for acquiring a plurality of high-magnification images having an angle of view smaller than the angle of view of a given imaging region. it can. Therefore, it is possible to efficiently image a tissue, a cell group, or a lesion candidate having a wider distribution than the high-magnification image and having a different distribution depending on the eye to be inspected.

Further, in the present embodiment, the imaging condition is set based on at least one of the position of the representative point of the selected basic pattern in the imaging region, the constraint condition instructed by the operator, and the change amount of the imaging condition. adjust. Therefore, it is possible to easily set an appropriate imaging condition according to the imaging target. For example, there are individual differences in the shape and concentration of cell groups, tissues, lesions, etc., and the region to be observed or measured in detail differs depending on the eye to be examined. According to the configuration of the present embodiment, the cell group, tissue, and lesion area of the observation target are specified for each test eye, and the acquisition parameters of the plurality of high-magnification images D _Hj are set according to the shape and density of this area. It can be set automatically.

In the present embodiment, as the imaging conditions, a position for capturing a high-magnification image in a wide-angle image, an imaging order, the number of images captured at the same position, an angle of view and a pixel size of the high-magnification image, and a frame for imaging. Although the number, the frame rate, and the in-focus position are illustrated, the invention is not limited to this.

<<Embodiment 2>>
The ophthalmologic apparatus according to the present embodiment relates to acquisition of a plurality of high-magnification images based on image features extracted from an image having a wider angle of view than a high-magnification image when acquiring a plurality of high-magnification adaptive optical SLO images. It is configured to determine the parameter value. Specifically, when a plurality of high-magnification moving images D _1j are acquired in an annular shape for a _parafoveal capillary region, the acquisition position and the angle of view of each high-magnification image D _1j are obtained based on this image feature. , Pixel size, number of frames, frame rate, and number of repeated acquisitions are determined. Furthermore, if it is determined whether the acquired high-magnification image includes an exceptional frame such as fixation disparity or blink, and if it is determined that the high-magnification image needs to be re-acquired based on the determination result, the same image Re-acquire a high-magnification image with the acquisition parameter value.

(overall structure)
The configuration of equipment connected to the ophthalmologic apparatus 10 according to the present embodiment is shown in FIG. The present embodiment differs from the configuration of the first embodiment in that, in addition to the SLO image capturing device 20 and the data server 40, the temporal data acquisition device 50 is connected to the ophthalmologic apparatus 10. The time-phase data acquisition device 50 is a device that acquires biological signal data (referred to as “time-phase data”) that changes autonomously and periodically, and includes, for example, a pulse wave meter or an electrocardiograph. The time phase data acquisition device 50 acquires the time phase data Sj at the same time as acquiring the high-magnification image D _Hj in response to an operation by an operator (not shown). The obtained time phase data Sj is transmitted to the ophthalmologic apparatus 10 and the data server 40. In the present embodiment, the high-magnification image is acquired and displayed on the monitor 305 in synchronization with the periodic timing indicated by the time phase data measured by the subject. Therefore, it is possible to acquire or reproduce the high-magnification image at an appropriate timing according to the change of the living body.

The data server 40, wide field of view image D _L of the eye, the high magnification image D _H, and fixation target position F _L used at the time of its acquisition, in addition to obtaining condition data such as F _H, the pulse data Sj, a normal value regarding the image feature of the eye part and the distribution of the image feature of the eye part are also held. In the present embodiment, the retinal blood vessel, the capillary blood vessel Q, and the blood cell W are held as image features of the eye portion, but the present invention is not limited to this. The time phase data Sj output by the time phase data acquisition device 50 and the image feature of the eye part output by the ophthalmologic apparatus 10 are stored in the data server 40. Further, in response to a request from the ophthalmologic apparatus 10, normal value data regarding the time phase data Sj, the eye image feature and the distribution of the eye image feature are transmitted to the ophthalmologic apparatus 10.

(Ophthalmology device)
Next, FIG. 10 shows functional blocks of the ophthalmologic apparatus 10 according to the present embodiment. In addition to the configuration of the first embodiment, the ophthalmologic apparatus 10 of the present exemplary embodiment includes the data acquisition unit 110, the temporal data acquisition unit 113, the image processing unit 130, the image feature acquisition unit 134, and the determination unit 132, which determines whether reacquisition is necessary. The unit 1325 and the alignment unit 133 include an exceptional frame determination unit 1331. The time phase data acquisition unit 113 is a functional block that acquires time phase data of the subject. The image feature acquisition unit 134 is a functional block that analyzes a wide-angle image and acquires information indicating the feature. The re-acquisition necessity determination unit 1325 is a functional block that determines whether to acquire a high-magnification image again. The exceptional frame determination unit 1331 is a functional block that detects, as an “exceptional frame”, a frame that is not suitable for optometry, such as a frame having a large positional displacement due to poor fixation. When the "exception frame" is detected, the high-magnification image is re-acquired.

(Processing procedure)
The image processing flow in this embodiment is the same as that in FIG. 5, and the processing in S510, S560, S570, and S580 is the same as in the first embodiment. Moreover, the step of S540 is omitted. Therefore, in this embodiment, the processing of S520, S530, and S550 will be described.

<Step S520>
The image acquisition pattern presentation unit 1311 acquires at least one type of image acquisition pattern when acquiring a plurality of high-magnification images from the storage unit 120, and displays it on the monitor 305. In the present embodiment, the image acquisition pattern presentation unit 1311 presents linear, cross, radial, rectangular, disc, annular, multi-magnification, multi-position, and composite basic patterns.

Next, the instruction acquisition unit 140 externally acquires an instruction as to which image acquisition pattern to select. In the present embodiment, the case where the observation target is an annular parafoveal capillary region as shown in FIG. 7C will be described as an example. In this case, since it is necessary to determine the inner boundary of the annular region based on the avascular region, the operator selects the multi-magnification image acquisition pattern. Here, in the multi-magnification type image acquisition pattern, D _1j is annular and D _2k is rectangular.

In the present embodiment, the image acquisition pattern selection process is not an essential process. Even if the high-magnification image acquisition target region is a circular region and the intermediate-magnification image acquisition target region is a rectangular region from the beginning, the process of this step may be omitted. Good.

<Step S530>
Determination unit 132 performs an acquisition request for the intermediate magnification image D _2k the high magnification image acquisition unit 112, the high magnification image acquisition unit 112 acquires the intermediate magnification image D _2k. Next, the image feature acquisition unit 134 acquires image features on the wide-angle image D _L and the intermediate-magnification image D _2k , determines acquisition parameters of a plurality of high-magnification images based on the image features, and determines the high-magnification images. Get D _1j . Further, inter-frame alignment and exceptional frame determination are performed on the acquired high-magnification image D _1j , and when it is determined that re-imaging is necessary based on the determination result of the exceptional frame, the same high-magnification image D _1j is captured again. .. Further, the intermediate-magnification image D _2k and the high-magnification image D _1j are aligned on the wide-angle image D _L. The processing of this step (high-magnification image acquisition processing) will be described later in detail using the flowchart shown in FIG.

<Step S550>
The display control unit 131 superimposes and displays the high-magnification image D _H on the wide-angle image D _{L as} shown in FIG. 7E based on the alignment parameter value obtained in S1270 (described later). Further, in the present embodiment, next to the superimposed image, a capillary blood vessel image as shown in FIG. 7F is also displayed as an image in which the distribution of the capillaries in the parafovea can be observed in more detail. As described above, in the present embodiment, display control is performed to superimpose a plurality of captured high-magnification images on an image showing the entire imaging region and display the monitor 305, so that only a necessary portion of the wide-angle image is displayed. It is possible to observe a precise image. The process of this step (image display process) will be described later in detail with reference to the flowchart shown in FIG.

(High-magnification image acquisition process)
Next, the details of the process executed in S530 will be described with reference to the flowchart shown in FIG.

<Step S1210>
The determination unit 132 acquires the intermediate magnification image D _2k based on the D _2k image acquisition pattern selected in S520.

In the present embodiment, a rectangular image acquisition pattern such as D _{2k in} FIG. 12A is set, and the fixation target is presented so that the center point C of the image acquisition pattern is near the fovea. .. Further, the values of the angle of view, the pixel size, the number of frames, and the frame rate are 600 [μm]×600 [μm], and the pixel size is 2 [μm/pixel]×2 [μm/pixel], 256 sheets, 64 [frame]. /sec] is set. The overlapping area between adjacent intermediate-rate images is 10% of the intermediate-magnification image area. In the present exemplary embodiment, the acquisition order of the high-magnification images is the order in which the intermediate-magnification image D ₂₅ at the center of the image acquisition pattern is moved to the right next to the first acquisition position, and then the adjacent images are moved counterclockwise. To do. The alignment unit 133 performs inter-frame alignment of the acquired intermediate-magnification image D _2k and performs alignment (image pasting) of the intermediate-magnification image D _{2k on} the wide-angle image D _L. Since the coordinate conversion method and the similarity evaluation function used for alignment are the same as those in the first embodiment, detailed description will be omitted.

<Step S1220>
The image feature acquisition unit 134 detects capillaries from the wide-angle image D _L or the intermediate-magnification image D _2k acquired in S1210, and detects the boundary of the avascular region from the detected capillary region. Further, in order to set the region near the avascular region as the acquisition target region of the high-magnification image, an annular (doughnut-shaped) region equidistant from the boundary position of the avascular region is detected.

In the present embodiment, first, the capillaries are specified as the moving range of the blood cell component from the intermediate magnification image D _{2k by the} following procedure.
(A) The difference processing is performed between the adjacent frames of the intermediate magnification image D _{2k that has undergone} the inter-frame alignment. That is, a differential moving image is generated.
(B) A luminance statistic (for example, variance) in the frame direction is calculated at each xy position of the differential moving image generated in (a).
(C) At each xy position of the differential moving image, a region where the luminance dispersion is equal to or larger than a predetermined threshold Tv is specified as a region where blood cells have moved, that is, a capillary region.
The capillary blood vessel detection process is not limited to this method, and any known method may be used. For example, a blood vessel may be detected by applying a filter that emphasizes the linear structure to a specific frame of the wide-angle image D _L or the intermediate magnification image D _2k .

Next, the image feature acquisition unit 134 detects the boundary of the avascular region from the obtained capillary region. In the vicinity of the fovea of the retina, there is a region where no retinal blood vessels exist (referred to as "avascular region"), as in the inside of the inner broken line region in FIG. 7C. The shape of the border of the avascular region varies greatly among individuals, and initial lesions of retinal blood vessels are likely to occur around the border of the avascular region. Therefore, the avascular region boundary is important as an object of observation and analysis.

In this embodiment, a variable shape model (solid line portion in FIG. 12A) having a radius Tr smaller than a circle (dotted line portion in FIG. 12A) connecting the image centers of the high-magnification images D _1j in the image acquisition pattern is wide-imaged in S1210. The intermediate magnification image D _2k aligned on the corner image is arranged on the combined image. In this embodiment, the center of this model is arranged so as to coincide with the center C of the intermediate magnification image D ₂₅ of FIG. The position (B i in FIG. 12B) of the deformable model that has been deformed in accordance with the image features of the intermediate magnification image D _2k on the combined image is defined as the avascular region boundary, and the center of gravity C of the avascular region boundary is set. 'Decide. Further, using a distance image (image having a pixel distance value from the boundary) obtained by performing Euclidean distance conversion on the avascular area boundary, predetermined threshold values To and To/ Positions at a distance of 2 (Bo and Bm in FIG. 12B) are determined. Although an arbitrary value can be set as the threshold value To, since it is often set to about 150 [μm] in a healthy person, this value is also used in this embodiment. An annular (doughnut-shaped) high-magnification imaging target area is determined using the identified inner boundary Bi and outer boundary Bo, and the broken line portion Bm is a candidate for the acquisition position (image center) of the high-magnification image D _1j. Become.

Although the distance from the avascular region boundary (the thickness of the annular region) is fixed by the threshold value To in the present embodiment, the present invention is not limited to this. For example, in diseases such as diabetic retinopathy in which lesions occur in retinal capillaries in the parafovea, the capillaries become obstructed as the disease progresses, and the avascular region becomes large. Further, when the avascular region becomes large, there is a possibility that a vascular lesion occurs in a wider area around the avascular region. Therefore, a value obtained by multiplying the threshold value To by a value proportional to the area of the avascular region may be set as the distance from the avascular region boundary. In this case, in S1230, the angle of view is used as a variable parameter, and the distance from the avascular region boundary, that is, the thickness is determined to be sufficiently larger than the thickness of the annular region.

<Step S1230>
The magnification determining unit 1321 determines the number of magnifications, the angle of view, and the pixel size of the high-magnification image D _1j . Further, the position determining unit 1322 determines the acquisition position and the focus position of each high-magnification image D _1j . In the present embodiment, the number of magnifications, the angle of view, the pixel size, and the focus position are fixed parameters (2, 200 [μm]×200 [μm], 1 [μm/pixel]×1 [μm/pixel], retinal blood vessels, respectively. ), the high magnification image D _1j acquisition position is determined as a variable parameter as follows.

First, points (Bms in FIG. 12B) obtained by sampling the boundary Bm determined in S1220 at equal intervals Td are candidates for the acquisition position of the high-magnification image D _1j , and the high-magnification from the specific candidate point Bm0. The acquisition position of the image D _1j is sequentially determined.
In the present embodiment, the interval Td=the angle of view of the high-magnification image D _1j ×(100−the standard value of the ratio of the overlapping area between the high-magnification images D _1j )/100 (1)
And the candidate point immediately above the center of gravity of the avascular region is Bm0. The detailed acquisition position of the high-magnification image D _1j is determined so that both the following conditions a) and b) are satisfied. That is,
a) With respect to the radial direction of the annular region determined in S1220 (the line direction connecting the barycentric position C′ of the avascular region and the acquisition position candidate point Bms), the high magnification protruding outside the annular region under the condition that the annular region is not blank. The sum of the pixels of the image D _1j is the minimum.
b) In the tangential direction of the boundary position Bm, match the ratio of the overlapping area between the high-magnification images described below.
Here, the ratio [%] of the overlapping region between the high-magnification images D _1j is the circularity Cr of the avascular region boundary specified in S1220.
Cr=4πS/(L*L) (2)
(S is the area of the avascular region, L is the perimeter)
Using, the standard setting value (20% as an example in the present embodiment) is multiplied by a value that is inversely proportional to the circularity Cr, and is set as the ratio of the overlapping area between the high-magnification images D _1j. .. Therefore, the lower the circularity, that is, the larger the unevenness, the larger the ratio of the overlapping area between the high-magnification images D _1j is set.

The method of determining the overlapping area between the high-magnification images is not limited to this, and any known method may be used. For example, from the intersection point Bis of the line connecting the center of gravity C′ of the avascular region boundary and the acquisition position candidate point Bms to the avascular region, along the avascular region boundary, in the direction of the high-magnification image adjacent to the curvature of a certain distance The absolute value is calculated, and the average value Ch of the obtained absolute values of the curvature is calculated. The standard setting value regarding the ratio of the overlapping area between the high-magnification images is multiplied by a value proportional to the average curvature value Ch and weighted. However, when the average curvature value Ch is 0, the standard setting value is used as it is without weighting. By using such a setting method, the ratio of the overlapping region between the high-magnification images can be set large in the vicinity of the position where the absolute value of the curvature of the avascular region boundary is large.

<Step S1240>
The time determination unit 1323 determines the number of frames and frame rate of the high-magnification image D _1j , and the number of times of repeated acquisition. In the example of the present embodiment, the number of frames, the frame rate, and the number of repeated acquisitions at the same acquisition position are 256, 64 [frame/sec], and two, respectively. However, the present invention is not limited to this, and any setting method may be used. For example, the thinning process is performed on the capillary region in each high-magnification image D _1j determined in S1220 and S1230, and the blood vessel diameter is calculated in the direction orthogonal to the central axis of the obtained blood vessel. Then, only when there is a region where the blood vessel diameter has an abnormal value, the number of frames of the high-magnification image D _{1j and} the number of times of repeated acquisition may be increased by the thresholds Tf and Tc, respectively.

<Step S1250>
The order determination unit 1324 determines the acquisition order of the high-magnification images D _Hj . Similar to S850 in the first embodiment, of the following i) to iii), i) is the innermost (highest priority), ii) is the second innermost, and iii) is the outermost (lowest priority). Iterative processing is performed in a loop. That is, the following procedure is executed after setting the acquisition start position to the ear side and the acquisition magnification to the minimum magnification.
i) The images at the same acquisition magnification and the same image acquisition position are acquired the number of times of repeated acquisition.
ii) Images at the same acquisition magnification are acquired from adjacent image acquisition positions (counterclockwise in this embodiment) in the same manner as i).
When iii) ii) is completed, the value of the acquisition magnification is increased and the operation of ii) is executed again, and the same operation is repeated for the number of magnifications.
The method for determining the order of high-magnification images is not limited to the above procedure, and any order setting method may be used.

<Step S1260>
The high-magnification image acquisition unit 112 acquires the high-magnification image and the time phase data according to the high-magnification image acquisition parameter determined in S1210 to S1250. The time phase data acquisition unit 113 requests the time phase data acquisition device 50 to acquire the time phase data Sj regarding the biological signal. In the present embodiment, a pulse wave meter is used as the time phase data acquisition device, and the pulse wave data Sj is acquired from the ear lobe (ear lobe) of the subject. Here, the pulse wave data Sj is represented as a periodic point sequence having an acquisition time on one axis and a pulse wave signal value measured by a pulse wave meter on the other axis. Since the time phase data acquisition device 50 acquires and transmits the corresponding time phase data Sj in response to this acquisition request, the time phase data acquisition unit 113 transmits the pulse wave data Sj from the time phase data acquisition device 50 via the LAN 30. To receive. The time phase data acquisition unit 113 stores the received time phase data Sj in the storage unit 120.

The data acquisition unit 110 requests the SLO image capturing device 20 to acquire a wide-angle image D _L , a plurality of high-magnification images D _Hj captured at different fixation target positions Fj, and fixation target position Fj data. .. Here, the case where the data acquisition unit 110 starts to acquire the high-magnification image D _Hj in accordance with a certain phase of the time-phase data Sj acquired by the time-phase data acquisition device 50 and the pulse immediately after the acquisition request of the high-magnification image D _Hj. It can be considered that the acquisition of the wave data Sj and the high-magnification image D _Hj are started at the same time. After the present embodiment acquisition request of the high magnification image D _Hj, it starts acquiring the pulse data Sj and the high magnification image D _Hj immediately.

<Step S1270>
The alignment unit 133 performs inter-frame alignment on the acquired high-magnification image D _1j, aligns the high-magnification image D _1j with the wide-angle image D _L , and displays it on the monitor 305. It should be noted that in the present embodiment, when the inter-frame alignment of each high-magnification moving image D _1j is performed, exceptional frame determination is performed to determine whether or not the exceptional frame described below is applicable. Any known registration method may be used as an inter-frame registration method for each moving image and a registration (image combining) method between images having different magnifications. In this embodiment, both of them are affine transformation and correlation. Align using numbers.

Here, the exceptional frame is, as shown in FIG. 12C, in each frame of the high-magnification moving image D _H, a frame Es with a large positional deviation due to a fixation defect, a low-luminance frame Eb due to a blink, and a low frame due to an aberration correction defect. An image quality frame (not shown). The exceptional frame can be determined based on whether the degree of brightness abnormality, the magnitude of distortion, the magnitude of noise with respect to the signal, the displacement amount with respect to the reference frame, or the like is equal to or greater than a certain value. In particular,
a) When translation is equal to or more than a threshold among the alignment parameter values between frames b) When the average luminance value of each frame is less than the threshold c) When the S/N ratio of each frame is less than the threshold, it is determined as an exceptional frame .. As a result of the exceptional frame determination, if the maximum value of the occurrence intervals of the exceptional frames in each high-magnification moving image D _1j is less than or equal to the threshold value Te, or if the total number of exceptional frames is greater than or equal to the threshold value Ts, the reacquisition necessity determination unit 1325 It is determined that the high-magnification image D _1j needs to be acquired again. Reacquisition necessity determining unit 1325, the high magnification image D to _1j requests to reacquire the high magnification image D _1j when reacquisition is determined to require the high magnification image acquisition unit 112 of the high magnification image acquisition The unit 112 reacquires the high-magnification image D _1j according to this request.

It should be noted that inter-frame alignment, exceptional frame determination, re-acquisition necessity determination, and re-acquisition do not necessarily have to be performed after all high-magnification images have been acquired. As soon as it is determined that reacquisition is necessary, it may be reacquired immediately. Alternatively, when the inter-frame alignment of the intermediate-magnification image D _2k in S1210 is performed, the exceptional frame determination and the re-acquisition necessity determination are performed, and the intermediate-magnification image D _2k is re-acquired as soon as it is determined that re-acquisition is necessary. Good. Further, the exceptional frame determination is not limited to the inter-frame alignment processing of the SLO moving image, and, for example, the anterior segment camera is connected to the ophthalmologic apparatus 10 to perform image processing of the anterior segment camera, for example, low brightness. The determination may be performed using frame detection, pupil position detection, or the like.

(Image display processing)
Next, the details of the processing executed in S550 will be described with reference to the flowchart shown in FIG. Since the steps other than S910 and S930 are the same as those in the first embodiment, the processing in S910 and S930 will be described in the present embodiment.

<Step S910>
The display control unit 131 performs processing for generating a superimposed image of the high-magnification image D _H on the wide-angle image D _L as illustrated in FIG. 7E based on the alignment parameter value obtained in S1270. To do. In this embodiment, since the high-magnification image D _H is not displayed as a still image but as a moving image, as described in S930, the representative image is not generated. However, in the image after the inter-frame alignment, a region having a pixel value of 0 may be generated at the image end portion, which may be a hindrance in display. Only the pixels whose pixel values are greater than 0 are displayed through the frame.

Further, in the present embodiment, as an image for adhering the distribution of the capillaries in the parafovea in more detail next to the stitched moving image, a capillaries image as shown in FIG. 7F is pasted. The display is also done. As for the capillary blood vessel image, the process of identifying the capillary blood vessel region performed on the intermediate magnification image D _2k in S1220 is similarly performed not only on the intermediate magnification image D _2k but also on the high magnification image D _1j to obtain a binary image. It is generated and is displayed by pasting based on the alignment parameter obtained in S1270. Further, similarly to the case of combining moving images, only the pixels having pixel values larger than 0 are displayed on the capillary blood vessel image throughout all the frames except the exceptional frame.

<Step S930>
When a plurality of intermediate-magnification images D _2k and high-magnification images D _1j are displayed on the wide-angle image D _L , each intermediate-magnification image is based on time-phase data (periodic data based on biological signals such as pulse waves). The reproduction timings of D _2k and the high-magnification image D _1j are synchronized. Specifically, the display control unit 131 acquires the time phase data Sj, Sk corresponding to each moving image (that is, each high-magnification image D _1j , intermediate-magnification image D _2k ) from the time-phase data acquisition unit 113, and The extreme value of the time phase data is detected to calculate the pulsation cycle. Next, an exceptional frame number series in each high-magnification image D _1j and intermediate-magnification image D _2k is acquired, and a continuous frame series that does not include an exceptional frame is selected as a display target. Furthermore, when the pulsation cycle in the selected frame differs between the moving images (high-magnification image D _1j , intermediate-magnification image D _2k ), the adjustment process of the display frame interval between the moving images (referred to as “frame interpolation process”) is performed. To do. Furthermore, while adjusting the reproduction start time of each moving image so that the reproduction timing of the frame corresponding to the extreme value of the time phase data corresponding to each moving image matches, the frame for an integral number of pulsation cycles is reproduced. Display the pasted movie with.

Note that the display method of the present invention is not limited to this, and if the time phase data is not acquired, this step is omitted and the display is performed as a moving image without adjusting the reproduction time. Good.

As described above, when acquiring a plurality of high-magnification adaptive optics SLO images, the ophthalmologic apparatus 10 according to the present embodiment makes a plurality of images based on the image features extracted from the image having a wider angle of view than the high-magnification image. The parameter value regarding the high-magnification image acquisition of is determined. Therefore, it is possible to efficiently image a tissue, a cell group, or a lesion candidate that has a wider distribution than the high-magnification image and has a different distribution depending on the eye to be inspected.

Further, in the present embodiment, at least one blood vessel image from the captured high-magnification images is displayed on the monitor 305 based on the characteristics of the image relating to the region where blood vessels or blood cells have moved. Therefore, it is possible to appropriately extract only a portion that requires particularly careful observation from the wide-angle image and automatically perform accurate imaging/display.

<<Embodiment 3>>
The ophthalmologic apparatus according to the present embodiment acquires a plurality of high-magnification adaptive optical OCT tomographic images based on image features extracted from an OCT tomographic image having a wider angle of view than the high-magnification images. It is configured to determine a parameter value related to acquisition of a magnified image. Specifically, the operator selects a basic pattern for obtaining a plurality of high-magnification images in a disk shape with respect to the photoreceptor layer near the fovea where the outer retinal layer is deformed by the serous retinal detachment RD, and the image acquisition parameter is selected. The initial value of. Next, a case will be described in which the acquisition parameters (acquisition position, angle of view, pixel size, coherence gate) of a plurality of high-magnification images are changed based on the image characteristics of the layer shape acquired from the wide-angle-of-view OCT tomographic image, and imaging is performed. To do.

(overall structure)
A configuration of a device connected to the ophthalmologic apparatus 10 according to this embodiment is shown in FIG. In the present embodiment, the ophthalmologic apparatus 10 is different from the first embodiment in that it is connected to a tomographic image capturing apparatus 60 having an adaptive optical system instead of the SLO image capturing apparatus 20. The tomographic image capturing device 60 is a device that captures a tomographic image of the eye. The tomographic image capturing apparatus 60 is configured as, for example, a spectral domain type OCT (SD-OCT: Spectral Domain Optical Coherence Tomography). The eye tomographic image capturing device 60 three-dimensionally captures a tomographic image of the eye to be inspected according to an operation by an operator (not shown). The captured tomographic image is transmitted to the ophthalmologic apparatus 10.

(Ophthalmology device)
Next, FIG. 13 shows functional blocks of the ophthalmologic apparatus 10 according to the present embodiment. The difference from the configuration of the first embodiment is that the image processing unit 130 includes an image feature acquisition unit 134 that acquires a feature of a wide-angle image. Further, the data server 40 holds normal value data regarding the image characteristics of the eye and the distribution of the image characteristics of the eye. Here, a case where normal value data regarding the retinal layer boundary, its shape, and thickness is held as these data will be described.

(Tomographic image capturing device)
Next, the configuration of the tomographic image capturing apparatus 60 including the adaptive optics will be described with reference to FIG. In FIG. 14, 201 is a light source, and in this embodiment, an SLD light source with a wavelength of 840 nm is used. The light source 201 only needs to have low coherence, and an SLD light source having a wavelength width of 30 nm or more is preferably used. Further, an ultra-short pulse laser such as a titanium sapphire laser can be used as a light source. The light emitted from the light source 201 is guided to the fiber coupler 520 through the single mode optical fiber 202. The fiber coupler 520 splits the measurement light path 521 and the reference light path 522. Here, a fiber coupler having a branching ratio of 10:90 is used, and is configured so that 10% of the input light amount reaches the measurement light path 521. The light passing through the measurement light path 521 is emitted by the collimator 203 as parallel measurement light.

The configuration after the collimator 203 is the same as that of the SLO image pickup device 20 described in the first embodiment. That is, the eye 211 is irradiated through the adaptive optical system and the scanning optical system, and the reflected and scattered light from the eye 211 follows the same path again and is guided to the optical fiber 521 to reach the fiber coupler 520. On the other hand, the reference light that has passed through the reference light path 522 is emitted by the collimator 523, reflected by the optical path length varying unit 524, and returns to the fiber coupler 520 again. The measurement light and the reference light that have reached the fiber coupler 520 are combined and guided to the spectroscope 526 through the optical fiber 525. The control unit 217 configures a tomographic image of the eye based on the interference light information dispersed by the spectroscope 526. The control unit 217 controls the optical path length varying unit 524 and can acquire an image at a desired depth position.

The tomographic image capturing apparatus 60 operates as a normal tomographic image capturing apparatus by increasing the swing angle of the scanning optical system in the configuration of FIG. 14 and instructing the adaptive optical control unit 216 not to perform aberration correction. A wide-angle tomographic image (wide-angle image D _L ) can be captured. In addition, in the present embodiment, the tomographic image capturing apparatus 60 including the adaptive optics is configured as SD-OCT, but SD-OCT is not an essential requirement. For example, it may be configured as time domain OCT or SS-OCT (Swept Source Optical Coherence Tomography). In the case of SS-OCT, light sources that generate lights of different wavelengths at different times are used, and a spectroscopic element for acquiring spectrum information is unnecessary. Further, with SS-OCT, it is possible to acquire a deep image including not only the retina but also the choroid.

(Processing procedure)
An image processing flow of the ophthalmologic apparatus 10 according to this embodiment is shown in FIG. The processing contents other than S510, S520, S530, S540, and S550 are the same as the processing of the first embodiment described with reference to FIG. Therefore, in this embodiment, the processing of S510, S520, S530, S540, and S550 will be described.

<Step 510>
The wide-angle image acquisition unit 111 requests the tomographic image capturing apparatus 60 to acquire the wide-angle image D _L and the fixation target position F _L. In the present embodiment, an example in the case of obtaining a wide angle image D _L by setting the fixation target position F _L fovea of the macula portion. The method for setting the imaging position is not limited to this, and may be set at any other position.

The tomography apparatus 60, the wide field of view wide angle image D _L in response to the acquisition request from the image acquisition unit 111 acquires the fixation target position F _L, and transmits to the wide field of view image obtaining unit 111. The wide view angle image acquisition unit 111 receives the wide view angle image D _L and the fixation target position F _L from the tomographic image capturing apparatus 60 via the LAN 30. The wide view angle image acquisition unit 111 stores the received wide view angle image D _L and the fixation target position F _L in the storage unit 120.

<Step S520>
The image acquisition pattern presentation unit 1311 acquires at least one type of basic setting pattern (image acquisition pattern) relating to the parameters when acquiring a plurality of high-magnification images from the storage unit 120, and displays it on the monitor 305. Although any pattern can be set as the image acquisition pattern, the present embodiment presents basic patterns as shown in (a) to (f) of FIG. That is, FIG. 15(a) is linear, FIG. 15(b) is cross-shaped, FIG. 15(c) is radial, FIG. 15(d) is rectangular, FIG. 15(e) is disk-shaped, and FIG. f) is annular.

Next, the instruction acquisition unit 140 externally acquires an instruction as to which image acquisition pattern to select. In the present embodiment, a case in which the observation target is a region in which the outer layer of the retina is deformed by the serous retinal detachment RD as shown in FIG. 15I and the photoreceptor cells are damaged will be described. Therefore, as shown in FIG. Select a simple disk-shaped image acquisition pattern.

Similar to the case of the first embodiment, a multi-magnification type, multi-position type, or composite type image acquisition pattern may be presented in a three-dimensional tomographic image. For example, in the case of the multi-magnification type and the number of magnifications is 3, the acquisition pattern of the intermediate magnification image D _3m as shown in FIG. 15(h) and the acquisition pattern of the intermediate magnification image D _2k as shown in FIG. 15(g). , The acquisition pattern of the high-magnification image D _1j as shown in FIG. _15E can be selected. Furthermore, in the case of a multi-arrangement type image acquisition pattern, a plurality of image acquisition patterns may be arranged and presented in the depth direction (z-axis direction in the figure).

<Step S530>
The determination unit 132 adjusts the image acquisition parameters based on the image features acquired by the image feature acquisition unit 134, using the acquisition parameters of the plurality of images included in the image acquisition pattern selected in S520 as initial values. Determine image acquisition parameters. The processing of this step (high-magnification image acquisition processing) will be described later in detail using the flowchart shown in FIG.

<Step S540>
The alignment unit 133 aligns the wide-angle image D _L and the high-magnification image D _Hj, and determines the position of the high-magnification image D _Hj on the wide-angle image D _L. First, the alignment unit 133 acquires the fixation target position F _Hj used when capturing the high-magnification image D _Hj from the storage unit 120, and determines the position of the wide-angle image D _L and the high-magnification image D _Hj . The initial point for the registration parameter search. If there is an overlapping area between the high-magnification images D _Hj , first, the inter-image similarity is calculated for this overlapping area, and the positions of the high-magnification images D _Hj are matched to the position where the inter-image similarity is maximum. Next, when high-magnification images with different resolutions have been acquired in S530, alignment is performed in order from lower-magnification images, as in the case of the first embodiment. In this embodiment, since the resolution of the high-magnification image is one, only the alignment of the wide-angle image D _L and the high-magnification image D _H is performed.

Note that any known method can be used as the inter-image similarity degree or the coordinate conversion method. In the present embodiment, a three-dimensional correlation coefficient is used as the inter-image similarity degree, and a three-dimensional Affine conversion is used as the coordinate conversion method. To align.

<Step S550>
The display control unit 131 displays the high-magnification image D _Hj on the wide-angle image D _L based on the value of the alignment parameter obtained in S540. In the present embodiment, since the wide-angle image D _L and the high-magnification image D _Hj are both three-dimensional tomographic images, the following two types of display are performed.
i) A projection image of the wide-angle image D _L and the high-magnification image D _Hj is generated in the z-axis direction, and the projection image of the high-magnification image D _H is superimposed and displayed on the projection image of the wide-angle image D _L.
ii) In the position acquired only wide angle of view three-dimensional tomogram D _L in the pixel values of the wide field of view image 3-dimensional tomographic image D _L, wide angle of view three-dimensional tomogram D _L and the high magnification three-dimensional tomogram D _Hj At the position acquired together, a wide-field-angle three-dimensional tomographic image D _L "displayed with the pixel values of the high-magnification three-dimensional tomographic image D _Hj is generated. Further, a specific scan on the wide-field-angle three-dimensional tomographic image D _L ". The position is displayed on the superimposed image of i) by an arrow, and the two-dimensional tomographic image of the wide-field-angle three-dimensional tomographic image D _L "cut out at the position of this arrow is displayed side by side with the superimposed image of i). In this display, not only the two-dimensional tomographic image of the wide-field-angle three-dimensional tomographic image D _L , but also the two-dimensional tomographic image of the high-magnification three-dimensional tomographic image D _Hj is superimposed and displayed.
In the display of ii), the operator can move (up and down or left and right) the arrow indicating the display position of the wide-field tomographic image D _L "through the instruction acquisition unit 140. The display slices of the wide-angle image D _L and the high-magnification image D _Hj displayed as a change also change.

Further, when a plurality of high-magnification images D _Hj with different acquisition positions are acquired as in the present embodiment, the same method as in the first embodiment is used to determine the brightness characteristics between the high-magnification images D _Hj. Make similar adjustments. Furthermore, when the image pickup positions of the high-magnification images D _Hj are close to each other (including the case where the image pickup positions are the same), the display method of the overlapping regions is set to one of the following. That is, the image quality index value of the image is calculated and the image with the highest evaluation value is displayed, or the brightness of each high-magnification image D _Hj is blended by weighting the transparency based on the image quality index value described above. .. Here, any known index can be used as the image quality index value, and in this embodiment, the average luminance value of the image histogram is used.
The method of generating the projection image is not limited to average value projection, and any projection method may be used. The high-magnification image D _Hj is not limited to a still image and may be a moving image.

(High-magnification image acquisition process)
Next, the details of the process (high-magnification image acquisition process) executed in S530 will be described with reference to the flowchart shown in FIG. Since S1510 is the same as S810 of the first embodiment, the description thereof is omitted.

<Step S1520>
The image feature acquisition unit 134 uses the wide field angle image D _L stored in the storage unit 120, that is, the three-dimensional tomographic image of the eye portion, as image features, the inner limiting membrane B1, the nerve fiber layer boundary B2, and the inner plexiform layer boundary. B4, inner cell outer segment boundary B5, and retinal pigment epithelium boundary B6 are extracted. FIG. 7A, FIG. 15I, and FIG. 15J schematically show the boundary positions B1 to B6. Then, each extracted image feature is stored in the storage unit 120.

Here, the feature extraction procedure for the wide-angle image D _L will be specifically described. First, an extraction procedure for extracting a layer boundary will be described. Here, the three-dimensional tomographic image to be processed is considered as a set of two-dimensional tomographic images (B scan images), and the following process is performed on each two-dimensional tomographic image. First, the two-dimensional tomographic image of interest is subjected to smoothing processing to remove noise components. Next, edge components are detected from the two-dimensional tomographic image, and some line segments are extracted as layer boundary candidates based on their connectivity. Then, the first line segment from the extracted candidates is extracted as the inner limiting membrane B1, the second line segment from the top is extracted as the nerve fiber layer boundary B2, and the third line segment is extracted as the inner plexiform layer boundary B4. Further, a line segment having the maximum contrast on the outer layer side (the side with a larger z coordinate in FIG. 7A) than the inner limiting membrane B1 is extracted as the photoreceptor inner segment/outer segment boundary B5. Furthermore, the bottom line segment of the layer boundary candidate group is extracted as the retinal pigment epithelium boundary B6.

It should be noted that a variable shape model such as Snakes or the level set method may be applied with these line segments as initial values to perform more precise extraction. Further, the boundary of layers may be extracted by the graph cut method. Boundary extraction using a variable shape model or a graph cut may be performed three-dimensionally for a three-dimensional tomographic image or two-dimensionally for each two-dimensional tomographic image. Further, it goes without saying that any method may be used as the method of extracting the layer boundary as long as the layer boundary can be extracted from the tomographic image of the eye part.

<Step S1530>
The magnification determination unit 1321 determines the type of magnification (magnification number, angle of view, and pixel size) of the high-magnification image D _Hj . In this embodiment, the case where the number of magnifications and the pixel size are fixed (each 1 and 1 [μm]×1 [μm]×1 [μm]) will be described, and detailed description thereof will be omitted. The angle of view and the pixel size are different from those of the first embodiment in that they also include parameters in the z-axis direction. The angle of view is a variable parameter, and only the high-magnification image in which the distance between the photoreceptor inner-segment outer-segment boundary B5 and the retinal pigment epithelium boundary B6 obtained in S1520 is equal to or greater than the threshold Trd is the threshold Ta [%]. Enlarge. The reason for increasing the angle of view is to prevent image leakage due to fixation disparity because it is an important region for observation, and the outer segment of photoreceptor cells above the retinal detachment region is icy like a retinal pigment epithelium boundary. This is because it may extend in the B6 direction, and the entire photoreceptor cells can be acquired in a high-magnification image.

Next, the position determination unit 1322 determines the acquisition position and the coherence gate position of each high-magnification image D _Hj . In this embodiment, all parameters are variable, and the acquisition position of the high-magnification image D _Hj is determined by the following procedure.
a) Arranging the representative positions of the image acquisition pattern.
b) Determine the arrangement of the image acquisition pattern in the xy plane direction.
c) Determine the arrangement of the image acquisition pattern in the z-axis direction.

Here, regarding a), the representative position of the image acquisition pattern is set as the center of the image acquisition pattern, and the center is arranged so that the center coincides with the position of the center of gravity on the retinal detachment area. The retinal detachment area refers to an area in which a distance between the photoreceptor inner-outer segment boundary B5 and the retinal pigment epithelium boundary B6 is equal to or larger than the threshold value Trd is projected on the xy plane.
Regarding b), in order to include the retinal detachment area within the area of the image acquisition pattern, the arrangement in the xy directions of the high-magnification image is determined by the following procedure. That is, the circle connecting the image centers of the high-magnification images at the outermost periphery is obtained, the circle is enlarged to the position where it becomes the circumscribed circle of the retinal detachment area, and the high magnification is applied so that the circle area is filled at regular intervals. Determine the xy position of the image.
Regarding the acquisition position in the z-axis direction of c), it is determined so that the photoreceptor inner-segment outer-segment boundary B5 acquired in S1520 coincides with the image center of the high-magnification image. Further, the coherence gate of each high-magnification image D _Hj is set to the position closest to the photoreceptor inner-segment outer-segment boundary B5 detected in S1520 among the settable positions.

FIG. 15I shows the initial acquisition position of the image acquisition pattern in this embodiment, and FIG. 15J shows the acquisition position determined in this step. However, in both figures, in order to make the acquired pattern easy to understand, the angle of view of the high-magnification image is large on the retinal detachment area and the overlap between the high-magnification images is large only in the acquisition positions of the two central rows in the image acquisition pattern. Are omitted. The types of variable parameters are not limited to the above, and any image acquisition parameter may be used as the variable parameter.

<Step S1540>
The order determination unit 1324 determines the acquisition order of the high-magnification images D _Hj . In the present embodiment, among the following i) to iv), i) is the innermost (highest priority), ii) is the second innermost, iii) is the third innermost, and iv) is the outermost (lowest priority). Iterative processing is performed so that it becomes a loop of (degree). That is, the procedure of the acquisition start position (in this embodiment, the upper end of the image acquisition pattern is set, the acquisition magnification is set to the minimum magnification, and then the following i) to iv) is executed.
i) The images at the same arrangement pattern, the same acquisition magnification, and the same image acquisition position are repeatedly acquired the number of times.
ii) Images having the same arrangement pattern and the same acquisition magnification are moved to adjacent image acquisition positions and then acquired again in the same manner as i).
When iii) ii) is completed, the value of the acquisition magnification is increased and the operation of ii) is executed again, and the same operation is repeated for the number of magnifications.
iv) When iii) is completed, the operation of iii) is executed in another arrangement, and the operation is repeated until images are acquired in all the arrangements.
In the above example, i) is not repeatedly acquired (the number of times of acquisition is only once), and the image acquisition pattern is not a multi-arrangement type, so the process of iv) is omitted. The adjacent image acquisition positions of ii) can be moved in any direction, and in the present embodiment, the acquisition positions are moved to adjacent positions in the lateral direction (diagonally below if not lateral, or directly below if not below). Let That is, among the high-magnification image acquisition positions, the first step is from right to left, the second step is from left to right, and the third step is from right to left. The order determination method of the present invention is not limited to the above procedure, and any known order setting method may be used.

<Step S1550>
The high-magnification image acquisition unit 112 requests the tomographic image capturing apparatus 60 to acquire the plurality of high-magnification images D _Hj and the fixation target position F _Hj using the image acquisition parameters instructed by the determination unit 132. Since the tomographic image capturing device 60 acquires and transmits the high-magnification image D _Hj and the fixation target position F _Hj in response to the acquisition request, the high-magnification image acquiring unit 112 transmits the high-magnification image acquiring unit 112 from the tomographic image capturing device 60 via the LAN 30. The high-magnification image D _Hj and the fixation target position F _Hj are received. The high-magnification image acquisition unit 112 stores the received high-magnification image D _Hj and fixation target position F _Hj in the storage unit 120.

In addition, in the present embodiment, the acquisition position of the high-magnification image D _Hj is determined using the image feature regarding the photoreceptor layer boundary, but the present invention is not limited to this. For example, as in the case of the first embodiment, it may be determined by the operator operating (moving, enlarging, reducing) the position of the image acquisition pattern of the high-magnification image and collectively adjusting the acquisition position.

As described above, when acquiring a plurality of high-magnification adaptive-optical OCT tomographic images, the ophthalmologic apparatus 10 is based on the image characteristics related to the layer shape extracted from the OCT tomographic image having a wider angle of view than the high-magnification image. Determine parameter values for acquisition of multiple high-magnification images. As a result, it is possible to efficiently image a tissue, a cell group, or a lesion candidate that has a wider distribution than the high-magnification image and has a different distribution depending on the eye to be examined.

<<Other Embodiments>>
In the above embodiment, the alignment target image is realized as an SLO image or an eye tomographic image, but the present invention is not limited to this. For example, the wide-angle image D _L may be realized as a fundus camera image, and the high-magnification image D _H may be realized as a compensated optical fundus camera image. Further, the wide-angle image D _L may be realized as a wide-angle SLO image, and the high-magnification image D _H may be realized as images having different modalities such as a projected image of an adaptive optical tomographic image. Further, it may be realized as a configuration in which the compound machine of the adaptive optics SLO image pickup device 20 and the tomographic image pickup device 60 and the ophthalmologic apparatus 10 are directly connected.

The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or apparatus reads the program. This is the process to be executed.

10: ophthalmic apparatus, 20: SLO image capturing apparatus, 60: tomographic image capturing apparatus,
130: Image processing unit, 134: Image feature acquisition unit

Claims (7)

Each of the plurality of imaging regions divided so as to partially imaging target area of the eye overlaps an image acquisition means for acquiring a plurality of images of each imaging area captured a plurality of times,
Using a plurality of images of each of the imaging area, and the information obtaining means for obtaining blood vessel information on the blood vessel of each of the imaging area,
An information processing apparatus , comprising: a generation unit configured to generate an image of blood vessel information of the imaging target area based on the acquired blood vessel information of each imaging area . Further comprising a detection means for detecting a blood vessel region from the plurality of images,
The information processing apparatus according to claim 1, wherein the information acquisition unit acquires, as the blood vessel information, a boundary of the avascular region from the detected blood vessel region . The information processing apparatus according to claim 1 , wherein the imaging area is an area obtained by dividing the imaging target area based on a preset rectangular image acquisition pattern . The information processing according to any one of claims 1 to 3, wherein the plurality of images are fundus images generated based on return light from the fundus of the eye to be inspected that is scanned with measurement light. apparatus. Data capturing means for capturing pulse wave data at the same time when capturing the plurality of images,
5. The display device according to claim 1, further comprising a display unit that displays a plurality of images for each of the imaging regions in the imaging target region generated by the generation unit in synchronization with the pulse wave data. The information processing device according to item. Each of the plurality of imaging regions divided so as to partially imaging target area of the eye overlaps an image acquiring step of acquiring a plurality of images of each imaging area captured a plurality of times,
Using a plurality of images of each of the imaging area, and the information acquisition step of acquiring blood vessel information on the blood vessel of each of the imaging area,
And a generation step of generating an image of the blood vessel information of the imaging target area based on the acquired blood vessel information of each imaging area . The means of the information processing apparatus according to any one of claims 1 to 5, a computer program for implementing by a computer.