JP2020044388A - Information processing apparatus, control method for information processing apparatus, and computer program - Google Patents

Abstract

To provide a technique capable of efficiently and appropriately imaging, in a wider range than the field angle of a high magnification image, candidates for tissue, cell group or lesion which are differently distributed for each subject eye.SOLUTION: An information processing apparatus that controls, in a given imaging area, the capturing of plural high magnification images having a field angle smaller than the field angle of the imaging area comprises: presentation means for presenting, to an operator in such a manner as to be selectable for the user, plural basic patterns showing distributions of plural positions in which the high magnification images are captured; adjustment means for adjusting, based on a direction of the operator, the imaging conditions about the capturing of the plural high magnification images associated with a basic pattern selected from among the plural basic patterns; and control means for causing an imaging device to capture the plural high magnification images in the imaging area according to the adjusted imaging conditions.SELECTED DRAWING: Figure 3

Description

  The present invention relates to an information processing device used for ophthalmic medical care, a method for controlling the information processing device, and a computer program.

  Eye examinations are widely performed for the purpose of early medical treatment of lifestyle-related diseases and diseases that are the leading cause of blindness. A scanning laser ophthalmoscope (SLO) is known as an ophthalmologic apparatus using the principle of a confocal laser microscope. 2. Description of the Related Art A scanning laser ophthalmoscope is a device that performs a raster scan of a laser, which is measurement light, on the fundus and obtains a planar image with high resolution and high speed from the intensity of the returned light. Hereinafter, a device that captures such a planar image is referred to as an SLO device, and the captured planar image is referred to as an SLO image.

  In recent years, by increasing the beam diameter of the measurement light in the SLO apparatus, it has become possible to acquire an SLO image of the retina having a lateral resolution improved compared to the related art. However, with the increase in the beam diameter of the measurement light, a decrease in the S / N ratio and the resolution of the SLO image due to aberrations in the eye to be examined has been a problem in acquiring an SLO image of the retina. In order to solve the problem, an adaptive optics SLO apparatus having an adaptive optics system that measures aberration in an eye to be inspected in real time by a wavefront sensor and corrects aberration of measurement light generated in the eye to be inspected and its return light by a wavefront correction device. Is being developed. As a result, it is possible to acquire an SLO image having a high lateral resolution.

  Such an SLO image with high lateral resolution can be acquired as a moving image, and various biological information can be measured using the moving image. For example, in order to observe blood flow dynamics in a non-invasive manner, a retinal blood vessel is extracted from each frame, and a moving speed of blood cells in a capillary blood vessel is measured. Further, in order to evaluate the association with the visual function using the SLO image, the density distribution and arrangement of the visual cells P are measured after detecting the visual cells P.

  However, since the angle of view of one SLO image with high lateral resolution that can be imaged by the adaptive optics SLO apparatus is generally small, if the imaging target area is larger than the angle of view of the SLO image with high lateral resolution, the imaging area in the imaging target area The problem is how to set. This will be described with reference to FIG. FIG. 7A is a diagram schematically illustrating a cross-sectional view of the eye to be inspected. FIGS. 7B to 7G are diagrams illustrating examples of an SLO image or an imaging target area.

  FIG. 7B is a diagram illustrating an example of an SLO image with high lateral resolution. In FIG. 7B, a low luminance area Q corresponding to the position of the photoreceptor cells P and capillaries, and a high luminance area W corresponding to the position of the white blood cell are observed. When observing the photoreceptor cells P or measuring the distribution of the photoreceptor cells P, the focus position is set near the outer layer of the retina (B5 in FIG. 7A) and an SLO image as shown in FIG. I do.

  On the other hand, retinal blood vessels and branched capillaries run in the inner layer of the retina (B2 to B4 in FIG. 7A). In particular, in the case of the affected eye, the imaging target area is often larger than the angle of view of one SLO image that can be imaged by the SLO apparatus. FIGS. 7C and 7D show examples in which the imaging target area is larger than the angle of view of the SLO image. FIG. 7 (c) shows an example of a site where capillary lesions are likely to occur (an annular region surrounded by a broken line), and FIG. 7 (d) shows an example of an extensive photoreceptor cell-deficient region (black closed region). I have. In the case of FIG. 7C and FIG. 7D, if all of the imaging target areas are acquired at a high magnification, setting of the imaging conditions of a large number of SLO images becomes complicated, or the imaging time becomes long, and There was a problem that the burden increased. In the imaging target area, a high-magnification image is acquired in an examination time that does not burden the subject because an area in which it is necessary to obtain a high-magnification image in medical care and an area in which the necessity is low are mixed. It is necessary to appropriately set the imaging region so that all the high-needed regions can be imaged.

  In this regard, Japanese Patent Application Laid-Open No. H11-163873 discloses a technique related to parameter setting for acquiring a plurality of high-magnification images, in which a plurality of adaptive optics SLO images are captured at different imaging positions and displayed as a panoramic image. Are listed.

JP 2012-213513 A

However, when the cell group, tissue, or lesion area to be observed or measured in detail is distributed more widely than the area covered by the image with high lateral resolution (high-magnification image D _H ), the area such as the cell group can be efficiently In the case of imaging, the conventional configuration has the following problems. That is,
i) The operator needs to individually specify the values of the acquisition parameters (for example, the acquisition position, the angle of view, the pixel size, the number of frames, the frame rate, etc.) of the plurality of high-magnification images D _Hj , and the efficiency of acquiring the plurality of images is high. Had been hindered.
ii) When an observation area wider than the high-magnification image _DH is imaged with the same high-magnification image acquisition parameter, the number of high-magnification images to be acquired (total number of frames) becomes enormous (thousands to tens of thousands), and efficiency is increased. Image acquisition was difficult.

  In the configuration of Patent Literature 1, the acquisition parameters of a large number of high-magnification images are manually determined for each image, and the operator has to perform complicated work for setting the acquisition parameters.

  The present invention has been made in view of the above problems, and has as its object to provide a technique capable of easily setting an appropriate imaging condition according to an imaging target.

In order to achieve the above object, an information processing apparatus according to the present invention has the following configuration. That is,
An information processing apparatus that controls imaging of a plurality of images in an imaging target region of the subject's eye,
Selecting means for selecting one basic pattern from a plurality of basic patterns indicating an arrangement of a plurality of imaging regions for imaging the plurality of images;
Setting means for setting the number of the plurality of imaging regions;
Control means for causing an imaging device to image the plurality of images in the area of the eye to be inspected in accordance with the number of imaging areas set by the setting means and the arrangement of the imaging areas of the selected basic pattern.

  According to the present invention, it is possible to easily set an appropriate imaging condition according to an imaging target.

FIG. 1 is a block diagram illustrating a configuration example of a system including an ophthalmologic apparatus. FIG. 2 is a block diagram illustrating a hardware configuration example of the ophthalmologic apparatus 10. FIG. 2 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. FIG. 2 is a diagram illustrating an overall configuration of an SLO image imaging device 20. 9 is a flowchart of a process executed by the ophthalmologic apparatus 10. The figure explaining an image acquisition pattern. FIG. 3 is a view for explaining image processing contents. 9 is a flowchart illustrating details of a high-magnification image acquisition process. 9 is a flowchart illustrating details of an image display process. FIG. 2 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. 9 is a flowchart illustrating details of a high-magnification image acquisition process. FIG. 7 is a diagram illustrating an exception frame included in an image acquisition pattern and a high-magnification moving image. FIG. 2 is a block diagram showing a functional configuration example of the ophthalmologic apparatus 10. FIG. 2 is a diagram illustrating an overall configuration of a tomographic imaging apparatus 60. The figure explaining an image acquisition pattern. 9 is a flowchart illustrating details of an image display process.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<< First Embodiment >>
When acquiring a plurality of high-magnification adaptive optics SLO images, the ophthalmologic apparatus as the information processing apparatus according to the present embodiment operates a basic pattern of parameters related to imaging conditions for acquiring a plurality of high-magnification images prepared in advance. To the operator (user) to make the operator select. Next, if necessary, the operator adjusts the image acquisition parameters according to the lesion shape, and determines the acquisition parameter values for the plurality of high-magnification images according to the details of the adjustment. Hereinafter, the operator selects a basic pattern for acquiring a plurality of images in a disc shape for a wide range of photoreceptor defect areas of the macula, and acquires the acquisition position, angle of view, pixel size, number of frames, frame of each high magnification image. A case where the rate and the focus position are determined will be described as an example.

(overall structure)
FIG. 1A is a configuration diagram of a system including the ophthalmologic apparatus 10 according to the present embodiment. As shown in FIG. 1A, an ophthalmologic apparatus 10 includes an SLO image capturing apparatus 20 or a data server 40 as an image capturing apparatus, a local area network (LAN) 30 including an optical fiber, USB, IEEE1394, or the like. Connected through. In addition, the form of connection of these devices is not limited to the example of FIG. For example, these devices may be connected via an external network such as the Internet, or the ophthalmologic apparatus 10 may be directly connected to the SLO image capturing apparatus 20.

The SLO image capturing device 20 is a device that captures (photographs) a wide angle-of-view image _DL and a high-magnification image _DH of the fundus. SLO image capturing apparatus 20, wide field of view image D _L and the high magnification image D _H, and fixation target position F _L used in the time of imaging, information F _H, and transmits to the ophthalmic device 10 and data server 40.

When images at respective magnifications are acquired at different imaging positions, the acquired images are represented as D _Li and D _Hj . That is, i and j are variables indicating the imaging position number, and i = 1, 2,. . . , Imax, j = 1, 2,. . . , Jmax. When acquiring high magnification images at a plurality of different magnifications, D _1j , D _{2k ,.} . . , And D _1j having the highest magnification is a high magnification image, D _2k,. . . Is referred to as an intermediate magnification image.

The data server 40 holds imaging condition data, image features of the eye, normal values related to the distribution of the image features of the eye, and the like. As the imaging condition data, output by the SLO image capturing apparatus 20, a fixation target position F _L, F _H, ophthalmologic apparatus 10 outputs using wide-angle image D _L of the eye, the high magnification image D _H, the time of imaging The image features of the eye are stored in the data server 40. In the present embodiment, image features related to photoreceptor cells P, capillaries Q, blood cells W, retinal vessels, and retinal layer boundaries are handled as image features of the eye. In addition, in response to a request from the ophthalmologic apparatus 10, the data server 40 transmits the wide-angle image D _L , the high-magnification image D _H , the image feature of the eye part, and normal value data of the image feature to the ophthalmologic apparatus 10.

(Ophthalmic equipment)
The ophthalmologic apparatus 10 is realized by an information processing device such as an embedded system, a personal computer (PC), and a tablet terminal. The hardware configuration of the ophthalmologic apparatus 10 will be described with reference to FIG. In FIG. 2, a CPU 301 is a central processing unit, and controls the operation of the entire ophthalmologic apparatus in cooperation with other components based on a computer program such as an operating system (OS) and an application program. The RAM 302 is a writable memory and functions as a work area for the CPU 301. The ROM 303 is a read-only memory and stores programs such as a basic I / O program, data used for basic processing, and the like. The external storage device 304 is a device that functions as a large-capacity memory, and is realized by a hard disk device and a semiconductor memory. The monitor 305 is a display device as a display unit that displays a command input from the keyboard 306 or the pointing device 307 and a response output of the ophthalmologic apparatus 10 to the command. The keyboard 306 and the pointing device 307 are devices that receive instructions and command inputs from the operator. The interface 308 is a device that relays data exchange with an external device.

  A control program for realizing the image processing function according to the present embodiment and data used when the control program is executed are stored in the external storage device 304. Under the control of the CPU 301, these control programs and data are appropriately loaded into the RAM 302 via the bus 309, executed by the CPU 301, and function as each unit described below.

  Next, a functional configuration of the ophthalmologic apparatus 10 according to the present embodiment will be described with reference to FIG. FIG. 3 is a block diagram illustrating a functional configuration of the ophthalmologic apparatus 10. As shown in FIG. 3, the ophthalmologic apparatus 10 includes a data acquisition unit 110, a storage unit 120, an image processing unit 130, and an instruction acquisition unit 140.

  The data acquisition unit 110 is a functional block that acquires data such as image data and imaging condition data. The data acquisition unit 110 includes a wide-angle image acquisition unit 111 that acquires a wide-angle image, and a high-magnification image acquisition unit 112 that acquires a high-magnification image. The storage unit 120 is a functional block that holds the data acquired by the data acquisition unit 110 and the image acquisition pattern set 121. The image acquisition pattern set 121 is a set of basic setting patterns (hereinafter, referred to as “image acquisition patterns”) related to parameters for acquiring a plurality of high-magnification images.

  The image processing unit 130 is a functional block that performs processing such as determination of imaging conditions, setting of the imaging conditions, and display of a captured image. The image processing unit 130 includes a display control unit 131 that performs display control of a captured image and the like, a determination unit 132 that determines imaging conditions, and a positioning unit 133 that performs positioning of an imaging region based on the imaging conditions. The display control unit 131 includes an image acquisition pattern presenting unit 1311 that displays an image acquisition pattern on a monitor and presents the image acquisition pattern to an operator. The determination unit 132 includes a magnification determination unit 1321 that determines the magnification of imaging, a position determination unit 1322 that determines the imaging position, a time determination unit 1323 that determines the timing of imaging, and an order determination unit 1324 that determines the order of imaging. .

(SLO image pickup device)
Next, a configuration example of the SLO image capturing apparatus 20 including the adaptive optical system will be described with reference to FIG. Note that the configuration of the SLO image capturing device described below is an example, and any type of capturing device can be used as long as a high-magnification image can be obtained.

  Reference numeral 201 denotes a light source, and FIG. 4 shows an example realized by an SLD light source (Super Luminescent Diode). In the present embodiment, both the light source for fundus imaging and the light source for wavefront measurement are realized by the light source 201, but they may be configured as separate light sources and multiplexed in the middle of the optical path.

  The light emitted from the light source 201 passes through a single-mode optical fiber 202 and is emitted by a collimator 203 as parallel measurement light 205. The irradiated measurement light 205 passes through the light splitting unit 204 composed of a beam splitter, and is guided to an optical system of adaptive optics.

  The adaptive optics includes a light splitting unit 206, a wavefront sensor 215, a wavefront correction device 208, and reflection mirrors 207-1 to 207-4 for guiding the light to them. Here, the reflection mirrors 207-1 to 207-4 are installed so that at least the pupil of the eye and the wavefront sensor 215 and the wavefront correction device 208 have an optically conjugate relationship. In the present embodiment, a beam splitter is used as the light splitting unit 206. In the present embodiment, a spatial phase modulator using a liquid crystal element is used as the wavefront correction device 208. Note that a configuration using a deformable mirror as the wavefront correction device may be adopted. The light that has passed through the adaptive optics is scanned one-dimensionally or two-dimensionally by the scanning optical system 209.

  In this embodiment, two galvanometer scanners are used for the main scanning (horizontal fundus direction) and the sub-scanning (vertical fundus direction) as the scanning optical system 209. However, a resonance scanner may be used on the main scanning side of the scanning optical system 209 for faster imaging.

  The measurement light 205 scanned by the scanning optical system 209 is applied to the eyeball 211 through the eyepieces 210-1 and 210-2. The measurement light 205 applied to the eyeball 211 is reflected or scattered by the fundus. By adjusting the positions of the eyepieces 210-1 and 210-2, it is possible to perform optimal irradiation in accordance with the diopter of the eyeball 211. Here, a lens is used for the eyepiece, but it may be constituted by a spherical mirror or the like.

  The reflected and scattered light (return light) reflected or scattered from the retina of the eyeball 211 travels in the same path as the path of the incident light in the opposite direction, and is partially reflected by the light splitting unit 206 to the wavefront sensor 215, Is used to measure the wavefront of The wavefront sensor 215 is connected to the adaptive optics controller 216 and transmits the received wavefront to the adaptive optics controller 216. The wavefront correction device 208 is also connected to the adaptive optics control unit 216, and performs the modulation specified by the adaptive optics control unit 216. Based on the wavefront measured by the wavefront sensor 215, the adaptive optics control unit 216 calculates a modulation amount (correction amount) for correcting the wavefront reaching the wavefront correction device 208 to a wavefront having no aberration, and performs wavefront correction. Instruct device 208 to do so. Note that the measurement of the wavefront and the instruction to the wavefront correction device 208 are repeatedly processed, and the feedback control is performed so as to always obtain the optimum wavefront.

A part of the reflected scattered light transmitted through the light splitting unit 206 is reflected by the light splitting unit 204, and is guided to the light intensity sensor 214 through the collimator 212 and the optical fiber 213. The light is converted into an electric signal by the light intensity sensor 214, formed into an image as an eye image by the control unit 217, and displayed on the display 218. By increasing the swing angle of the scanning optical system in the configuration of FIG. 4 and instructing the adaptive optics control unit 216 not to perform aberration correction, the SLO image capturing apparatus 20 operates as a normal SLO apparatus, and An SLO image with a wide angle of view (wide-angle image D _L ) can be captured.

(Processing procedure)
Specific processing contents executed by the ophthalmologic apparatus 10 will be described in detail in association with the role of each functional block. FIG. 5 is a flowchart illustrating a processing procedure executed by the ophthalmologic apparatus 10. The following steps are executed under the control of the CPU 301.

<Step S510>
Wide field of view image obtaining unit 111, to the SLO image capturing device 20, requests the acquisition of the wide field of view image D _L and fixation target position F _L. In the present embodiment, an example in the case of obtaining a wide angle image D _L by setting the fixation target position F _L fovea of the macula portion. The method of setting the imaging position is not limited to this, and may be set to another arbitrary position.

SLO image capturing device 20, a wide angle of view wide angle image D _L in response to the acquisition request from the image acquisition unit 111 acquires the fixation target position F _L, and transmits to the wide field of view image obtaining unit 111. Wide field of view image obtaining unit 111 receives the wide angle image D _L and fixation target position F _L through the LAN30 from SLO image capturing device 20. Wide field of view image acquisition unit 111 stores the wide-angle image D _L and fixation target position F _L received in the storage unit 120. In the present embodiment, the wide-angle-of-view image D _L is a moving image whose frames have been aligned.

<Step S520>
The image acquisition pattern presenting unit 1311 acquires at least one type of image acquisition pattern (a basic setting pattern relating to parameters for acquiring a plurality of high-magnification images) from the storage unit 120 and displays the acquired pattern on the monitor 305 in a selectable manner. Although any pattern can be presented as the image acquisition pattern, the present embodiment describes a case where basic patterns as shown in FIGS. 6A to 6F are presented. 6A is a linear shape, FIG. 6B is a cross shape, FIG. 6C is a radial shape, FIG. 6D is a rectangular shape, FIG. 6E is a disk shape, and FIG. Shows an example of an annular pattern.

  Next, the instruction acquiring unit 140 externally acquires an instruction related to selection of an image acquisition pattern desired by the operator. This instruction is input by the operator via the keyboard 306 or the pointing device 307, for example. Alternatively, when the monitor 305 includes a touch panel liquid crystal, the input may be performed via the touch panel. In the present embodiment, since the observation target is a disc-shaped photoreceptor cell defective area as shown in FIG. 7D, a disc-shaped image acquisition pattern as shown in FIG. 6E is selected.

Note that the image acquisition pattern is not only a pattern composed of only one type of high-magnification image as shown in FIGS. 6A to 6F, but also a combination of images of a plurality of magnifications. Can be presented as an image acquisition pattern. For example, as shown in FIG. 6G, not only the high-magnification image D _1j but also the intermediate-magnification image D _2k may be included in the acquisition pattern and defined. Hereinafter, such an acquisition pattern including not only a high-magnification image but also an intermediate-magnification image is referred to as a “multi-magnification type image acquisition pattern”. Such an image acquisition pattern, and if you want to reduce the number of acquired images are suitable if you want to align the wide field of view image D _L more accurately. In the multi-magnification image acquisition pattern, the shape of the image acquisition pattern formed by the high-magnification image or the intermediate-magnification image at each magnification may be different for each magnification or may be the same. For example, the intermediate magnification image D _2k having a lower magnification may be acquired in a rectangular shape, and the high magnification image D _1j may be acquired in a disk shape. If the image acquisition patterns are different between different magnifications, information of the type of the image acquisition pattern at each magnification is also acquired in this step. FIG. 6G shows a case where the shape of the image acquisition pattern formed by the high-magnification image D _1j and the shape of the image acquisition pattern formed by the intermediate-magnification image D _2k are both cross-shaped and identical. .

  In addition, as shown in FIG. 6 (h), a plurality of basic patterns arranged at different positions (hereinafter, referred to as a “multi-arranged image acquisition pattern”) may be presented. FIG. 6H shows a case where a plurality of rectangular image acquisition patterns are arranged. This is suitable for a case where there are a plurality of lesions, a case where the morphology and dynamics of the observation target are compared for each site, and the like. Note that the multiple arrangement type image acquisition pattern includes a case where an image is acquired by changing the focus position between basic patterns. Further, as shown in FIG. 6I, an image acquisition pattern defined by a combination of basic patterns (hereinafter, referred to as a “composite image acquisition pattern”) may be presented. The composite image acquisition pattern is suitable for efficiently acquiring images for different purposes in one inspection. For example, FIG. 6 (i) shows a case of measuring the shape of the avascular region (closed white line region in FIG. 6 (i)) in the fovea (the center of gravity of the avascular region) indicated by the black dots in FIG. ) And the case where the photoreceptor cell density is measured at a fixed distance from the fovea (cross-shaped).

<Step S530>
The determination unit 132 sets the acquisition parameters of the plurality of images included in the image acquisition pattern selected in S520 as initial values, and determines the acquisition parameters of the plurality of high-magnification images by adjusting the image acquisition parameters as needed by the operator. To obtain an image. The processing of this step (hereinafter referred to as “high-magnification image acquisition processing”) will be described later in detail with reference to the flowchart shown in FIG.

<Step S540>
Positioning unit 133 performs positioning of the wide field of view image D _L and the high magnification image D _H, determine the relative position of the high magnification image D _H on the wide field of view image D _L. And the positioning means to set the position of the D _H to the corresponding position of the automatic determination to D _L the positional relationship of the wide field of view image D _L and the high magnification image D _H. Note that the wide-angle image D _L is an image having a lower magnification than the high-magnification image showing the entire imaging region, and is a given image given in advance. When there is an overlapping area between the high-magnification images D _Hj , first, the inter-image similarity is calculated for the overlapping area, and the positions of the high-magnification images D _Hj are adjusted to the position where the inter-image similarity becomes maximum. Next, when high magnification images having different resolutions have been acquired in S530, alignment is performed sequentially from the image with the lower magnification. For example, when a high-magnification image D _1j and an intermediate-magnification image D _2k have been acquired as the high-magnification image D _H , first, alignment is performed between the wide-angle image D _L and the intermediate-magnification image D _2k, and then, Positioning is performed between the intermediate magnification image D _2k and the high magnification image D _1j . When the resolution of the high-magnification image is one type, it goes without saying that only the alignment between the wide-angle image D _L and the high-magnification image D _H is performed.

Incidentally, the positioning unit 133 acquires the fixation target position F _H used during imaging from the storage unit 120 the high magnification image D _H, position in alignment with the wide field of view image D _L and the high magnification image D _H Set as the initial point of search for matching parameters. In addition, any known method can be used as the similarity between images and the coordinate conversion method. In the present embodiment, the alignment is performed using the correlation coefficient as the similarity between images and the Affine transform as the coordinate conversion method.

<Step S550>
The display control unit 131, based on the obtained value of the alignment parameter in S540, displays a high magnification image D _H on wide field of view image D _L. The processing of this step (hereinafter referred to as “image display processing”) will be described later in detail with reference to the flowchart shown in FIG.

<Step S560>
The instruction acquisition unit 140 instructs whether or not to store the alignment parameter values acquired in the wide-angle image D _L , the high-magnification image D _H , the fixation target positions F _L , F _H , and S540 in the data server 40. From outside. This instruction is input by the operator via the keyboard 306 or the pointing device 307, for example. If storage is instructed (YES in S560), the process proceeds to S570, and if storage is not instructed, the process proceeds to S580 (NO in S560).

<Step S570>
The image processing unit 130 associates the examination date and time, the information for identifying the eye to be examined, the wide-angle image D _L and the high-magnification image D _H with the fixation target positions F _L and F _H and the alignment parameter values, and associates them with the data server 40. Send to

<Step S580>
The instruction acquisition unit 140 externally acquires an instruction as to whether or not to end the processing on the wide-angle image _DL and the high-magnification image _DH by the ophthalmologic apparatus 10. This instruction is input by the operator via the keyboard 306 or the pointing device 307. If an instruction to end the process has been obtained (YES in S580), the process ends. On the other hand, if an instruction to continue the process has been acquired (NO in S580), the process returns to S510, and the process for the next eye examination or the reprocessing for the same eye examination is performed.

(High magnification image acquisition processing)
Next, details of the high-magnification image acquisition processing executed in S530 will be described with reference to the flowchart shown in FIG.

<Step S810>
The determination unit 132 acquires from the storage unit 120 the type of the image acquisition pattern selected through the instruction acquisition unit 140 and the acquisition parameter value of each high-magnification image _{DH included in} the pattern. Specifically, among the acquisition parameters of each high magnification image held by the selected image acquisition pattern, the following parameter values are input as initial values. That is, the number of magnifications and the values of each angle of view and pixel size are obtained by the magnification determining unit 1321, the acquisition position and the in-focus position are obtained by the position determination unit 1322, the number of frames and the frame rate, and the number of times of repeated acquisition are obtained by the time determination unit 1323. The order is input to the order determining unit 1324 as an initial value.

<Step S820>
The deciding unit 132 acquires, through the instruction acquiring unit 140, the constraint condition regarding the set value of the acquisition parameter of each high-magnification image D _Hj constituting the selected image acquisition pattern. The constraint condition is a condition that defines a range that the imaging condition can take. The constraint conditions can be specified and set by the operator for any image acquisition parameters. In the present embodiment, a case where the following four constraints can be set will be described.
a) Total image acquisition time b) Type of magnification (magnification number, angle of view and pixel size)
c) Focus position d) Overlap area between adjacent high-magnification images
a) is the permissible time that the subject's eye can endure per examination,
b) the size of the image features expected to be obtained at the imaging position,
c) is the position in the depth (z-axis) direction where the observation target exists,
d) is an allowable constraint on the amount of fixation disparity of the subject's eye. In this embodiment, a) 15 minutes b) 1 and 300 [μm] × 300 [μm] and 1 [μm / pixel] × 1 [μm / pixel], respectively.
c) photoreceptor cell layer d) 20% of high magnification image area
An example will be described in the case where.

<Step S830>
The magnification determining unit 1321 determines the type of magnification (number of magnifications, angle of view, and pixel size) of the high-magnification image _DH . Further, the position determination unit 1322 determines the acquisition position and the in-focus position of each high-magnification image D _Hj .

In the present embodiment, the angle of view, the pixel size, and the in-focus position are fixed values due to the constraint conditions acquired in S820, but the acquisition position of each high-magnification image D _Hj is a variable parameter. Therefore, the operator first specifies the position on the fundus of the point (representative point) where the image acquisition pattern is to be formed. In the present embodiment, the representative point is the center point C in FIG. 6E and is set in the fovea of the eye to be inspected. Next, the operator enlarges or reduces the size of the entire image acquisition pattern, thereby increasing or decreasing the number of high-magnification image acquisition positions while maintaining the size of the overlapping area between adjacent high-magnification images. The acquisition position of the image D _Hj is determined. In the present embodiment, the size of the entire image acquisition pattern is enlarged by moving the position of the (one) high-magnification image located at the end of the image acquisition pattern to the outside of the disk by the operator, and FIG. The acquisition position of the high-magnification image D _Hj as shown by the white line rectangular area in g) is determined.

  Note that the representative point is not limited to the center point of the image acquisition pattern. For example, the position of a specific high-magnification image forming the image acquisition pattern may be used. In the case of a multi-magnification type image acquisition pattern as shown in FIG. 6 (g), the sizes of the image acquisition patterns of all magnifications may be enlarged or reduced together, or the image acquisition pattern may be acquired for each magnification. The size of the pattern may be changed.

  In the case of the multi-arrangement type image acquisition pattern shown in FIG. 6H, the image acquisition pattern sizes or arrangement intervals of all the arrangements may be changed collectively. Further, the size or arrangement interval of the image acquisition patterns may be different for each arrangement of the basic patterns. Further, in the case of a composite image acquisition pattern as shown in FIG. 6 (i), the sizes of all types of patterns may be changed collectively, or the size of the image acquisition pattern may be changed for each type of image acquisition pattern. May be changed.

<Step S840>
The time determining unit 1323 determines the number of frames, the frame rate, and the number of times of repeated acquisition of each high-magnification image. In the example of this embodiment, the frame rate and the number of times of repetition acquisition are each 32 [frame / sec] and one fixed value, and the number of frames is a variable parameter. The method of changing the variable parameter value can be specified using any known user interface (hereinafter abbreviated as “UI”). In the present embodiment, the parameter value (weight) changing UI as shown in FIG. 6 (j) is efficiently changed by the operator. This is a UI for adjusting the radial weight for the number of frames of each high magnification image D _Hj . In FIG. 6 (j), the center side of the disk-shaped image acquisition pattern on which Wc is arranged (FIG. 7 (g)), and Wo represents the outer weight. By reducing the outer weight Wo, the number of frames of each high-magnification image D _Hj is automatically determined so as to gradually decrease from the center of the pattern toward the outer side.

In the case of a multi-magnification pattern as shown in FIG. 6G, a parameter value (or weight) adjusting UI as shown in FIG. 6K can be used. Adjustment of the parameter value is performed in the following procedures i) to iv).
i) Select a variable parameter to be adjusted from the variable parameter list V.
ii) Select a target magnification and a target image for adjusting the parameter value from the adjustment map.
iii) Select a parameter value change (weighting) method R between a plurality of images at the selected magnification.
iv) The parameter value (weight) for the selected image is determined on the parameter value changing UI (B).
In i), the adjustment map of FIG. 6K is displayed for each type of the selected variable parameter. For ii), in FIG. 6 (k), a target magnification and D _1, shows a case where the D ₁ image D _1c of the heart selected as the adjustment target image.
Regarding iii), in FIG. 6 (k), as a parameter value setting (weighting) method R between images,
When the same value is set between a plurality of images of the same magnification (uniform)
・ When changing stepwise (gradual)
When the parameter value is changed for a specified individual image (individual)
In the example shown in FIG. In the example of FIG. 6 (k), the case of changing stepwise (gradual) is selected.
Regarding iv), in FIG. 6K, by specifying the maximum value (white) on the color bar B, the parameter value of the image having the same magnification as the selected target image D _1c is changed to the selected image D _1c. A case is shown in which automatic change is performed so as to gradually increase toward.

  In FIG. 6 (k), the parameter value changing UI is a color bar B, and the parameter values are expressed in gray scale. However, the present invention is not limited to this. For example, the parameter value changing UI may be a slider or a (numerical) list box. Further, the parameter value may be displayed as a numerical value (the parameter value itself or a weight) or may be displayed in color. Alternatively, the parameter values may be displayed using both numerical values and gray scale (color).

  Further, in the case of a multi-arrangement pattern as shown in FIG. 6 (h), as shown in FIG. 6 (l), the weight in each pattern (for example, Wm1dhi in FIG. 6 (l)) and the weight between the patterns (Wm1 and Wm2) are adjusted. Further, in the case of a composite image acquisition pattern as shown in FIG. 6 (m), the parameter values (Wai and Wci) for each pattern and the parameter values (Wbi) for the image common to both patterns are set. Do. The adjustment procedure is almost the same as in the case of the multi-magnification type image acquisition pattern shown in FIG. That is, the above procedure iii) is merely replaced with the procedure of selecting the parameter value change (weighting) method R between a plurality of images in the selected pattern or common area, instead of the selected magnification.

<Step S850>
The order determining unit 1324 determines the order of acquiring the high-magnification images D _Hj . In the present embodiment, among the following i) to iv), i) is the innermost (highest priority), ii) is the second innermost, iii) is the third innermost, and iv) is the outermost (lowest priority). Iterative processing is performed in a loop of degree). Specifically, the following procedure is executed after setting the acquisition start position to the position with the highest observational importance (the fovea in this embodiment) and the acquisition magnification to the minimum magnification.
i) The images at the same arrangement pattern, the same acquisition magnification, and the same image acquisition position are repeatedly acquired by the number of times of acquisition.
ii) An image with the same arrangement pattern and the same acquisition magnification is moved to an adjacent image acquisition position and acquired again in the same manner as in i).
iii) After ii) is completed, the value of the acquisition magnification is increased, the operation of ii) is performed again, and the same operation is repeated for the number of magnifications.
iv) When iii) is completed, the operation of iii) is executed in another arrangement, and the operation is repeated until images are acquired in all arrangements.

  In the present embodiment, i) is not repeatedly acquired (the number of acquisitions is only one), and the image acquisition pattern is not the multi-arrangement type as shown in FIG. The processing is omitted. In addition, the moving direction to the adjacent image in ii) may be moved in an arbitrary direction. However, in the present embodiment, the closer to the fovea, the stronger the influence on the visual function and the higher the importance of observation. Move in a spiral from the fovea.

  By the processing in S830 to S850, the operator can easily change the image acquisition parameters indicating the imaging conditions of the high-magnification image.

<Step S860>
The high-magnification image acquisition unit 112 requests the SLO imaging apparatus 20 to acquire a plurality of high-magnification images D _Hj and fixation target positions F _Hj using the image acquisition parameters instructed by the determination unit 132. . The SLO image capturing apparatus 20 acquires and transmits the high magnification image D _Hj and the fixation target position F _Hj in response to the acquisition request. The high magnification image D _Hj and the fixation target position F _Hj are received. The high-magnification image acquisition unit 112 stores the received high-magnification image D _Hj and fixation target position F _Hj in the storage unit 120. In the present embodiment, the high-magnification image D _Hj is a moving image whose frames have been aligned.

(Image display processing)
Next, details of the image display processing executed in S550 will be described with reference to the flowchart shown in FIG.

<Step S910>
A representative image is generated from each moving image acquired by the wide-angle-of-view image acquiring unit 111 and the high-magnification image acquiring unit 112. In the present embodiment, a superimposed image for each moving image is generated, and this superimposed image is set as a representative image. The method of generating the representative image is not limited to this. For example, a reference frame set at the time of inter-frame alignment of a moving image may be set as the representative image. As a method of setting the reference frame, any known setting method can be used. For example, the frame with the leading number can be set as the reference frame.

<Step S920>
The display controller 131 corrects the density difference between the high-magnification images when a plurality of high-magnification images D _Hj have been acquired. Any known luminance correction method can be applied to the correction of the density difference. As an example, in the present embodiment, a histogram is generated Hj in each high magnification image D _Hj, the luminance of each high magnification image D _Hj as mean and variance of the histogram Hj is common values between the high magnification image D _Hj The density difference is corrected by linearly converting the value.

<Step S930>
The display control unit 131, when displaying on the wide angle image D _L high magnification image D _H as a moving image, and sets the reproduction speed of the high magnification image D _H. The reproduction speed is adjusted by arranging a reproduction speed adjustment slider or a frame advance button in the image display area and designating the operator through the instruction acquisition unit 140.

  In the present embodiment, since the still image (superimposed image) generated in S910 is pasted and displayed, this processing is omitted.

<Step S940>
The display control unit 131 controls the display / non-display and the display magnification of each high magnification image D _Hj . For display / non-display of an image, a list of acquired images is displayed on the monitor 305, a UI (check box in the present embodiment) is arranged near the image name of the acquired image list, and the UI of the UI is acquired through the instruction acquisition unit 140. It is set by the operator specifying ON / OFF. A UI (check box) for collectively specifying all images and a collective specification UI (check box) for each acquisition magnification are also provided to facilitate switching between display and non-display of a large number of images.

In this step, not only the display / non-display of the image but also the setting of the overlapping order when there is an overlapping area between the adjacent high-magnification images D _Hj or when the same fixation target position is imaged a plurality of times. . As for the setting method of the overlapping order of the moving images, any setting method including manual setting can be used. In the present embodiment, the image quality index and the fixation disparity amount of the image are calculated, and the image having the highest evaluation value is set and displayed as the foremost layer using the linear sum of the image quality index value and the fixation disparity amount as an evaluation function. . Here, any known index can be used as the image quality index value, and in this embodiment, the average luminance value of the image histogram is used. As the fixation disparity amount, a value obtained by adding the absolute value of the translational movement distance between adjacent frames over all frames is used. Any index may be used as long as fixation disparity can be evaluated. As for the display magnification, a high-magnification image specified by the operator through the instruction acquisition unit 140 is enlarged and displayed on the monitor 305.

Although in the example above was taken up when wide field of view image D _L is a single wide angle SLO image, the present invention is not limited thereto. For example, a composite image obtained by aligning the wide angle image D _L i between the different acquisition position may be wide field of view image D _L.

  As described above, in the ophthalmologic apparatus 10 according to the present embodiment, the operator selects from a plurality of basic patterns relating to the high-magnification image acquisition parameters (imaging conditions), adjusts the parameter values according to the lesion shape, and A high-magnification image is obtained based on the adjusted parameter values. That is, the ophthalmologic apparatus 10 presents a plurality of basic patterns indicating the distribution of a plurality of positions where the high-magnification images are captured to the operator in a selectable manner. According to the operator's selection, the imaging conditions relating to the imaging of the high-magnification image, which are associated in advance with the basic pattern selected from the plurality of basic patterns, are adjusted according to the operator's instruction. Further, according to the adjusted imaging conditions, the imaging device captures a plurality of high-magnification images in the imaging region. For this reason, according to the present embodiment, it is possible to easily set appropriate imaging conditions for obtaining a plurality of high-magnification images having a smaller angle of view than the angle of view of a given imaging area. it can. Therefore, it is possible to efficiently image a tissue, a cell group, or a lesion candidate having a different distribution depending on the eye to be examined in a wider range than the high-magnification image.

In the present embodiment, the imaging condition is set based on at least one of the position of the representative point of the selected basic pattern in the imaging region, the constraint condition specified by the operator, and the amount of change in the imaging condition. adjust. Therefore, it is possible to easily set an appropriate imaging condition according to an imaging target. For example, there are individual differences in the shapes and concentrations of cell groups, tissues, and lesions, and the region to be observed or measured in detail differs depending on the subject's eye. According to the configuration of the present embodiment, after specifying an area of a cell group, a tissue, and a lesion to be observed for each test eye, the acquisition parameters of a plurality of high-magnification images D _Hj are determined according to the shape and density of the area. Can be set automatically.

  In the present embodiment, as the imaging conditions, a position at which a high-magnification image in a wide-angle image is imaged, an imaging order, the number of images to be imaged at the same position, an angle of view and a pixel size of the high-magnification image, an imaging frame Although the number, the frame rate, and the in-focus position have been illustrated, the invention is not limited thereto.

<< Embodiment 2 >>
The ophthalmologic apparatus according to the present embodiment relates to acquiring a plurality of high-magnification images based on image features extracted from an image having a wider angle of view than the high-magnification image when acquiring a plurality of high-magnification adaptive optical SLO images. It is configured to determine a parameter value. Specifically, when acquiring a plurality of high-magnification moving images D _1j in an annular shape with respect to the capillary region in the _parafovea , the acquisition position and angle of view of each high-magnification image D _1j based on this image feature , The pixel size, the number of frames, the frame rate, and the number of times of repeated acquisition are determined. Furthermore, after determining whether the acquired high-magnification image includes an exceptional frame such as fixation disparity or blinking, if it is determined that re-acquisition of the high-magnification image is necessary based on the determination result, the same image is obtained. Re-acquire the high magnification image with the acquisition parameter value.

(overall structure)
FIG. 1B illustrates a configuration of a device connected to the ophthalmologic apparatus 10 according to the present embodiment. This embodiment is different from the configuration of the first embodiment in that a time phase data acquisition device 50 is connected to the ophthalmologic apparatus 10 in addition to the SLO image capturing apparatus 20 and the data server 40. The time phase data acquisition device 50 is a device that acquires biological signal data that changes autonomously and periodically (referred to as “time phase data”), and includes, for example, a pulse wave meter or an electrocardiograph. The time-phase data acquisition device 50 acquires the time-phase data Sj simultaneously with the acquisition of the high-magnification image D _Hj in response to an operation by an operator (not shown). The obtained temporal phase data Sj is transmitted to the ophthalmologic apparatus 10 and the data server 40. In the present embodiment, a high-magnification image is acquired and displayed on the monitor 305 in synchronization with the periodic timing indicated by the time phase data measured from the subject. Therefore, it is possible to acquire or reproduce a high-magnification image at an appropriate timing in accordance with a change in a living body.

In addition, the data server 40 obtains time-phase data other than the acquisition condition data such as the wide-angle image D _L of the subject's eye, the high-magnification image D _H , and the fixation target positions F _L and F _H used at the time of acquisition. Sj also holds normal values relating to the image features of the eye and the distribution of the image features of the eye. In the present embodiment, the retinal blood vessels, the capillaries Q, and the blood cells W are held as the image features of the eye, but the present invention is not limited to this. The temporal phase data Sj output by the temporal phase data acquisition device 50 and the image features of the eye portion output by the ophthalmologic apparatus 10 are stored in the data server 40. Further, in response to a request from the ophthalmologic apparatus 10, the time phase data Sj and normal value data relating to the image feature of the eye and the distribution of the image feature of the eye are transmitted to the ophthalmologic apparatus 10.

(Ophthalmic equipment)
Next, FIG. 10 shows functional blocks of the ophthalmologic apparatus 10 according to the present embodiment. In addition to the configuration of the first embodiment, the ophthalmologic apparatus 10 according to the present embodiment includes a data acquisition unit 110, a temporal phase data acquisition unit 113, an image processing unit 130, an image feature acquisition unit 134, and a determination unit 132, which determines whether reacquisition is necessary. The unit 1325 and the positioning unit 133 include an exceptional frame determination unit 1331. The time phase data acquisition unit 113 is a functional block that acquires the time phase data of the subject. The image feature acquisition unit 134 is a functional block that analyzes a wide-angle image and acquires information indicating the feature. The reacquisition necessity determination unit 1325 is a functional block that determines whether or not to acquire a high-magnification image again. The exceptional frame determination unit 1331 is a functional block that detects, as an “exceptional frame”, a frame that is not suitable for optometry, such as a frame with a large displacement due to fixation failure. When an “exception frame” is detected, a high-magnification image is obtained again.

(Processing procedure)
The image processing flow in the present embodiment is the same as that in FIG. 5, and the processes in S510, S560, S570, and S580 are the same as those in the first embodiment. Also, the step of S540 is omitted. Thus, in the present embodiment, the processing of S520, S530, and S550 will be described.

<Step S520>
The image acquisition pattern presentation unit 1311 acquires at least one type of image acquisition pattern when acquiring a plurality of high-magnification images from the storage unit 120 and displays the acquired image on the monitor 305. In the present embodiment, the image acquisition pattern presentation unit 1311 presents a basic pattern of a linear, cross, radial, rectangular, disk, annular, multi-magnification, multi-arrangement, or composite type.

Next, the instruction obtaining unit 140 externally obtains an instruction on which image obtaining pattern to select. In the present embodiment, a case where the observation target is an annular parafovea capillary region as shown in FIG. 7C will be described as an example. In this case, since it is necessary to determine the inner boundary of the annular region based on the avascular region, the operator selects a multi-magnification type image acquisition pattern. Here, in the multi-magnification type image acquisition pattern, D _1j is annular and D _2k is rectangular.

  In the present embodiment, the image acquisition pattern selection process is not an indispensable process. Even if the high magnification image acquisition target region is a circular region and the intermediate magnification image acquisition target region is a rectangular region from the beginning, the process of this step may be omitted. Good.

<Step S530>
Determination unit 132 performs an acquisition request for the intermediate magnification image D _2k the high magnification image acquisition unit 112, the high magnification image acquisition unit 112 acquires the intermediate magnification image D _2k. Next, the image feature acquisition unit 134 acquires image features on the wide-angle image D _L and the intermediate magnification image D _2k , determines acquisition parameters for a plurality of high-magnification images based on the image characteristics, and Obtain D _1j . In addition, the obtained high-magnification image D _{1j is} subjected to inter-frame alignment and exceptional frame determination, and when it is determined that re-imaging is necessary based on the exceptional frame determination result, the same high-magnification image D _1j is captured again. . Furthermore, to align the intermediate magnification image D _2k and the high magnification image D _1j on wide field of view image D _L. The process of this step (high-magnification image acquisition process) will be described later in detail with reference to the flowchart shown in FIG.

<Step S550>
The display control unit 131, based on the alignment parameter value obtained in S 1270 (described later), superimposed and displayed high magnification image D _H on wide field of view image D _L as shown in FIG. 7 (e). Further, in the present embodiment, a display of a capillary image as shown in FIG. 7F is displayed next to the superimposed image as an image in which the distribution of the capillaries in the parafovea can be observed in more detail. As described above, in the present embodiment, display control is performed to display a plurality of captured high-magnification images on the monitor 305 so as to be superimposed on an image showing the entire imaging region. It is possible to observe a precise image. The process of this step (image display process) will be described later in detail with reference to the flowchart shown in FIG.

(High magnification image acquisition processing)
Next, the details of the processing executed in S530 will be described with reference to the flowchart shown in FIG.

<Step S1210>
The determination unit 132 acquires the intermediate magnification image D _2k based on the D _2k image acquisition pattern selected in S520.

In the present embodiment, a rectangular image acquisition pattern such as D _{2k in} FIG. 12A is set, and the fixation target is presented such that the center point C of the image acquisition pattern is near the fovea. . Further, the values of the angle of view, the pixel size, the number of frames, and the frame rate are each 600 [μm] × 600 [μm], and the pixel size is 2 [μm / pixel] × 2 [μm / pixel], 256 sheets, 64 [frame] / sec] is set. The overlapping area between the adjacent intermediate ratio images is set to 10% with respect to the area of the intermediate magnification image. Acquisition order of the high-magnification images, in this embodiment, the order of moving the intermediate magnification image D ₂₅ in the center of the image acquisition pattern after movement to the next right as a first acquisition position, the adjacent image counterclockwise I do. Positioning unit 133 performs inter-frame alignment of the obtained intermediate magnification image D _2k, performs positioning to the intermediate magnification of the image D _2k wide angle image on D _L (image pasting). Note that the coordinate transformation method and the similarity evaluation function used for the alignment are the same as those in the first embodiment, and thus detailed description is omitted.

<Step S1220>
The image feature acquisition unit 134 detects a capillary from the wide-angle image D _L or the intermediate magnification image D _2k acquired in S1210, and detects a boundary of the avascular region from the detected capillary region. Further, an annular (donut-shaped) region equidistant from the boundary position of the avascular region is detected in order to set a region near the avascular region as an acquisition target region of a high-magnification image.

In the present embodiment, first, a capillary vessel is specified from the intermediate magnification image _D2k as the moving range of the blood cell component in the following procedure.
(A) A difference process is performed between adjacent frames of the intermediate magnification image D _2k whose frames have been aligned. That is, a difference moving image is generated.
(B) At each xy position of the difference moving image generated in (a), a luminance statistic (for example, variance) in the frame direction is calculated.
(C) At each xy position of the differential moving image, a region where the luminance variance is equal to or greater than a predetermined threshold Tv is specified as a region where blood cells have moved, that is, a capillary region.
Note that the capillary blood vessel detection processing is not limited to this method, and any known method may be used. For example, a blood vessel may be detected by applying a filter that emphasizes a linear structure to a specific frame of the wide-angle image D _L or the intermediate magnification image D _2k .

  Next, the image feature acquisition unit 134 detects a boundary of the avascular region from the obtained capillary region. Near the fovea of the retina, there is an area where no retinal blood vessels exist (referred to as an “avascular area”), such as inside the inside broken line area in FIG. 7C. The shape of the border of the avascular region greatly varies from individual to individual, and an initial lesion of a retinal blood vessel is likely to occur around the avascular region boundary. Therefore, the avascular region boundary is important as a target for observation and analysis.

In the present embodiment, in S1210, a variable shape model (solid line in FIG. 12A) having a smaller radius Tr than a circle (dashed line in FIG. 12A) connecting the image centers of the high magnification images D _1j in the image acquisition pattern is widened. The intermediate magnification image D _2k aligned on the corner image is arranged on the combined image. In this embodiment, arranged such that the center of the model coincides with the center C of the intermediate magnification image D ₂₅ of FIG. 12 (a). The position (Bi in FIG. 12B) of the deformable model that has been deformed in accordance with the image feature on the combined image of the intermediate magnification image D _2k is defined as the avascular region boundary, and the center of gravity C of the avascular region boundary 'To decide. Further, predetermined threshold values To and To / are set outward from the avascular region boundary using a distance image (an image having a distance value from the boundary as a pixel value) obtained by performing Euclidean distance conversion on the avascular region boundary. 2 (Bo and Bm in FIG. 12B) are determined. Although any value can be set as the threshold value To, it is generally set to about 150 [μm] for a healthy person in general, so this value is also used in the present embodiment. An annular (doughnut-shaped) high-magnification imaging target area is determined using the two specified inner boundary Bi and outer boundary Bo, and a broken line portion Bm is a candidate for the acquisition position (image center) of the high-magnification image D _1j. Become.

  In the present embodiment, the distance from the avascular region boundary (the thickness of the annular region) is fixed at the threshold value To, but the present invention is not limited to this. For example, in a disease such as diabetic retinopathy in which a lesion occurs in the retinal capillaries in the parafovea, as the disease progresses, the capillaries are blocked and the avascular region becomes larger. Further, when the avascular region becomes large, there is a possibility that a vascular lesion has occurred in a wider area around the avascular region. Therefore, a value obtained by multiplying the threshold value To by a value proportional to the area of the avascular region may be set as the distance from the avascular region boundary. In this case, the angle of view is set as a variable parameter in S1230, and the angle of view is determined to be a sufficiently large value compared to the distance from the avascular region boundary, that is, the thickness of the annular region.

<Step S1230>
The magnification determining unit 1321 determines the number of magnifications, the angle of view, and the pixel size of the high-magnification image D _1j . Further, the position determining unit 1322 determines the acquisition position and the in-focus position of each high magnification image D _1j . In this embodiment, the magnification number, the angle of view, the pixel size, and the in-focus position are fixed parameters (2, 200 [μm] × 200 [μm], 1 [μm / pixel] × 1 [μm / pixel], retinal blood vessels, respectively. ), And the acquisition position of the high-magnification image D _1j is determined as a variable parameter as follows.

First, a point (Bms in FIG. 12B) obtained by sampling the boundary Bm determined at S1220 at equal intervals Td is set as a candidate for the acquisition position of the high-magnification image D _1j , and from a specific candidate point Bm0 to the high-magnification image The acquisition position of the image D _1j is determined in order.
In the present embodiment, the interval Td = the angle of view of the high-magnification image D _1j × (100−standard value of the ratio of the overlapping area between the high-magnification images D _1j ) / 100 (1)
And the candidate point immediately above the center of gravity of the avascular region is Bm0. The detailed acquisition position of the high-magnification image D _1j is determined so as to satisfy both the following conditions a) and b). That is,
a) In the radial direction of the annular region determined in S1220 (the line direction connecting the center of gravity position C 'of the avascular region and the acquisition position candidate point Bms), a high magnification that protrudes outside the annular region under the condition that no blank space occurs in the annular region The sum of the pixels of the image D _1j is minimized.
b) The tangential direction of the boundary position Bm matches the ratio of the overlapping area between the high-magnification images described below.
Here, the ratio [%] of the overlapping region between the high-magnification images D _1j is the circularity Cr of the avascular region boundary specified in S1220.
Cr = 4πS / (L * L) (2)
(S is the area of the avascular region, L is the perimeter)
, A value obtained by multiplying a standard set value (in the present embodiment, for example, 20%) by a value inversely proportional to the circularity Cr is set as a ratio of the overlapping area between the high-magnification images D _1j. . Therefore, the lower the circularity, that is, the larger the unevenness, the larger the ratio of the overlapping area between the high-magnification images D _1j is set.

  The method of determining the overlapping area between the high-magnification images is not limited to this, and any known method may be used. For example, from the intersection Bis of the avascular region and a line connecting the center of gravity C ′ of the avascular region boundary and the acquisition position candidate point Bms, the curvature of the curvature is within a certain distance in the direction of the adjacent high-magnification image along the avascular region boundary. The absolute value is calculated, and the average value Ch of the obtained absolute value of the curvature is calculated. A standard setting value relating to the ratio of the overlapping area between the high-magnification images is weighted by multiplying by a value proportional to the average curvature value Ch. However, when the average curvature value Ch is 0, the standard set value is used without weighting. By using such a setting method, it is possible to set a large ratio of the overlapping area between the high-magnification images in the vicinity of the position where the absolute value of the curvature of the border of the avascular region is large.

<Step S1240>
The time determining unit 1323 determines the number of frames and the frame rate of the high-magnification image D _1j , and the number of times of repeated acquisition. In the example of this embodiment, the number of frames, the frame rate, and the number of times of repetitive acquisition at the same acquisition position are 256, 64 [frame / sec], and twice, respectively. However, the present invention is not limited to this, and any setting method may be used. For example, a thinning process is performed on the capillary region in each of the high-magnification images _D1j determined in the processes of S1220 and S1230, and a blood vessel diameter is calculated in a direction orthogonal to the central axis of the obtained blood vessel. Then, only when there is an area where the blood vessel diameter indicates an abnormal value, the number of frames and the number of times of repeated acquisition of the high-magnification image D _1j may be increased by the thresholds Tf and Tc, respectively.

<Step S1250>
The order determining unit 1324 determines the order of acquiring the high-magnification images D _Hj . As in the case of S850 in the first embodiment, of the following i) to iii), i) is the innermost (highest priority), ii) is the second innermost, and iii) is the outermost (lowest priority). Is repeated so as to form a loop. That is, the following procedure is executed after setting the acquisition start position to the ear side and the acquisition magnification to the lowest magnification.
i) The images at the same acquisition magnification and the same image acquisition position are repeatedly acquired by the number of times of acquisition.
ii) Images at the same acquisition magnification are acquired from adjacent image acquisition positions (counterclockwise in this embodiment) in the same manner as in i).
iii) After ii) is completed, the value of the acquisition magnification is increased, the operation of ii) is performed again, and the same operation is repeated for the number of magnifications.
Note that the method of determining the order of the high-magnification images is not limited to the above procedure, and any order setting method may be used.

<Step S1260>
The high-magnification image acquisition unit 112 acquires the high-magnification image and the time-phase data according to the high-magnification image acquisition parameters determined in S1210 to S1250. The time phase data acquisition unit 113 requests the time phase data acquisition device 50 to acquire the time phase data Sj related to the biological signal. In the present embodiment, a pulse wave meter is used as a time phase data acquisition device, and pulse wave data Sj is acquired from the earlobe (earlobe) of the subject. Here, the pulse wave data Sj is expressed as a periodic point sequence having the acquisition time on one axis and the pulse wave signal value measured by the pulse wave meter on the other axis. Since the time phase data acquisition device 50 acquires and transmits the corresponding time phase data Sj in response to the acquisition request, the time phase data acquisition unit 113 converts the pulse wave data Sj from the time phase data acquisition device 50 via the LAN 30. Receive. The time phase data acquisition unit 113 stores the received time phase data Sj in the storage unit 120.

The data acquisition unit 110 requests the SLO image capturing device 20 to acquire a wide-angle image D _L , a plurality of high-magnification images D _Hj captured at different fixation target positions Fj, and fixation target position Fj data. . Here, the data acquisition unit 110 starts acquisition of the high-magnification image D _Hj in accordance with a certain phase of the time-phase data Sj acquired by the time-phase data acquisition device 50, and a pulse immediately after the acquisition request of the high-magnification image D _Hj. It may be possible to start the acquisition of the wave data Sj and the high-magnification image D _Hj at the same time. After the present embodiment acquisition request of the high magnification image D _Hj, it starts acquiring the pulse data Sj and the high magnification image D _Hj immediately.

<Step S1270>
Positioning unit 133 performs inter-frame alignment on High magnification image D _1j obtained is displayed on the monitor 305 by the high magnification image D _1j aligned on wide field of view image D _L. Note that, in the present embodiment, at the time of frame-to-frame alignment of each high-magnification moving image D _1j, an exception frame determination is performed to determine whether or not the frame corresponds to an exception frame described below. Any known positioning method may be used as a method of aligning frames in each moving image or a method of aligning (image bonding) between images of different magnifications. In this embodiment, both of the affine transformation and the phase relationship are used. Align using numbers.

Here, the exceptional frame is a frame Es having a large displacement due to poor fixation, a low luminance frame Eb due to blinking, and a low frame due to aberration correction failure in each frame of the high-magnification moving image _DH as shown in FIG. Refers to an image quality frame (not shown). The exception frame can be determined based on whether or not the degree of the luminance abnormality, the magnitude of the distortion, the magnitude of the noise with respect to the signal, the displacement amount with respect to the reference frame, and the like are equal to or more than a certain value. In particular,
a) When the translation among the alignment parameter values between the frames is equal to or greater than the threshold value b) When the average luminance value of each frame is less than the threshold value c) When the S / N ratio of each frame is less than the threshold value, it is determined that the frame is an exceptional frame . As a result of the exception frame determination, if the maximum value of the occurrence interval of the exception frames in each high-magnification moving image D _1j is equal to or smaller than the threshold Te, or if the total number of the exception frames is equal to or greater than the threshold Ts, the re-acquisition necessity determination unit 1325 is performed. It is determined that the high magnification image D _1j needs to be reacquired. Reacquisition necessity determining unit 1325, the high magnification image D to _1j requests to reacquire the high magnification image D _1j when reacquisition is determined to require the high magnification image acquisition unit 112 of the high magnification image acquisition The unit 112 reacquires the high-magnification image D _1j according to this request.

It should be noted that the inter-frame alignment and the exceptional frame determination, the necessity of reacquisition determination, and the reacquisition are not necessarily performed after all the high-magnification images are acquired. As soon as it is determined that re-acquisition is necessary, it may be immediately acquired again. Alternatively, at the time of inter-frame alignment of the intermediate magnification image D _2k in S1210, the exceptional frame determination and the reacquisition necessity determination are executed, and as soon as it is determined that the reacquisition is necessary, the intermediate magnification image D _2k is reacquired. Is also good. Further, the exceptional frame determination is not limited to the inter-frame alignment processing of the SLO moving image. For example, the anterior ocular segment camera is connected to the ophthalmologic apparatus 10 and image processing of the anterior ocular segment camera, for example, low brightness The determination may be made using frame detection, pupil position detection, or the like.

(Image display processing)
Next, details of the processing executed in S550 will be described with reference to the flowchart shown in FIG. Note that since the steps other than S910 and S930 are the same as those in the first embodiment, the processing in S910 and S930 will be described in this embodiment.

<Step S910>
The display control unit 131, based on the alignment parameter value obtained in S 1270, the process of generating a superimposed image of a high magnification image D _H to wide field of view image D _L as shown in FIG. 7 (e) Do. In the present embodiment, since the high-magnification image _DH is displayed as a moving image instead of a still image as described in S930, no representative image is generated. However, in the image after the inter-frame alignment, an area having a pixel value of 0 may be generated at the edge of the image, which may hinder the display. Only pixels having a pixel value greater than 0 are displayed throughout the frame.

Further, in the present embodiment, a capillaries image as shown in FIG. 7 (f) is attached next to the combined video image as an image for enabling a more detailed observation of the distribution of the capillaries in the parafovea. The matching display is also performed. For the capillary image, the binary image is obtained by similarly executing the capillary region identification processing performed on the intermediate magnification image D _2k in step S1220 not only on the intermediate magnification image D _2k but also on the high magnification image D _1j . It is generated and pasted and displayed based on the positioning parameters obtained in S1270. Also, as in the case of pasting a moving image to a capillary blood vessel image, only pixels having a pixel value larger than 0 are displayed through all frames other than the exceptional frame.

<Step S930>
When displaying a plurality of intermediate magnification images D _2k and high magnification images D _1j on the wide-angle image D _L , each intermediate magnification image is based on time-phase data (periodic data based on a biological signal such as a pulse wave). D _2k synchronizes the reproduction timing of the high-magnification image D _1j . Specifically, the display control unit 131 acquires the time-phase data Sj and Sk corresponding to each moving image (that is, each of the high-magnification images D _1j and the intermediate-magnification images D _2k ) from the time-phase data acquisition unit 113. Pulsation cycle is calculated by detecting the extremum of the time phase data. Next, an exceptional frame number sequence in each of the high-magnification image D _1j and the intermediate-magnification image D _2k is obtained, and a continuous frame sequence that does not include the exceptional frame is selected as a display target. Further, when the beat cycle in the selected frame is different between the moving images (high-magnification image D _1j and intermediate-magnification image D _2k ), the process of adjusting the display frame interval between the moving images (referred to as “frame interpolation process”) is performed. Do. Furthermore, while adjusting the reproduction start time of each moving image so that the reproduction timing of the frame corresponding to the extreme value of the time phase data corresponding to each moving image is adjusted, the frames corresponding to an integer number of beat periods are reproduced. To display the combined video.

  Note that the display method of the present invention is not limited to this. If time phase data has not been acquired, this step is omitted, and the display is performed as a moving image without adjusting the playback time. Is also good.

  As described above, when acquiring a plurality of high-magnification adaptive optics SLO images, the ophthalmologic apparatus 10 according to the present embodiment uses a plurality of images based on image features extracted from an image having a wider angle of view than the high-magnification image. Is determined for the high magnification image acquisition. For this reason, it becomes possible to efficiently image a tissue, a cell group, or a lesion candidate having a different distribution depending on the subject's eye in a wider range than the high-magnification image.

  Further, in the present embodiment, at least one blood vessel image is displayed on the monitor 305 from the captured high-magnification image based on the characteristics of the image related to the region where the blood vessel or blood cell has moved. For this reason, it is possible to automatically extract only a portion that requires particularly careful observation from the wide-angle image, and automatically perform precise imaging and display.

<< Embodiment 3 >>
When acquiring a plurality of high-magnification adaptive optical OCT tomographic images, the ophthalmologic apparatus according to the present embodiment uses a plurality of high-magnification adaptive optical OCT tomographic images based on image features extracted from the OCT tomographic images having a wider angle of view than the high-magnification images. It is configured to determine a parameter value related to obtaining a magnification image. Specifically, the operator selects a basic pattern for obtaining a plurality of high-magnification images in a disk shape with respect to the photoreceptor layer near the fovea where the outer layer of the retina is deformed by the serous retinal detachment RD, and the image acquisition parameter is selected. The initial value of. Next, a case in which the acquisition parameters (acquisition position, angle of view, pixel size, coherence gate) of a plurality of high-magnification images are changed based on the image characteristics of the layer shape acquired from the wide-angle OCT tomographic image, and imaging is performed will be described. I do.

(overall structure)
FIG. 1C illustrates a configuration of a device connected to the ophthalmologic apparatus 10 according to the present embodiment. In the present embodiment, the ophthalmologic apparatus 10 is different from the first embodiment in that the ophthalmologic apparatus 10 is connected to a tomographic image capturing apparatus 60 having an adaptive optical system instead of the SLO image capturing apparatus 20. The tomographic image capturing device 60 is a device that captures a tomographic image of the eye. The tomographic imaging apparatus 60 is configured, for example, as a spectral domain type OCT (SD-OCT: Spectral Domain Optical Coherence Tomography). The ophthalmic tomographic imaging apparatus 60 three-dimensionally captures a tomographic image of the subject's eye in response to an operation by an operator (not shown). The captured tomographic image is transmitted to the ophthalmologic apparatus 10.

(Ophthalmic equipment)
Next, FIG. 13 shows functional blocks of the ophthalmologic apparatus 10 according to the present embodiment. The difference from the configuration of the first embodiment is that the image processing unit 130 is provided with an image feature acquisition unit 134 for acquiring features of a wide-angle image. Further, the data server 40 holds normal value data relating to the image feature of the eye and the distribution of the image feature of the eye. Here, a case in which normal value data relating to the retinal layer boundary, its shape, and thickness is held as these data will be described.

(Tomographic imaging device)
Next, the configuration of the tomographic imaging apparatus 60 including the adaptive optics will be described with reference to FIG. In FIG. 14, reference numeral 201 denotes a light source. In this embodiment, an SLD light source having a wavelength of 840 nm is used. The light source 201 only needs to have low coherence, and an SLD light source having a wavelength width of 30 nm or more is suitably used. Further, an ultrashort pulse laser such as a titanium sapphire laser can be used as a light source. Light emitted from the light source 201 is guided to the fiber coupler 520 through the single-mode optical fiber 202. The fiber coupler 520 splits the light into a measurement light path 521 and a reference light path 522. Here, a fiber coupler having a branching ratio of 10:90 is used, and is configured such that 10% of the input light amount reaches the measurement light path 521. The light passing through the measurement light path 521 is irradiated by the collimator 203 as parallel measurement light.

  The configuration after the collimator 203 is the same as that of the SLO image capturing apparatus 20 described in the first embodiment. That is, the light is radiated to the eyeball 211 through the adaptive optical system or the scanning optical system, and the reflected and scattered light from the eyeball 211 follows the same path again, is guided to the optical fiber 521, and reaches the fiber coupler 520. On the other hand, the reference light that has passed through the reference light path 522 is emitted by the collimator 523, reflected by the optical path length variable unit 524, and returns to the fiber coupler 520 again. The measurement light and the reference light that have reached the fiber coupler 520 are multiplexed and guided to the spectroscope 526 through the optical fiber 525. The control unit 217 forms a tomographic image of the eye based on the interference light information split by the spectroscope 526. The control unit 217 controls the optical path length variable unit 524 to obtain an image at a desired depth position.

Note that the tomographic image capturing apparatus 60 operates as a normal tomographic image capturing apparatus by increasing the swing angle of the scanning optical system in the configuration of FIG. 14 and instructing the adaptive optics control unit 216 not to perform aberration correction. A wide-angle tomographic image (wide-angle image D _L ) can be captured. Further, in the present embodiment, the tomographic image capturing apparatus 60 including the adaptive optics is configured as an SD-OCT, but the SD-OCT is not an essential requirement. For example, it may be configured as time domain OCT or SS-OCT (Swept Source Optical Coherence Tomography). In the case of SS-OCT, a light source that generates light of different wavelengths at different times is used, and a spectroscopic element for acquiring spectral information is not required. Further, in SS-OCT, a high-depth image including not only the retina but also the choroid can be acquired.

(Processing procedure)
FIG. 5 shows an image processing flow of the ophthalmologic apparatus 10 according to the present embodiment. The processing contents other than S510, S520, S530, S540, and S550 are the same as the processing of the first embodiment described with reference to FIG. Therefore, in the present embodiment, the processing of S510, S520, S530, S540, and S550 will be described.

<Step 510>
Wide field of view image obtaining unit 111, to the tomographic image capturing apparatus 60, requests the acquisition of the wide field of view image D _L and fixation target position F _L. In the present embodiment, an example in the case of obtaining a wide angle image D _L by setting the fixation target position F _L fovea of the macula portion. The method of setting the imaging position is not limited to this, and may be set to another arbitrary position.

The tomography apparatus 60, the wide field of view wide angle image D _L in response to the acquisition request from the image acquisition unit 111 acquires the fixation target position F _L, and transmits to the wide field of view image obtaining unit 111. Wide field of view image obtaining unit 111 receives the wide angle image D _L and fixation target position F _L through the LAN30 from tomography apparatus 60. Wide field of view image acquisition unit 111 stores the wide-angle image D _L and fixation target position F _L received in the storage unit 120.

<Step S520>
The image acquisition pattern presentation unit 1311 acquires at least one type of basic setting pattern (image acquisition pattern) related to parameters when acquiring a plurality of high-magnification images from the storage unit 120, and displays the acquired pattern on the monitor 305. Although an arbitrary pattern can be set as the image acquisition pattern, in the present embodiment, basic patterns as shown in FIGS. 15A to 15F are presented. 15 (a) is linear, FIG. 15 (b) is cross-shaped, FIG. 15 (c) is radial, FIG. 15 (d) is rectangular, FIG. 15 (e) is disk-shaped, and FIG. f) is annular.

  Next, the instruction obtaining unit 140 externally obtains an instruction on which image obtaining pattern to select. In the present embodiment, a case will be described in which the observation target is a region in which the outer layer of the retina is deformed due to the serous retinal detachment RD as shown in FIG. Select a disk-shaped image acquisition pattern.

As in the case of the first embodiment, a multi-magnification type, a multiple arrangement type, or a composite type image acquisition pattern may be presented even in a three-dimensional tomographic image. For example, in the case of the multi-magnification type and the magnification number is 3, an acquisition pattern of the intermediate magnification image D _3m as shown in FIG. _{15H and} an acquisition pattern of the intermediate magnification image D _2k as shown in FIG. 15 (e), the acquisition pattern of the high magnification image D _1j can be selected. Furthermore, in the case of a multi-location image acquisition pattern, a plurality of image acquisition patterns may be arranged and presented in the depth direction (the z-axis direction in the figure).

<Step S530>
The determination unit 132 adjusts the image acquisition parameters based on the image features acquired by the image feature acquisition unit 134, using the acquisition parameters of the plurality of images of the image acquisition pattern selected in S520 as initial values, to thereby obtain a plurality of high magnifications. Determine image acquisition parameters. The processing of this step (high-magnification image acquisition processing) will be described later in detail with reference to the flowchart shown in FIG.

<Step S540>
The positioning unit 133 performs positioning between the wide-angle image D _L and the high-magnification image D _Hj, and determines the position of the high-magnification image D _Hj on the wide-angle image D _L. First, the positioning unit 133 obtains the fixation target position F _Hj used at the time of capturing the high-magnification image D _Hj from the storage unit 120, and _adjusts the position of the wide-field-of-view image D _L and the high-magnification image D _Hj . This is the initial search point for the alignment parameter. When there is an overlapping area between the high-magnification images D _Hj , first, the similarity between images is calculated for the overlapping area, and the positions of the high-magnification images D _Hj are adjusted to the position where the similarity between the images is maximum. Next, when high magnification images having different resolutions have been acquired in S530, alignment is performed in order from the lower magnification image, as in the first embodiment. In the present embodiment, since the resolution of the high-magnification image is one, only the alignment between the wide-angle image D _L and the high-magnification image D _H is performed.

  Note that any known method can be used as the inter-image similarity or coordinate conversion method. In the present embodiment, a three-dimensional correlation coefficient is used as the inter-image similarity, and a three-dimensional Affine transform is used as the coordinate conversion method. To align.

<Step S550>
The display control unit 131 displays the high-magnification image D _Hj on the wide-angle image D _L based on the value of the alignment parameter obtained in S540. Since the wide angle image D _L and the high magnification image D _Hj in this embodiment are both three-dimensional tomographic image, performs two types of display below.
i) A projection image of the wide-angle image D _L and the high-magnification image D _Hj is generated in the z-axis direction, and the projection image of the high-magnification image D _H is superimposed on the projection image of the wide-angle image D _L.
ii) In the position acquired only wide angle of view three-dimensional tomogram D _L in the pixel values of the wide field of view image 3-dimensional tomographic image D _L, wide angle of view three-dimensional tomogram D _L and the high magnification three-dimensional tomogram D _Hj At the positions obtained together, a wide-field-of-view three-dimensional tomographic image _DL "represented by the pixel values of the high-magnification three-dimensional tomographic image _DHj is generated. Further, a specific scan on the wide-field-of-view three-dimensional tomographic image _DL " The position is indicated by an arrow on the superimposed image of i), and the two-dimensional tomographic image of the wide-angle 3D tomographic image D _L "cut out at the position of the arrow is displayed side by side with the superimposed image of i). in this display, not only the 2-dimensional tomographic image of the wide field of view three-dimensional tomogram D _L, 2-dimensional tomographic image of high magnification three-dimensional tomogram D _Hj are also superimposed.
In the display of ii), the operator can move (up and down or left and right) the arrow indicating the display position of the wide-angle tomographic image D _L "through the instruction acquisition unit 140. The operation is cut out in conjunction with this operation. The displayed slices of the wide-angle image D _L and the high-magnification image D _{Hj that} are displayed on the screen also change.

When a plurality of high-magnification images D _Hj having different acquisition positions are acquired as in the present embodiment, the luminance characteristics between the high-magnification images D _Hj are determined using the same method as in the first embodiment. Make similar adjustments. Further, when the imaging positions of the high-magnification images D _Hj are close to each other and overlap (including the case where the imaging positions are the same), the display method of the overlapping area is set to one of the following. That is, the image quality index value of the image is calculated, and the image having the highest evaluation value is displayed, or the brightness of each high-magnification image D _Hj is blended by weighting the transparency based on the image quality index value described above. . Here, any known index can be used as the image quality index value, and in this embodiment, the average luminance value of the image histogram is used.
The method of generating the projection image is not limited to the mean value projection, and any projection method may be used. The high magnification image D _Hj is not limited to a still image, but may be a moving image.

(High magnification image acquisition processing)
Next, the details of the processing (high-magnification image acquisition processing) executed in S530 will be described with reference to the flowchart shown in FIG. Note that step S1510 is the same as step S810 in the first embodiment, and a description thereof will not be repeated.

<Step S1520>
The image feature acquisition unit 134 uses the wide angle-of-view image D _L stored in the storage unit 120, that is, the three-dimensional tomographic image of the eye as image features, as inner boundary membrane B1, nerve fiber layer boundary B2, and inner plexiform layer boundary. Each boundary position of B4, photoreceptor inner / outer segment boundary B5, and retinal pigment epithelium boundary B6 is extracted. FIGS. 7A, 15I and 15J schematically show boundary positions B1 to B6. Then, the extracted image features are stored in the storage unit 120.

Here, a feature extraction procedure for the wide angle-of-view image D _L will be specifically described. First, an extraction procedure for extracting a layer boundary will be described. Here, the three-dimensional tomographic image to be processed is considered as a set of two-dimensional tomographic images (B-scan images), and the following processing is performed on each two-dimensional tomographic image. First, a smoothing process is performed on a two-dimensional tomographic image of interest to remove noise components. Next, edge components are detected from the two-dimensional tomographic image, and some line segments are extracted as layer boundary candidates based on the connectivity. Then, the top line segment is extracted from the extracted candidates as the inner boundary membrane B1, the second line segment from the top as the nerve fiber layer boundary B2, and the third line segment as the inner plexiform layer boundary B4. In addition, a line segment having the maximum contrast on the outer layer side (the side where the z coordinate is larger in FIG. 7A) than the inner limiting membrane B1 is extracted as a photoreceptor inner / outer segment boundary B5. Further, the lowest line segment of the group of layer boundary candidates is extracted as the retinal pigment epithelium boundary B6.

  Note that a variable shape model such as Snakes or the level set method may be applied using these line segments as initial values, and a more precise extraction may be performed. Further, a configuration may be adopted in which the boundary between layers is extracted by a graph cut method. The boundary extraction using the variable shape model or the graph cut may be performed three-dimensionally for a three-dimensional tomographic image, or may be performed two-dimensionally for each two-dimensional tomographic image. In addition, it goes without saying that any method may be used as the method for extracting the layer boundary, as long as the method can extract the layer boundary from the tomographic image of the eye.

<Step S1530>
The magnification determining unit 1321 determines the type of magnification (magnification number, angle of view, and pixel size) of the high-magnification image D _Hj . In the present embodiment, the case where the magnification number and the pixel size are fixed (1, 1 [μm] × 1 [μm] × 1 [μm], respectively) will be described, and detailed description will be omitted. Note that the angle of view and the pixel size are different from those of the first embodiment in that parameters in the z-axis direction are also included. The angle of view is a variable parameter, and the angle of view is set to the threshold Ta [%] only for the high-magnification image in which the distance between the photoreceptor inner / outer segment boundary B5 and the retinal pigment epithelium boundary B6 obtained in S1520 is equal to or larger than the threshold Trd. Enlarge. The reason for increasing the angle of view is to prevent image leakage due to fixation disparity because it is an important area for observation, and that the outer segment of photoreceptor cells above the retinal detachment area has an icicle-like retinal pigment epithelium boundary. This is because the entire photoreceptor cells can be acquired as a high-magnification image in some cases in which the photoreceptor cells extend in the B6 direction.

Next, the position determination unit 1322 determines the acquisition position and the coherence gate position of each high-magnification image D _Hj . In the present embodiment, each parameter is a variable parameter, and the acquisition position of the high-magnification image D _Hj is determined by the following procedure.
a) Determination of arrangement of representative positions of image acquisition patterns.
b) Determining the arrangement of the image acquisition pattern in the xy plane direction.
c) Determining the arrangement of the image acquisition pattern in the z-axis direction.

Here, in a), the representative position of the image acquisition pattern is set as the center of the image acquisition pattern, and the center is arranged so as to coincide with the position of the center of gravity on the retinal detachment area. The retinal detachment region refers to a region in which the distance between the boundary B5 between the photoreceptor inner and outer segments and the boundary B6 of the retinal pigment epithelium is equal to or larger than the threshold Trd and projected on the xy plane.
Regarding b), the arrangement of the high-magnification image in the xy directions is determined by the following procedure so that the retinal detachment area is included in the area of the image acquisition pattern. That is, a circle connecting the image centers of the high-magnification images at the outermost periphery is obtained, the circle is enlarged to a position where the circle becomes a circumscribed circle of the retinal detachment region, and the high magnification is set so as to fill the circle region at regular intervals. Determine the position in the xy direction of the image.
Regarding the acquisition position in the z-axis direction of c), it is determined such that the photoreceptor inner / outer segment boundary B5 acquired in S1520 matches the image center of the high-magnification image. Further, the coherence gate of each high-magnification image D _Hj is set at a position closest to the photoreceptor inner / outer segment boundary B5 detected in S1520 among the settable positions.

  FIG. 15I shows an initial acquisition position of an image acquisition pattern in the present embodiment, and FIG. 15J shows an acquisition position determined in this step. However, in both figures, in order to make the acquired pattern easy to understand, the angle of view of the high-magnification image becomes large on the retinal detachment region only at the acquisition position of the center two rows in the image acquisition pattern, and the overlap between the high-magnification images Is omitted. The types of the variable parameters are not limited to the above, and any image acquisition parameter may be used as the variable parameter.

<Step S1540>
The order determining unit 1324 determines the order of acquiring the high-magnification images D _Hj . In the present embodiment, among the following i) to iv), i) is the innermost (highest priority), ii) is the second innermost, iii) is the third innermost, and iv) is the outermost (lowest priority). Iterative processing is performed in a loop of degree). That is, the acquisition starting position (in this embodiment, the upper end of the image acquisition pattern is set, the acquisition magnification is set to the minimum magnification, and the following steps i) to iv) are executed.
i) The images at the same arrangement pattern, the same acquisition magnification, and the same image acquisition position are acquired by the number of times of repeated acquisition.
ii) Images having the same arrangement pattern and the same acquisition magnification are moved to adjacent image acquisition positions, and then acquired again in the same manner as in i).
iii) After ii) is completed, the value of the acquisition magnification is increased, the operation of ii) is performed again, and the same operation is repeated for the number of magnifications.
iv) When iii) is completed, the operation of iii) is executed in another arrangement, and the operation is repeated until images are acquired in all arrangements.
In the above example, i) is not repeatedly acquired (the number of acquisitions is only one), and the image acquisition pattern is not a multi-arrangement type, so that the processing of iv) is omitted. The image acquisition position adjacent to ii) can be moved in an arbitrary direction. In the present embodiment, the image acquisition position is moved to the adjacent position in the horizontal direction (diagonally downward if not horizontal, and directly downward if not diagonally downward). Let it. That is, among the high-magnification image acquisition positions, the high-magnification images are sequentially acquired in the order of the first stage from right to left, the second stage from left to right, and the third stage from right to left. Note that the order determination method of the present invention is not limited to the above procedure, and any known order setting method may be used.

<Step S1550>
The high-magnification image acquisition unit 112 requests the tomographic imaging apparatus 60 to acquire a plurality of high-magnification images D _Hj and fixation target positions F _Hj using the image acquisition parameters instructed by the determination unit 132. The tomographic image capturing apparatus 60 acquires and transmits the high magnification image D _Hj and the fixation target position F _Hj in response to the acquisition request. The high magnification image D _Hj and the fixation target position F _Hj are received. The high-magnification image acquisition unit 112 stores the received high-magnification image D _Hj and fixation target position F _Hj in the storage unit 120.

Note that, in the present embodiment, the acquisition position of the high-magnification image D _Hj is determined using the image feature related to the photoreceptor cell layer boundary, but the present invention is not limited to this. For example, as in the case of the first embodiment, the determination may be made by the operator operating (moving, enlarging, reducing) the position of the image acquisition pattern of the high-magnification image, and adjusting the acquisition position collectively.

  As described above, when acquiring a plurality of high-magnification adaptive optics OCT tomographic images, the ophthalmologic apparatus 10 is configured based on the image features related to the layer shape extracted from the OCT tomographic images having a wider angle of view than the high-magnification images. A parameter value for acquiring a plurality of high-magnification images is determined. Thereby, it is possible to efficiently image a tissue, a cell group, or a lesion candidate having a different distribution depending on the eye to be examined in a wider range than the high-magnification image.

<< Other embodiments >>
In the above embodiment, the alignment target image is realized as an SLO image or an eye tomographic image, but the present invention is not limited to this. For example, a wide field of view image D _L fundus camera image may be realized a high-magnification image D _H as adaptive optical fundus camera image. It is also possible to achieve wide field of view image D _L wide field of view SLO image, as modalities of different images as projected image of high magnification image D _H compensating optical tomographic image. Further, the ophthalmologic apparatus 10 may be realized as a configuration in which the multifunction peripheral of the adaptive optics SLO image capturing apparatus 20 and the tomographic image capturing apparatus 60 and the ophthalmologic apparatus 10 are directly connected.

  The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, or the like) of the system or apparatus reads the program and reads the program. This is the process to be performed.

10: ophthalmic apparatus, 20: SLO image capturing apparatus, 60: tomographic image capturing apparatus,
130: image processing unit, 134: image feature acquisition unit

The present invention has been made in view of the above problem, and it is possible to generate an image of blood vessel information of an imaging target area obtained by combining blood vessel information of each imaging area acquired based on characteristics of an image regarding an area where blood cells have moved. The aim is to provide technologies that can be used.

In order to achieve the above object, an information processing apparatus according to the present invention has the following configuration. That is,
Each of the plurality of imaging regions obtained by dividing the imaging target region of the subject's eye so that a part thereof overlaps, an image acquisition unit that acquires a plurality of images for each of the imaging regions captured a plurality of times ,
Using a plurality of images of each of the imaging area, and the information obtaining means for obtaining blood vessel information on the blood vessel of each of the imaging area,
Generating means for generating an image of blood vessel information of the imaging target area based on the acquired blood vessel information of each imaging area .

According to the present invention, it is possible to easily generate an image of the blood vessel information of the imaging target area by combining the blood vessel information of each of the imaging areas acquired based on the characteristics of the image regarding the area where the blood cells have moved .

Claims (6)

An information processing apparatus that controls imaging of a plurality of images in an imaging target region of the subject's eye,
Selecting means for selecting one basic pattern from a plurality of basic patterns indicating an arrangement of a plurality of imaging regions for imaging the plurality of images;
Setting means for setting the number of the plurality of imaging regions;
Control means for causing an imaging device to image the plurality of images in the area of the eye to be inspected, according to the number of imaging areas set by the setting means and the arrangement of the imaging areas of the selected basic pattern. Information processing device.   The information processing apparatus according to claim 1, wherein the setting unit sets the number of the plurality of imaging regions by setting an entire size of the selected basic pattern.   The information processing apparatus according to claim 2, wherein the setting of the size is performed by changing the size according to an operation of an operator.   The information processing apparatus according to claim 1, wherein the information processing apparatus has an overlapping area between adjacent images of the plurality of images. A control method of an information processing apparatus that controls imaging of a plurality of images in an imaging target region of an eye to be inspected,
A selection step of selecting one basic pattern from a plurality of basic patterns indicating an arrangement of a plurality of imaging regions for imaging the plurality of images;
A setting step of setting the number of the plurality of imaging regions;
A control step of causing an imaging device to capture the plurality of images in the region of the subject's eye according to the number of imaging regions set in the setting step and the arrangement of the imaging regions of the selected basic pattern. Control method for an information processing device to perform.   A computer program for realizing each means of the information processing apparatus according to claim 1 by a computer.