JP6405092B2 - Image generating apparatus and image generating method - Google Patents

Description

  The present invention relates to an image generation apparatus and an image generation method, and more particularly to an image generation apparatus and an image generation method used for ophthalmic medical care and the like.

  Examination of the fundus is widely performed for the purpose of early diagnosis of lifestyle diseases and diseases that occupy the top causes of blindness. For the examination of the fundus, for example, a scanning laser opthalmoscope (SLO) which is an ophthalmologic apparatus using the principle of a confocal laser microscope is used. The SLO is a device that performs a raster scan with respect to the fundus of the laser as measurement light and obtains a planar image with high resolution and high speed from the intensity of the return light.

  2. Description of the Related Art In recent years, compensation optical SLO (AO-SLO: adaptive) has a compensation optical system that measures aberration of a subject's eye in real time with a wavefront sensor and corrects aberration of measurement light generated in the subject's eye and its return light with a wavefront correction device. Optics SLO) has been developed. As a result, it is possible to acquire a planar image with high lateral resolution (hereinafter also referred to as an AO-SLO image).

  An apparatus for obtaining a fundus image scans measurement light two-dimensionally using a resonance scanner or a galvano scanner, and performs measurement while scanning the measurement light in both the forward direction and the backward direction of the scanner in order to increase the imaging speed. . The control circuit of the scanner converts the detection signal from the photodetector into an image and displays it. For this purpose, the control circuit generates a synchronization signal for image generation in synchronization with the scanning of the scanner. A fundus image is formed by combining the synchronization signal indicating the scanner scanning position and the electrical sampling position by light detection.

  Here, the scanner drive waveform and the image generation synchronization signal are often output at the same timing. However, in practice, there are variations in the frequency and amplitude of the resonant scanner, and a time delay in the electric circuit system, so that the optical scanning position obtained from the synchronization signal may not always match the electrical sampling position. Furthermore, since the adaptive optics SLO has a high resolution, it is strongly affected by the error of the scanner position, and a positional deviation is conspicuous between images sampled based on the synchronization signal, resulting in image quality degradation such as image distortion. There was a possibility.

  In order to solve such a problem, in the invention disclosed in Patent Document 1, an accurate position scanned by the scanner is acquired using dedicated hardware, or the scanner is controlled using the hardware. doing. Patent Document 2 discloses a technique called a pixel clock. As this method, control is performed so as to acquire only a signal at a position corresponding to a pixel of an image, but this method also requires a hardware configuration for accurately detecting the scanner position.

Japanese Patent Laid-Open No. 2000-039560 JP 2012-255978 A

  The solution to add the special hardware configuration described above can increase the cost of creating the device. In addition, the size of the apparatus is affected, the design is limited, and the number of parts increases, which also affects the man-hours during assembly.

  In view of the above problems, the present invention provides an image generation apparatus and an image generation method that can acquire an image in which image distortion due to the characteristics of a scanner is corrected without using a special hardware configuration.

To solve the above Symbol issues, engaging Ru image generating device according to an embodiment of the present invention,
Photoelectric conversion means for receiving return light from measurement light that is reciprocally scanned on the fundus of the eye to be examined by a scanner and converting the return light into an electrical signal;
An extraction unit that extracts a reference signal for specifying a scanning position of the scanner while the scanner performs one reciprocating scanning;
Image generating means for generating image data of the fundus by reciprocating scanning of the measurement light from the electrical signal;
Means for setting the reference signal as a sampling reference position when sampling the return light at the time of the reciprocating scanning, and setting the scanning amplitude of the scanner for the reciprocating scanning at the time of generating the fundus image data as a reference amplitude;
Correction value input means for inputting respective correction values of the sampling reference position and the reference amplitude;
And a second correction means for correcting the first correction means and the reference amplitude correcting the sampled reference position based on the respective correction value input by the correction value inputting means,
The image generation means regenerates the fundus image data based on the corrected sampling reference position and the reference amplitude.

  According to the present invention, it is possible to obtain an image in which image distortion due to the characteristics of the resonant scanner is corrected.

It is a figure which shows the function structure of the image generation apparatus which concerns on Example 1 of this invention. 2 is a schematic diagram illustrating a configuration of an adaptive optics SLO according to Example 1. FIG. 3 is a flowchart illustrating a processing procedure of the image generation apparatus according to the first embodiment. 4 is a flowchart for explaining in detail an operation at the time of scanner position estimation in the processing procedure shown in FIG. 3. 4 is a flowchart for explaining in detail an operation at the time of scanner amplitude estimation in the processing procedure shown in FIG. 3. It is a schematic diagram which shows the relationship between the trigger signal of a galvano scanner and a resonance scanner, and a reflected signal. It is a figure explaining sine correction. It is a schematic diagram which shows the influence on the image of the shift | offset | difference of an image start point. It is a figure explaining the search method of the resonance scanner reference | standard amplitude of FIG. It is a figure which shows the function structure of the image generation apparatus which concerns on Example 2 of this invention. 10 is a flowchart illustrating a processing procedure of the image generation apparatus according to the second embodiment. FIG. 10 is a diagram illustrating an example of a UI screen of a utility application according to the second embodiment.

(Example 1)
In the present embodiment, a process for reconstructing an image from signal values acquired by a combination of a resonance scanner and a galvano scanner, which is performed when an image obtained by capturing the retina as a subject with the adaptive optics SLO, is described. Specifically, a trigger signal group of a galvano scanner corresponding to one image in the horizontal direction and a trigger signal group of a resonant scanner corresponding to two reciprocating rows in the image vertical direction are acquired. An image is generated using the scanner position estimated from the trigger signal transmission timing as the sampling reference position. Then, the sampling reference position that minimizes the amount of deviation between each reciprocating image in the image is calculated, and the deviation of the sampling reference position due to the frequency variation of the resonant scanner and the time delay of the electric circuit system is corrected.

  Next, the image is further reconstructed based on the obtained sampling reference position, and the sampling reference amplitude that minimizes the amount of deviation between two adjacent images in the image is calculated, and the variation in the amplitude of the resonant scanner is calculated. to correct. Then, from the calculated sampling reference position and amplitude, the actual scanner position when the resonance scanner trigger signal is transmitted and the amplitude at each scanning of the scanner are obtained. By reconstructing an image using parameters corrected in this way, it is possible to acquire a retina image corrected for distortion by the resonant scanner at high speed without adding a special hardware function. .

<Configuration of Image Generation Device>
FIG. 1 shows a functional configuration of an image generation apparatus 10 according to the present embodiment, which is connected to an adaptive optics SLO apparatus whose schematic configuration is illustrated in FIG.

  In FIG. 1, the signal acquisition unit 100 acquires the trigger signals of the resonance scanner and the galvano scanner and the reflection signal from the retina from the adaptive optics SLO device. Here, the trigger signal is a signal indicating that the resonant scanner and the galvano scanner are at a specific position (reference position). The acquired information is stored in the storage unit 130 through the control unit 120. The image generation unit 140 includes a trigger extraction unit 141, a scanner position estimation unit 142, a scanner amplitude estimation unit 143, and an image reconstruction unit 144. The image generation unit 140 according to the present exemplary embodiment corresponds to an image generation unit, and generates image data by reciprocating scanning of measurement light from an electrical signal obtained from the optical sensor 211.

  The trigger extraction unit 141 extracts trigger signals of the galvano scanner and the resonance scanner from the acquired signal, and the image generation unit 140 images the reflected signal using the extracted trigger signal. The trigger extracting unit 141 in this embodiment corresponds to an extracting unit that extracts a reference signal for specifying the scanning position of the scanner while the scanner performs one reciprocal scanning. The scanner position estimation unit 142 uses the generated image to calculate an image shift amount between the image obtained in the forward pass during scanning and the image obtained in the return pass. The scanner amplitude estimation unit 143 calculates the scanner reference amplitude, and obtains the deviation from the actual position of the resonant scanner and the actually scanned amplitude when the trigger signal is transmitted for each reciprocating scan. Then, based on the obtained scanner position and amplitude, the image reconstruction unit 144 reconstructs an image of the retina. The output unit 150 outputs an image generated through the above processing to a monitor (not shown) or the like.

  Next, the configuration of the adaptive optics SLO used in this embodiment will be described with reference to FIG. The adaptive optics SLO includes an SLD 201, a Shack-Hartmann wavefront sensor 206, a compensation optical system 204, a beam splitter (202, 203), an XY scanning mirror 205, a focus lens 209, an aperture 210, and an optical sensor 211. It is connected to the image forming apparatus 10 of FIG.

  Light emitted from an SLD (Super Luminescent Diode) 201 that is a light source is reflected from the fundus. A part of the reflected return light is input to the Shack-Hartmann wavefront sensor 206 via the second beam splitter 203, and the other part is input to the optical sensor 211 via the first beam splitter 202. The Shack-Hartmann wavefront sensor 206 is a device for measuring eye aberration, and a CCD 208 is connected to a lens array 207.

  When incident light passes through the lens array 207, a bright spot group appears on the CCD 208, and wavefront aberration is measured based on the positional deviation of the projected bright spot. Based on the wavefront aberration measured by the Shack-Hartmann wavefront sensor 206, the compensation optical system 204 drives an aberration correction device (a deformable mirror or a spatial light phase modulator) to correct the aberration. The aberration-corrected light is received by the optical sensor 211 via the focus lens 209 and the diaphragm 210. The optical sensor 211 receives, as photoelectric conversion means, return light from measurement light scanned on the fundus of the subject's eye and converts it into an electrical signal. The obtained electrical signal is transmitted to the signal acquisition unit 100. As described above, this adaptive optical system corrects aberrations in the measurement light and return light generated in the eye to be examined.

  The scanning position of the measurement light on the fundus can be controlled by moving the X / Y scanning mirror 205 which is a scanner. Under the control of the scanner, data on the imaging target region and time (number of frames / frame rate) of the subject specified in advance by the operator is acquired. The X / Y scanning mirror 205 uses a resonance scanner in the X direction as the main scanning, and uses a galvano scanner in the Y direction as the sub scanning. The data is transmitted to the image generation apparatus 10 and used when forming image data (moving image or still image).

  In order to focus on a specific depth position on the fundus, adjustment using an aberration correction device in the compensation optical system 204 or a focus adjustment lens (not shown) is installed in the optical system, and the lens is moved. It is possible to use at least one of adjustments made by doing. In addition, when obtaining an image of the fundus, etc., it is necessary to reduce the influence of involuntary eye movements called fixation fixation, eye movements due to poor fixation, or eye movements associated with facial movements. As a countermeasure for this, there is a case where eye movement is tracked (tracked) and reflected in an image. For measurement of fundus movement, a fundus image generated by the image generation device 10 is used to extract a template image, which is a small region image having a feature, by a calculation unit (not shown), and a part most similar to the template image For example, there is a method for measuring the movement of the fundus by pattern matching. In this case, a tracking scanner (not shown) is installed in the optical system, and a control unit (not shown) is installed at a position where the tracking scanner and the calculation unit are connected, so that the position of the irradiation light is tracked. It is possible to follow the movement of the fundus measured by. In the present embodiment, the above configuration corresponds to means for correcting the imaging position by tracking the movement of the eye to be examined.

<Processing procedure of image processing apparatus>
Next, with reference to the flowchart of FIG. 3, a procedure for each process performed in the image generation apparatus 10 of the present embodiment will be described.

<Step S310>
In step S <b> 310, the signal acquisition unit 100 acquires signal information for at least one image acquired from the adaptive optics SLO device connected to the image generation apparatus 10. There are three types of signal information: the trigger signal of the galvano scanner used for photographing the retina, the trigger signal of the resonant scanner, and the reflected signal from the retina obtained by photographing. The acquired signal information is stored in the storage unit 130 through the control unit 120.

  At this time, hardware control information accompanying the acquired signal information is acquired and stored in the storage unit 130 through the control unit 120. Here, the control information is a frame rate corresponding to a sampling frequency or a galvano scanner frequency when acquiring a retina reflection signal. Such control information may be described in a shooting information file added to signal information, or may be included as tag information of signal information.

<Step S320>
In step S320, the trigger extraction unit 141 acquires the trigger signal positions (trigger positions) of the galvano scanner and the resonant scanner from the signal information acquired by the adaptive optics SLO device stored in the storage unit 130. Then, the extracted trigger position is stored in the storage unit 130 through the control unit 120.

  Here, the trigger signal is a signal obtained from the scanner in response to the measurement light irradiation at a predetermined posture, that is, a predetermined angle when the scanner is scanning, or obtained from the scanner at a predetermined time interval. It corresponds to a signal used when estimating the posture of the scanner. The trigger position corresponds to the position where the trigger signal is extracted on the time axis, as will be described later, and corresponds to the orientation or position of the scanner serving as a reference for sampling the reflected signal to obtain the pixel value.

  6A is a schematic diagram of the trigger signal of the galvano scanner, FIG. 6B is a schematic diagram of the trigger signal of the resonant scanner, and FIG. 6C is a schematic diagram of the reflected signal. Since the trigger signals of the galvano and resonant scanner are signals as shown in FIGS. 6A and 6B, respectively, the trigger signal is extracted by threshold processing.

  Specifically, it is detected as a trigger signal when the change in signal intensity is a certain threshold value or more. The galvano scanner scans the laser beam in the horizontal direction of the image. While the next trigger signal is detected from one trigger signal, the horizontal scan is performed once, during which one image is taken. The While one trigger signal is detected, the resonant scanner scans two lines in the vertical direction (outward and backward) of the image, and during this time, two reciprocating lines in the vertical direction of the image in one image. Is filmed. Therefore, the trigger signal detection start timing is extracted as a trigger position, that is, a sampling reference position.

The reflected signal in FIG. 6C is a value obtained by detecting the intensity of the reflected light from the fundus with the optical sensor 211 in FIG. Sampling of the reflected light is executed only while the trigger signal from the resonant scanner is detected as shown in FIG. 6B, and the obtained image data is used for image generation.
Various methods can be considered as the trigger position acquisition method described here, and the method is not limited to the method used in this embodiment.

<Step S330>
In step S330, the scanner position estimation unit 142 generates an image based on the trigger position acquired in step S320, and estimates the scanner position when the trigger signal is transmitted using the generated image. The estimated scanner position is stored in the storage unit 130 through the control unit 120.

  FIG. 4 shows a flowchart for explaining the details of the scanner position estimation. Here, the following scanner position estimation is performed for each reciprocating scan of each resonance scanner of each of a plurality of images taken continuously. This process is repeated for reciprocal scanning in one image, and the same is performed for a plurality of images. Each estimated scanner position estimated in this way is stored in the storage unit 130 through the control unit 120.

  In this embodiment, the resonance scanner position at the time of trigger signal transmission is estimated every two lines of the image of the resonance scanner reciprocal scanning. This is because the adaptive optical SLO device has a high lateral resolution, so that a slight error that the scanner frequency or the time delay of the electric circuit system exerts on the scanner position has an effect of non-uniform image distortion for each reciprocating scan. Because. Another reason is that variations in scanning of the resonance scanner are also affected by temperature in the apparatus main body and instability of the power supply, so that variations in scanner position errors themselves are large.

  Depending on the scanner setting of the adaptive optics SLO device and the captured condition of the acquired image, this image distortion may appear at a certain period. In this case, it is also possible to reduce the number of estimation steps by estimating the scanner position every number of reciprocating scans corresponding to the period in which image distortion appears.

<Step S410>
In step S410, the scanner position estimating unit 142, based on the acquired control information together with signaling information step S310, the resonance scanner position during trigger outgoing, is set as an initial position x ₀ of the design value of the picture start position. Here, there are several methods for setting the initial value. For example, the design value of this apparatus is acquired from the point of consideration of the trigger delay from the trigger position. In addition, when several images are continuously captured, the estimated position is searched by setting an average scanner estimated position for each round-trip scanning of the image captured before the captured image as an initial position. A method of reducing the number of steps is also used.

<Step S420>
In step S420, the scanner position estimating unit 142, based on the image start position x ₀ set in step S410, imaging is performed one reciprocation image 2 line of sine correction of the resonant scanner. Since the resonant scanner does not scan within the scanning range at the same speed, if data is evenly divided into positions in the vertical axis direction, the image is distorted due to the speed corresponding to the scanning position of the resonant scanner. For this reason, the image distortion is eliminated by approximating the scanning trajectory of the resonant scanner to a sine wave and weighting the sampling number for each pixel.

  FIG. 7 shows a schematic diagram of sine correction. In the graph, the horizontal axis indicates time t, the vertical axis indicates position x, the resonance scanner scanning trajectory is assumed to be a sine wave having the frequency acquired in step S310, and its amplitude is 1.0. . The vertical axis of this sine wave is divided into 10 equal parts, and the divided sections are named as time 0.0 to n = 0, 1, 2,. In addition, the times of intersections between the straight line obtained by dividing the vertical axis and the sine wave are named t (0), t (1),... T (9), respectively. Data sampled while the resonance scanner is scanning the section obtained by dividing the vertical axis is assigned to the pixel at this position. Then, sampling is performed at 15 MHz at regular intervals in time, but it can be seen that one section of the corresponding resonant scanner is scanned, that is, the sampling time differs depending on the pixel. Therefore, the luminance value of the reflection signal acquired is associated with the pixel value at the corresponding position, and when a plurality of signals correspond to one pixel, the pixel value is acquired as an average value of the plurality of signals.

<Step S430>
In step S430, the scanner position estimation unit 142 calculates an index D based on the pixel value of each pixel in the line image generated in step S420. The purpose of calculating the index D is to quantitatively evaluate the deterioration in image quality caused by the shift of the image start position. That is, the image quality of the image data is evaluated. The index D may be calculated by, for example, calculating the luminance value difference between pixels adjacent to the left and right of two adjacent image lines for one reciprocation of the resonance scanner using the following equation.

Here, the luminance of the pixel in the j-th row of the adjacent line in the line 2i-1, 2i column (i = 1, 2, 3,...) Is expressed as I, which is the pixel of all the two lines of the image. The value is being calculated.

Here, the j rows used for this calculation may be performed by designating pixels having adjacent values of the two lines or a certain range. In addition, there is a method of calculating with the following formula when weighting pixels in a high luminance region. In this method, only a region with high luminance may be calculated.

  In addition, there is a method of evaluation using the sum of absolute values of luminance differences, normalized cross-correlation, or the like. These evaluation indices are considered to differ in the optimum method depending on what kind of image shift is considered, but for images with many structures with large luminance differences such as photoreceptor cells and blood vessels in the focused fundus image. In this case, it is considered that the above two equations may be used.

  FIG. 8 shows a schematic diagram of a photographed image of a fundus photoreceptor. As shown in FIG. 8A, when the sampling reference position of the image is set correctly, there is almost no shift amount between pixels adjacent in the horizontal direction. On the other hand, as shown in FIG. 8B, when a deviation occurs in the image sampling reference position, a deviation amount between pixels adjacent in the horizontal direction increases. Comparing these two indices D, the index D becomes a smaller value because the brightness difference between adjacent pixels is smaller in FIG. 8A. That is, it can be seen that the smaller the index D, the smaller the shift amount between pixels.

<Step S440>
In step S440, the scanner position estimation unit 142 calculates an index D with respect to the sampling reference position shifted by ± Δx from the sampling reference position set in step S410. Specifically, a line image is created using sampling reference positions of x ₀ + Δx and x ₀ −Δx, and an index D is calculated for each. The magnitude of Δx depends on the resolution of the image and the scan rate, but here it is assumed to be one sampling unit.

<Step S450>
In step S450, the scanner position estimation unit 142 determines whether or not the index D with respect to the sampling reference position of x ₀ + Δx, x ₀ , x ₀ −Δx obtained in step S440 satisfies the following relationship. Here, the value of the index corresponding to the sampling reference position x ₀ is D (x ₀ ).
D (x ₀ ) <D (x ₀ −Δx)
D (x ₀ ) <D (x ₀ + Δx)

If the above relationship is satisfied, the process proceeds to step S470. Otherwise, the process proceeds to step S460. Here, the case where the above relationship is satisfied is a case where the sampling reference position where the index D is the smallest is included between x ₀ −Δx and x ₀ + Δx. In such a case, processing for obtaining a more accurate sampling reference position is performed in step S470. On the other hand, if there is a possibility that the sampling reference position where the index D is the smallest is not included between x ₀ −Δx and x ₀ + Δx, the range for searching for the sampling reference position is changed in step S460. .

<Step S460>
In step S460, the scanner position estimation unit 142 shifts the search range for the sampling reference position where the index D of the line image is the smallest by Δx. Specifically, when D (x ₀ −Δx)> D (x ₀ + Δx), x ₀ + Δx is set as the initial value of the sampling reference position, and the process returns to step S820. If D (x ₀ + Δx)> D (x ₀ −Δx), x ₀ −Δx is set as the initial value of the sampling reference position, and the process returns to step S420.

<Step S470>
In step S <b> 470, the scanner position estimation unit 142 searches the sampling reference position where the index D of the line image is the smallest in the sampling reference with x ₀ or x ₀ −Δx / 2, x ₀ + Δx / 2 as the center. The position is limited to a range of ± Δx / 2. That is, it corresponds to halving the value of Δx. Here, the selection of the sampling reference position as the center selects the sampling reference position where the index D is the smallest.

Then, after performing the process of S420 to S460 again, the process of performing the process of S420 to S460 by repeating the process of S420 to S460 by further halving the value of Δx again in S470, sampling that generates the smallest image It becomes possible to improve the accuracy of the reference position. Specifically, the estimation accuracy of the sampling reference position becomes ± Δx / 2 ^M by repeating the process M times. The number of repetitions is determined according to the accuracy to be obtained, but here, M = 3.

<Step S480>
In step S480, the scanner position estimation unit 142 determines how many times the process in step S470 has been repeated. If the number of repetitions is less than M, Δx = Δx / 2 and the process returns to step S420. When the number of repetitions reaches M, the image start position at that time is acquired. The sampling reference position thus extracted is stored in the storage unit 130 through the control unit 120 as the actual resonance scanner position at the time of trigger signal transmission.

  The operations performed by the scanner position estimation unit 142 in this embodiment are performed by the first evaluation unit, the first correction unit, and the unit that inputs a correction value to the first correction unit, and the scanner position estimation unit The unit 142 has a module area for executing operations in these means. As described above, the first evaluation unit evaluates the image quality of the image based on the image data by comparing the image data obtained by the forward scanning of the scanner with the image data obtained by the backward scanning. The above repetitive operation is executed by the module area in the scanner position estimation unit 142 that has a function of inputting a correction value corresponding to the evaluation result by the first evaluation unit to the first correction unit. In the present embodiment, the first evaluation unit evaluates the image quality of the image data in the forward scanning and the backward scanning in the reciprocating scanning of the scanner using the image data of adjacent pixels. .

  Further, the operation by the first correction means described above estimates the sampling reference position with the least influence of distortion on the image obtained from the image data, and in the estimation, the first correction means uses the electric signal between the reference signals. It is also possible to grasp that the sampling reference value is corrected so as to determine the sampling reference position.

<Step S340>
In step S340, the scanner amplitude estimation unit 143 generates an image based on the corrected sampling reference position acquired in step S330, and estimates the scanner reference amplitude at the time of resonant scanner scanning using the generated image. The scanner amplitude thus estimated is stored in the storage unit 130 through the control unit 120.

  FIG. 5 shows a flowchart for explaining the details of the scanner reference amplitude estimation. Here, the following scanner amplitude estimation is performed on the image data obtained for each of the two reciprocating scans of the resonance scanner for each of a plurality of continuously captured images, similarly to the scanner position estimation. In addition, the process is repeated for each of two reciprocal scans in one image, and is similarly performed for a plurality of images. Each scanner amplitude estimated by the above processing is stored in the storage unit 130 through the control unit 120.

  In this embodiment, the resonance scanner reference amplitude is estimated every two reciprocating scans of the resonance scanner. This is because the adaptive optics SLO device has a high lateral resolution, so that a slight error in scanner amplitude also affects non-uniform image distortion for each reciprocal scan. Similarly, the reconstruction of the obtained image is also affected by the temperature in the apparatus main body and the instability of the power supply, and therefore, the variation in the error of the scanner reference amplitude itself is a cause.

  Note that this image distortion may appear every several reciprocating scans depending on the scanner setting of the adaptive optics SLO device and the imaging conditions of the acquired image. In this case, it is possible to reduce the number of estimation steps by estimating the scanner reference amplitude for each reciprocating scan in which image distortion appears.

<Step S510>
In step S510, the scanner amplitude estimation unit 143, based on the acquired control information in step S310, the scanner reference amplitude at resonance scanner scans is set as the initial value y ₀ of the design value of the resonant scanner. In addition, when the search for the scanner reference amplitude has already been completed for one round-trip image that is one prior to the image for two round-trips of the target resonant scanner, and the value has been determined, The calculated amplitude of the reciprocating image is set as an initial value.

  In addition, the setting of the reference amplitude is combined with the setting of the sampling reference position described above, the reference signal is set as the sampling reference position for sampling the return light during the reciprocating scan, and the reciprocating scan scanner for generating the image data is used. It is executed by a module area in the image generation unit 140 that functions as means for setting the scanning amplitude as a reference amplitude.

<Step S520>
In step S520, the scanner amplitude estimation unit 143, based on the sampled reference position for each acquired scanner one reciprocation image scanner reference amplitude y ₀ and step S330 which is set in step S510, image 4 of the two round-trip of the resonance scanner The line is imaged by performing sine correction as in step S420.

<Step S530>
In step S530, the scanner amplitude estimation unit 143 calculates the index E based on the pixel value of the line image generated in step S520. The purpose of calculating the index E is to quantitatively evaluate image quality degradation caused by amplitude deviation. In addition, as a method for calculating the index E, for example, among the two reciprocating resonance scanners, pixels adjacent to the left and right of the adjacent two image lines with respect to the first round-trip return image sequence and the second round-trip outgoing image sequence For example, the brightness value difference between them is calculated by the following equation.

Here, the luminance of the pixel in the j-th row of the adjacent line of the lines 2i, 2i + 1 column (i = 1, 2, 3,...) Is represented by I, and this represents the pixel values of all the two lines of the image. I'm calculating. The j rows used for this calculation may be performed by designating pixels having adjacent values of the above two lines or a certain range.

  In addition, there is a method of evaluation using the sum of absolute values of luminance differences, normalized cross-correlation, or the like. These evaluation indices are considered to differ in the optimum method depending on what kind of image shift is considered, but for images with many structures with large luminance differences such as photoreceptor cells and blood vessels in the focused fundus image. In this case, it is considered that the above equation may be used.

<Step S540>
In step S540, the scanner amplitude estimation unit 143 calculates an index E with respect to the scanner reference amplitude that is shifted by ± Δy only in the second round-trip image from the scanner reference amplitude set in step S510. Specifically, a second round-trip line image is created using scanner reference amplitudes y ₀ + Δy and y ₀ −Δy, and an index E is calculated for each. FIG. 9 is a diagram for explaining a scanner reference amplitude search method. FIG. 9 shows a curve indicating drive scanning for two reciprocations of the resonant scanner, and the initial amplitude y ₀ corresponds to the amplitude of the first reciprocating scanner scanning. In addition, the areas corresponding to the forward and backward image sequences of the first and second round trips are indicated by four square areas.

  Here, as described in step S530, two lines of the return image sequence for the first round trip and the forward pass image sequence for the second round trip are used to calculate the index E. It can be seen that these two lines are connected in the upper part of the figure, that is, the upper ends of the respective amplitudes are always in the same position. Therefore, assuming that the lower end of the amplitude of the second round-trip is at a position shifted by ± Δy, the second round-trip is re-imaged with the respective amplitude. Then, the index E is calculated using the re-imaged second round trip image. The size of Δy depends on the resolution of the image and the scan rate, but here is the unit of length per pixel to be displayed.

<Step S550>
In step S550, the scanner amplitude estimation unit 143 determines whether or not the index E with respect to the second reference scanner amplitude of y ₀ + Δy, y ₀ , y ₀ −Δy obtained in step S540 satisfies the following relationship. judge. Here, the value of the index corresponding to the scanner reference amplitude y ₀ is E (y ₀ ).
E (y ₀ ) <E (y ₀ −Δy)
E (y ₀ ) <E (y ₀ + Δy)

If the above relationship is satisfied, the process proceeds to step S570. Otherwise, the process proceeds to step S560. Here, the case where the above relationship is satisfied is a case where the second reference scanner reference amplitude with the smallest index E is included between y ₀ −Δy and y ₀ + Δy. In such a case, a process for obtaining a scanner reference amplitude with higher accuracy is performed in step S570. On the other hand, if there is a possibility that the second round-trip scanner reference amplitude at which the index E becomes the smallest is not included between y ₀ −Δy and y ₀ + Δy, the second round-trip scanner reference amplitude is determined in step S560. Change the search range.

<Step S560>
In step S560, the scanner amplitude estimation unit 143 shifts the range for searching for the second reference scanner reference amplitude that minimizes the index E of the line image by Δy. Specifically, if E (y ₀ −Δy)> E (y ₀ + Δy), y ₀ + Δy is set as the initial value of the second reference scanner reference amplitude, and the process returns to step S520. When E (y ₀ + Δy)> E (y ₀ −Δy), y ₀ −Δy is set as the initial value of the second reference scanner reference amplitude, and the process returns to step S520.

<Step S570>
In step S <b> 570, the scanner amplitude estimation unit 143 searches for a scanner reference amplitude in which the index E of the line image is the smallest in the scanner reference centered at y ₀ or y ₀ −Δy / 2, y ₀ + Δy / 2. The amplitude is limited to a range of ± Δy / 2. That is, it corresponds to halving the value of Δy. Here, the selection of the central scanner reference amplitude selects the scanner reference amplitude with the smallest index E.

Then, after the processing of S520 to S560 is performed again, the process of performing the processing of S520 to S560 by halving the value of Δy again in S570 and repeating the processing of S520 to S560, thereby generating a scanner with the smallest index E. It becomes possible to improve the accuracy of the reference amplitude. Specifically, the estimation accuracy of the scanner reference amplitude becomes ± Δy / 2 ^M by M repetition processes. The number of repetitions is determined according to the accuracy to be obtained, but here, M = 3.

<Step S580>
In step S580, the scanner amplitude estimation unit 143 determines how many times the process of step S570 has been repeated. If the number of repetitions is less than M, Δy = Δy / 2 and the process returns to step S520. If the number of repetitions reaches M, the scanner reference amplitude at that time is acquired. The scanner reference amplitude extracted in this manner is stored in the storage unit 130 through the control unit 120 as the actual resonance scanner amplitude at the time of resonance scanner scanning.

  The operations performed by the scanner amplitude estimation unit 143 in this embodiment are performed by the second evaluation unit, the second correction unit, and a unit that inputs a correction value to the second correction unit, and the scanner amplitude estimation The unit 143 has a module area for executing processing in each of these means. As described above, the second evaluation unit compares the images obtained by the reciprocal scanning of the continuous scanner from the image data regenerated by the image generation unit based on the sampling reference position corrected by the first correction unit. Evaluate the image quality of the image data. The above repetitive operation functions as a means for inputting a correction value for correcting the reference amplitude for the scanning amplitude of the scanner when sampling the return light to the second correction means based on the evaluation result of the second evaluation means. This is executed by the module area in the scanner amplitude estimation unit 143.

  It is executed by inputting a correction value corresponding to the evaluation result by the second evaluation means to the second correction means. Further, in the present embodiment, the second evaluation means is the image data obtained by the two reciprocating scans in the reciprocating scan of the scanner, and the image data in the second reciprocating scan and the image data in the second reciprocating scan. The image quality is evaluated using image data of adjacent pixels.

  Further, the first evaluation means, the means for inputting the correction value to the first correction means, the second evaluation means, and the means for inputting the correction value to the second correction means described in the present embodiment are the same as those in this embodiment. A correction value input means is configured. That is, the correction value input means inputs at least one correction value of the sampling reference position and the reference amplitude to the corresponding correction means. The first correction unit and the second correction unit correct the sampling reference position and the reference amplitude, respectively, based on the correction value input by the correction value input unit.

  Further, the operation of the second correction means described above estimates the reference amplitude that has the least influence of distortion on the image obtained from the image data, and at the time of the estimation, the second correction means converts the electric signal between the reference signals. On the other hand, it is possible to grasp that the reference amplitude is corrected so as to determine the reference amplitude.

<Step S350>
In step S350, the image reconstruction unit 144 performs sin correction based on the sampling reference position for each reciprocating image acquired in step S330 and the scanner reference amplitude for each reciprocating image acquired in step S340. Perform reconfiguration. That is, the image generation means regenerates image data based on the corrected sampling reference position and reference amplitude, and executes image reconstruction. At that time, the image reconstruction unit 144 performs approximate correction using the sin function for the scanning of the resonant scanner. Then, the retina image by the reconstructed adaptive optics SLO device is stored in the storage unit 130 through the control unit 120.

  Here, the image reconstruction is performed for each of a plurality of images taken continuously. That is, the above processing is performed for one image corresponding to each trigger signal of the galvano scanner acquired in step S320, and this is repeated for the number of images. Then, after image reconstruction is performed on all images, an image group in which these images are integrated is configured and stored in the storage unit 130.

<Step S360>
In step S360, the output unit 150 displays the retina reconstructed image stored in the storage unit 130 in step S350 on a monitor or the like through the output unit 150. Further, the sampling reference position and the scanner reference amplitude for each scanner reciprocating scan stored in the storage unit 130 in steps S310 to S350 are stored in the database.

  With the above configuration, when acquiring an image of the retina by the adaptive optics SLO device, it is possible to generate an image in which distortion due to the resonant scanner is corrected without having a special hardware configuration for specifying the scanner position. become.

  In the first embodiment, after calculating the sampling reference position, the scanner reference amplitude is calculated and an image is generated. However, when the optimum sampling reference position differs depending on the change in amplitude, the amplitude is calculated again, and then the sampling reference position is calculated. May be calculated. Alternatively, although the calculation is complicated, it is also possible to calculate optimum values by simultaneously shaking two parameters of the sampling reference position and the scanner reference amplitude. Also, depending on the image, if the image distortion due to either correction is small, it is possible to calculate both parameters for the first image only and to perform only one of the calculation processing for the subsequent images in order to reduce the amount of calculation. is there.

  Further, the image quality evaluation by the first evaluation unit and the second evaluation unit described above uses high-luminance pixels and neighboring pixels among adjacent pixels for the image data of the two pixel columns to be evaluated. ing. However, the image quality evaluation may be performed by comparing the image data of the two pixel columns to be evaluated for each pixel. Alternatively, the image quality is evaluated by comparing the luminance in the image data of the two pixel columns to be evaluated, and the image quality is evaluated by the sum of the quotient of the square of the luminance difference and the square of the luminance sum in the image data of the two pixel columns to be evaluated. Evaluation of image quality by absolute value of luminance difference in image data of two pixel columns to be evaluated, evaluation of image quality by sum of squares of luminance difference in image data of two pixel columns to be evaluated, two pixel columns to be evaluated It is also possible to perform at least one of image quality evaluation based on the normalized cross correlation.

(Example 2)
In the first embodiment, the processing for automatically reconstructing an image by searching and calculating the sampling reference position and the scanner reference amplitude is shown.
In the second embodiment, the sampling reference position and the scanner reference position (hereinafter referred to as scanner correction parameters) are changed by the user while viewing the image.

  FIG. 10 shows a functional configuration of the image generation apparatus 10 according to the present embodiment. However, the functional configurations of 100, 120, 130, and 150 are the same as those in FIG. In this embodiment, the image generation unit 140 includes only the trigger extraction unit 141 and the image reconstruction unit 144, and only the sampling reference position acquired by the trigger extraction unit 141 in Example 1 and the control information stored in the storage unit 130 are included. Using the generated image, the scanner correction parameter acquisition unit 1010 reconstructs the image using the scanner correction parameter input by the user.

  In the present embodiment, the correction value input means in the first embodiment has a scanner correction parameter acquisition unit that acquires a correction parameter for a sampling reference position and a reference amplitude by an input from a user, and the first correction means and the second correction means described above. The correction means corrects the sampling reference position and the reference amplitude based on the correction parameter.

  Hereinafter, the processing procedure of the image generation processing apparatus 10 according to the present embodiment will be described with reference to the flowchart of FIG. Here, steps S310, S320, and S340 are not different from the processing procedure described in the first embodiment, and thus the description thereof is omitted.

<Step S1150>
In step S1150, the output unit 150 displays the retina reconstructed image stored in the storage unit in step S340 and the scanner correction parameter stored in the storage unit 130 on a monitor or the like through the output unit 150. As an example of this display, FIG. 12 shows an example of a UI screen configuration of a utility application for adjusting the scanner correction parameter displayed on the monitor screen. This application has a main screen 1210, a scanner correction parameter information display 1220, a retina reconstructed image display 1230, a scanner correction parameter change command button 1240, and the like.

<Step S1160>
In step S1160, the scanner correction parameter acquisition unit 1010 changes the scanner correction parameter that the user performs on the scanner correction parameter 1220 displayed on the main screen 1210 of the application and the generated retina reconstructed image 1230 in step S1150. To get. The change method includes a case where the same change amount is given to all the images of each one-way two-line image, and a case where the scanner correction parameter is changed for each one-way two-line image. In the example of FIG. 12, the change is executed in accordance with the user input using the command button 1240 for changing the scanner correction parameter. If there is no image change, the process ends. If there is a change, the process proceeds to step S1170.

<Step S1170>
In step S <b> 1170, the scanner correction parameter acquisition unit 1010 acquires scanner correction parameters input by the user and stores them in the storage unit 130 through the control unit 120. Thereafter, the process returns to step S340, and image reconstruction is performed again based on the newly acquired scanner correction parameter.

  With the above configuration, when generating an image of the retina acquired by the adaptive optics SLO device, the user can adjust the scanner correction parameters of the resonant scanner while viewing the image, and can select the optimum scanner correction parameters. Become.

(Other embodiments)
Using the method introduced in the first embodiment, the image distortion is regenerated using the scanner correction parameter calculated from the generated image, and then the scanner correction parameter is input by the user input using the method introduced in the second embodiment. By selecting and correcting the image, it is possible to perform appropriate distortion correction even for an image in which the image distortion is not sufficiently corrected by the evaluation method of the first embodiment or an image inappropriately generated due to erroneous detection. Can do.

  Another object of the present invention is to supply a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. Needless to say, this can also be achieved by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium.

DESCRIPTION OF SYMBOLS 10 Image generation apparatus 100 Signal acquisition part 120 Control part 130 Memory | storage part 140 Image generation part 141 Trigger extraction part 142 Scanner position estimation part 143 Scanner amplitude estimation part 144 Image reconstruction part 150 Output part

Claims (20)

Photoelectric conversion means for receiving the return light by the measurement light a fundus of the examinee eye reciprocally scanned into an electric signal the return light by the scanner,
An extraction unit that extracts a reference signal for specifying a scanning position of the scanner while the scanner performs one reciprocating scanning;
Image generating means for generating image data of the fundus by reciprocating scanning of the measurement light from the electrical signal;
Means for setting the reference signal as a sampling reference position when sampling the return light at the time of the reciprocating scanning, and setting the scanning amplitude of the scanner for the reciprocating scanning at the time of generating the fundus image data as a reference amplitude;
Correction value input means for inputting respective correction values of the sampling reference position and the reference amplitude;
And a second correction means for correcting the first correction means and the reference amplitude correcting the sampled reference position based on the respective correction value input by the correction value inputting means,
The image generation unit regenerates the fundus image data based on the corrected sampling reference position and the reference amplitude. The correction value input means includes
First evaluation means for evaluating the image quality of the fundus image data by comparing the fundus forward image data obtained by the scanner forward scan and the fundus return image data obtained by the backward scan;
Means for inputting a correction value according to the evaluation result of the first evaluation means to the first correction means;
From the fundus image data by the image generating means is re-generated based on the corrected sampled reference position by the first correction means, the fundus regenerated by comparing the images with each other by sequential scanning one reciprocation scanning A second evaluation means for evaluating the image quality of the image data;
And a means for inputting a correction value for correcting a reference amplitude of the scanning amplitude of the scanner when sampling the return light to the second correction means based on the evaluation result of the second evaluation means. The image generation apparatus according to claim 1. The correction value input means includes a scanner correction parameter acquisition unit that acquires the correction parameter of the sampling reference position and the reference amplitude by an input from a user,
The image generation apparatus according to claim 1, wherein the first correction unit and the second correction unit correct the sampling reference position and the reference amplitude based on the correction parameter. Said first evaluation means, for the return image data that by the forward image data and the backward scan that by the forward scan in the reciprocating scanning, the image quality by using the image data of the fundus of the respective adjacent pixels The image generation apparatus according to claim 2, wherein evaluation is performed. The second evaluation means includes, among the image data of the fundus oculi obtained by the two reciprocating scans in the reciprocating scan, the return image data in the first reciprocating scan and the forward image data in the second reciprocating scan. 5. The image generation apparatus according to claim 2, wherein the image quality is evaluated using image data of the fundus of each adjacent pixel. The first evaluation unit and the second evaluation unit evaluate the image quality of the fundus image data of the two pixel columns to be evaluated using a pixel having high luminance and neighboring pixels among adjacent pixels. The image generation apparatus according to claim 2, wherein the image generation apparatus performs the image generation apparatus. The first evaluation unit and the second evaluation unit evaluate the image quality by comparing the fundus image data of two pixel columns to be evaluated for each pixel. The image generation apparatus according to any one of 4 and 5. The first evaluation unit and the second evaluation unit evaluate the image quality by comparing the luminance in the fundus image data of the two pixel columns to be evaluated, and the luminance difference in the fundus image data of the two pixel columns to be evaluated. square and evaluation of the image quality due to the total sum of the square of the quotient of the intensity sum, evaluation of the image quality due to the absolute value of the luminance difference in the fundus image data of two pixel columns to evaluate, the two pixel columns to evaluate claims, characterized in that performing the square evaluation of the image quality due to the total sum of the evaluation of the image quality due to a normalized cross-correlation between two pixel rows to be evaluated for at least one of the luminance difference in the image data of the fundus The image generation device according to any one of 2 , 4 , and 5. The first correction means estimates the sampling reference position with the least influence of distortion on the image obtained from the fundus image data,
In the estimation, the first correction means, according to claim 2, and claim and corrects the sampled reference position to determine the sampling reference position with respect to the electrical signal between said reference signal The image generation device according to any one of 4 to 8. The second correction means estimates the reference amplitude with the least influence of distortion on an image obtained from the fundus image data,
In the estimation, the second correction means, according to claim 2, characterized in that correcting the reference amplitude so as to determine the reference amplitude with respect to an electric signal between said reference signal, and claims 4 to 9 The image generation device according to any one of the above. The scanner includes a resonant scanner,
11. The image generation apparatus according to claim 1, wherein the image generation unit performs approximate correction using a sin function for scanning of the resonance scanner. Measuring the aberration of the pre-Symbol subject's eye, any one of claims 1 to 11 characterized in that said compensation further Ru comprising an optical system for correcting the aberrations in the measuring beam and the return light generated in the subject's eye The image generation apparatus according to one item. Before Symbol image generating apparatus according to any one of claims 1 to 12, characterized in that further Ru comprising means for correcting the imaging position and tracking the movement of the eye.   A program for causing a computer to perform operations of each component in the image generation apparatus according to any one of claims 1 to 13.   A storage medium storing the program according to claim 14. A photoelectric conversion step of converting the fundus of the examinee eye into an electrical signal and receiving the return light by reciprocating scanning measurement luminous by the scanner,
An extraction step of extracting a reference signal for specifying the scanning position of the scanner while the scanner performs one reciprocating scan;
An image generating step of generating image data of the fundus by reciprocating scanning from the electrical signal;
Setting the reference signal as a sampling reference position when sampling the return light at the time of the reciprocating scanning, and setting the scanning amplitude of the scanner for the reciprocating scanning at the time of generating image data of the fundus as a reference amplitude;
A correction value input step of inputting a respective correction value of the sampled reference position and the reference amplitude,
Correcting the sampling reference position and the reference amplitude based on each correction value input in the correction value input step ,
In the image generation step, the fundus image data is regenerated based on the corrected sampling reference position and the reference amplitude. The correction value input step includes
A first evaluation step for evaluating the image quality of the fundus image data by comparing the fundus forward image data by the forward scan of the scanner and the fundus return image data by the backward scan;
Obtaining a correction value according to the evaluation result obtained by the first evaluation step;
From the fundus image data regenerated based on the sampling reference position corrected by the correction value, the image quality of the image by the fundus image data regenerated by comparing the images by successive reciprocating scanners is determined. A second evaluation step to evaluate;
Claims, characterized in that it comprises a step of obtaining a correction value for correcting the reference amplitude for the basis of the second evaluation evaluation results obtained by the process, the scan amplitude of the scanner at the time of sampling the return light 16. The image generation method according to 16. The correction value input step includes a scanner correction parameter acquisition step of acquiring a correction parameter for the sampling reference position and the reference amplitude by an input from a user,
The image generation method according to claim 16, wherein the step of correcting the sampling reference position and the step of correcting the reference amplitude are performed based on the correction parameter.   A program for causing a computer to execute each step of the image generation method according to any one of claims 16 to 18.   A storage medium storing the program according to claim 19.