JP2018143434A - Ophthalmologic image display apparatus, ophthalmologic image display method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an ophthalmologic image display apparatus, an ophthalmologic image display method, and a program that allow a temporal change of ophthalmologic images acquired from the same patient at different time points to be visually grasped easily.SOLUTION: An ophthalmologic image display apparatus includes image acquisition part for acquiring a plurality of captured images of the same eye to be examined or a plurality of analysis images on the shape of the same eye to be examined; a positioning part for performing positioning of the plurality of acquired images; a moving image creation part for creating a moving image by connecting the plurality of positioned images in a time-series manner; and a display part for displaying the created moving image.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an ophthalmologic image display apparatus, an ophthalmologic image display method, and a program used for diagnosing an eye to be examined.

  In the field of ophthalmology, various images are acquired to observe the state of the eye to be examined. An image acquired in the ophthalmic field in this way is called an ophthalmic image.

  The ophthalmologic image is acquired by various ophthalmologic photographing apparatuses. As an example of an ophthalmologic photographing apparatus, an optical coherence tomography (Optical Coherence Tomography, OCT) is used to obtain a tomographic image, a fundus camera for photographing a fundus, a fundus image by laser scanning using a confocal optical system. There are scanning laser ophthalmoscope (Scanning Laser Ophthalmoscope, SLO) which obtains an image, a slit lamp which obtains an image by cutting out a light section of the cornea using slit light, and the like.

  Further, based on the fundus layer thickness information obtained from the OCT signal, a thickness map representing a two-dimensional distribution of the layer thickness is generated. This thickness map and a thickness change map representing a time-series change in layer thickness are also included in the ophthalmic image.

  By displaying these ophthalmologic images on the display means, the state of the eye to be examined can be observed.

  For example, Patent Document 1 describes a device that simultaneously displays a frontal image and a tomographic image of a fundus acquired at different dates and times on the same eye on a monitor in a follow-up mode for performing follow-up observation. Yes.

JP 2013-27443 A

  However, there is a drawback in that it is difficult to visually recognize temporal changes and to recognize minute changes only by displaying the front images and tomographic images side by side.

  The present invention has been made in view of such circumstances, and an ophthalmic image display device and an ophthalmic image that make it possible to easily visually grasp temporal changes in ophthalmic images obtained at different times of the same patient. An object is to provide a display method and a program.

  In order to achieve the above object, one aspect of an ophthalmologic image display device includes an image acquisition unit that acquires a plurality of captured images related to the same eye or a plurality of analysis images related to the shape of the same eye, and a plurality of acquired images A positioning unit that performs positioning, a moving image creation unit that creates a moving image by joining a plurality of aligned images in time series, and a display unit that displays the created moving image.

  According to this aspect, a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the eye to be examined are acquired, the obtained images are aligned, and the aligned images are time-sequentially acquired. Since a moving image is created by connecting them in order and the created moving image is displayed, it is possible to easily grasp the temporal change of ophthalmic images obtained at different times of the same patient visually.

  The alignment unit includes a feature point extracting unit that extracts a plurality of feature points from a first image that is one of a plurality of images, and a plurality of second images other than the first image among the plurality of images. A corresponding point detecting unit that detects a plurality of corresponding points corresponding to the feature points, and an image deforming unit that deforms the second image to match the positions of the plurality of corresponding points with the positions of the corresponding feature points, respectively. It is preferable to provide. Thereby, the alignment of the acquired several image can be performed appropriately.

  The corresponding point detection unit preferably detects a plurality of corresponding points using template matching. Thereby, a plurality of corresponding points corresponding to a plurality of feature points can be appropriately detected from the second image.

  The image deformation unit preferably matches the positions of the plurality of corresponding points to the positions of the corresponding feature points using affine transformation. Thereby, the alignment of the acquired several image can be performed appropriately.

  The alignment unit includes a rotational movement amount calculation unit that calculates a rotational movement amount between a first image that is one of a plurality of images and a second image other than the first image, and a rotational movement amount calculation unit. An image rotation unit that performs alignment in the rotation direction between the first image and the second image based on the rotational movement amount calculated by the above, and the first image and the second image that have been aligned by the image rotation unit Is subjected to phase-only correlation processing to calculate a parallel movement amount between the first image and the second image, and a parallel movement amount calculated by the parallel movement amount calculation unit. It is preferable to include an image translation unit that performs alignment in the parallel direction between the first image and the second image. Thereby, the alignment of the acquired several image can be performed appropriately.

  Preferably, the rotational movement amount calculation unit calculates the rotational movement amount between the first image and the second image by performing phase-only correlation processing on the first image and the second image. Thereby, the amount of rotational movement between the acquired images can be calculated appropriately.

  The rotational movement amount calculation unit includes a first conversion processing unit that performs a discrete Fourier transform on the second image, a polar coordinate conversion unit that performs a polar coordinate conversion on a calculation result of the first conversion processing unit for the second image, and a second A second conversion processing unit that performs discrete Fourier transform on the calculation result of the polar coordinate conversion unit for the image, first data obtained in advance for the first image, and a calculation result of the second conversion processing unit for the second image. A first phase-only combining unit that performs a phase-only combining process for combining the second data obtained based on the first data; a first inverse conversion processing unit that performs an inverse discrete Fourier transform on the calculation result of the first phase-only combining unit; Preferably, the rotational movement amount is calculated based on the calculation result by the first inverse conversion processing unit. Thereby, the amount of rotational movement between the acquired images can be calculated appropriately.

  It is preferable to provide a brightness changing unit that matches the brightness of each of the plurality of images. Thereby, the temporal change of the ophthalmologic image can be easily grasped visually.

  An interpolation image creation unit that creates an interpolation image that interpolates between two images that are adjacent in time series order among a plurality of images connected in time series order, and the moving image creation unit includes the interpolation image It is preferable to create a moving image. Thereby, the temporal change of the ophthalmologic image obtained at another time of the same patient can be easily grasped visually.

  The captured image is preferably a front image of the fundus or a tomographic image of the fundus. The analysis image is preferably a thickness map representing a two-dimensional distribution of the layer thickness of the fundus or a thickness change map representing a time-series change in the layer thickness. Thereby, the temporal change of the captured image or the analysis image can be easily grasped visually.

  In order to achieve the above object, one aspect of an ophthalmologic image display method includes an image acquisition step of acquiring a plurality of captured images related to the same eye to be examined or a plurality of analysis images relating to the shape of the same eye to be examined; A positioning step for performing positioning, a moving image creation step for creating a moving image by joining a plurality of aligned images in time series, and a display step for displaying the created moving image on a display unit.

  According to this aspect, a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the eye to be examined are acquired, the obtained images are aligned, and the aligned images are time-sequentially acquired. Since a moving image is created by connecting them in order and the created moving image is displayed, it is possible to easily grasp the temporal change of ophthalmic images obtained at different times of the same patient visually.

  In order to achieve the above object, one aspect of the program includes an image acquisition step of acquiring, in a computer, a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the same eye to be examined; A positioning process for performing the positioning, a moving picture creation process for creating a moving picture by connecting a plurality of aligned images in time series, and a display process for displaying the created moving picture on a display unit.

  According to this aspect, a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the eye to be examined are acquired, the obtained images are aligned, and the aligned images are time-sequentially acquired. Since a moving image is created by connecting them in order and the created moving image is displayed, it is possible to easily grasp the temporal change of ophthalmic images obtained at different times of the same patient visually.

  A computer-readable non-transitory recording medium recording the program is also included in this aspect.

  According to the present invention, temporal changes in ophthalmic images obtained at different times of the same patient can be easily grasped visually.

Block diagram showing the configuration of an ophthalmic image display device The flowchart which shows the moving image display process which an ophthalmologic image display apparatus performs The figure which shows an example of a base image Diagram showing an example of an image for creating a movie The figure which shows the order at the time of arranging the photography date of a base image and the image for animation creation, and time series order Block diagram showing the configuration of an ophthalmic image display device The flowchart which shows the moving image display process which an ophthalmologic image display apparatus performs The figure which showed the photography interval of two images which adjoin in time series order Block diagram showing the configuration of an ophthalmic image display device Block diagram showing an example of the alignment unit The flowchart which shows the moving image display process which an ophthalmologic image display apparatus performs

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

<First Embodiment>
[Configuration of ophthalmic image display device]
FIG. 1 is a block diagram illustrating a configuration of an ophthalmologic image display apparatus 10 according to the first embodiment. As shown in FIG. 1, the ophthalmic image display device 10 includes a control unit 12, a communication unit 14, an image memory 16, an alignment unit 18, an image selection unit 25, a brightness change unit 26, a moving image creation unit 28, and an operation unit 30. And the display unit 32 and the like.

  The control unit 12 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory) (not shown). Various programs and control data are stored in the ROM. The RAM temporarily stores programs and data executed by the CPU.

  The control unit 12 functions as a control unit that performs overall control of each unit of the ophthalmologic image display apparatus 10 by executing a program read from the ROM in the CPU, and also functions as a calculation unit that performs various calculation processes.

  The communication unit 14 includes a communication interface (not shown), and transmits / receives data to / from an external server 100 connected to the communication interface. The server 100 constitutes a database for various ophthalmic images and diagnostic data.

  The image memory 16 is a temporary storage unit for various data including image data, and data is read and written through the control unit 12. Image data captured from the server 100 via the communication unit 14 (an example of an image acquisition unit) is stored in the image memory 16.

  The alignment unit 18 is an image processing unit that aligns the positions of a plurality of input images, and includes a feature point extraction unit 20, a corresponding point detection unit 22, and an image deformation unit 24.

  The feature point extraction unit 20 extracts feature points included in the input image and specifies the position of the extracted feature points. Here, the feature point is a portion having a relatively large feature amount in the image, for example, an edge or a vertex. The feature point extraction unit 20 calculates a feature amount of the entire image, and extracts a pixel whose calculated feature amount exceeds a threshold value as a feature point.

  The corresponding point detection unit 22 extracts corresponding points included in the input image, and specifies the position of the extracted corresponding points. The corresponding points are points in the image having the same features as the feature points extracted by the feature point extraction unit 20. The corresponding point detection unit 22 calculates a feature amount of the entire image, and extracts a point having a high similarity between the calculated feature amount and the feature amount of the feature point as a corresponding point.

  The image deformation unit 24 performs parallel movement processing, rotational movement processing, enlargement processing, and reduction processing on the input image, and deforms the image. Here, in particular, the corresponding point detection unit 22 is based on the positional relationship between the feature point of the image from which the feature point is extracted by the feature point extraction unit 20 and the corresponding point of the image from which the corresponding point is detected by the corresponding point detection unit 22. The image in which the corresponding point is detected is deformed by.

  The image selection unit 25 selects a desired ophthalmic image from the ophthalmic images recorded in the server 100.

  The brightness changing unit 26 changes the brightness of the input image.

  The moving image creation unit 28 combines a plurality of images whose positions are aligned by the alignment unit 18 in time series, and generates a moving image that displays each image in turn.

  The operation unit 30 includes operation members such as operation buttons and a keyboard, and outputs operation information input from the operation members to the control unit 12. The control unit 12 executes various processes according to the operation information input from the operation unit 30.

  The display unit 32 displays various images including the moving image created by the moving image creating unit 28, diagnostic information about the patient, and the like. As the display unit 32, for example, a liquid crystal display, an organic EL display, or the like is applicable. A touch panel in which the operation unit 30 and the display unit 32 are integrated may be used.

[Video display method]
FIG. 2 is a flowchart showing a moving image display process (an example of an ophthalmic image display method) executed by the ophthalmic image display apparatus 10 according to the present embodiment.

  In the present embodiment, a plurality of ophthalmologic images of the same eye to be examined, which are the same eye of the same patient, are created to display a moving image, and the temporal change of the ophthalmic image is visually displayed to the observer. Make it easier to grasp.

  The ophthalmologic image includes a captured image obtained by photographing the eye to be examined with various ophthalmologic photographing devices, and an analysis image relating to the shape of the eye to be examined created from the analysis result. The captured image includes a front image of the fundus, a two-dimensional tomographic image of the fundus, a three-dimensional tomographic image, and a fluorescent image that captures fluorescence excited by a fluorescent substance. Further, the analysis image includes a thickness map representing a two-dimensional distribution of the layer thickness of the fundus and a thickness change map representing a time-series change in the layer thickness.

  Here, it is assumed that all ophthalmic images are recorded in the server 100.

  First, the control unit 12 initializes a variable n to 0 (step S1).

Next, the observer selects a base image P ₀ (an example of a first image) relating to the eye to be examined for which a moving image is to be created (step S2, an example of an image acquisition process).

In selecting the base image P ₀ , the control unit 12 displays a list of ophthalmic images recorded in the server 100 on the display unit 32, and the observer selects a desired image from the list display using the operation unit 30. Alternatively, the ophthalmologic image associated with the name inputted in the control unit 12 may be automatically selected by the observer inputting the name of the patient.

Accordingly, the selected image is acquired in ophthalmic image display device 10 via the communication unit 14, it is stored in the image memory 16 as the base image P _0. The base image P ₀ is not limited in the photographing time or acquisition time, and the observer can select any image as the base image P ₀ .

Subsequently, the feature point extraction unit 20 extracts a plurality of feature points from the base image P ₀ (step S3).

FIG. 3 is a diagram illustrating an example of the base image P ₀ stored in the image memory 16. Here, an example in which a fundus photograph taken by a fundus camera is selected as the base image P ₀ is shown.

As shown in FIG. 3, the feature point extracting unit 20 extracts feature points A ₁ including the optic papilla and feature points A ₂ and A ₃ including the branching portion of the blood vessel from the base image P ₀ . The plurality of feature points may be determined by designating arbitrary points by the observer.

The extraction of feature points from the base image P ₀ is completed, the control unit 12 increments the variable n (step S4).

Next, the image selection unit 25 selects a moving image creation image P _n (an example of a second image other than the first image) for creating a moving image (step S5, an example of an image acquisition process).

The image selection unit 25 selects an ophthalmic image of the same eye as the base image P _{0 from} among the ophthalmic images recorded in the server 100 and the same type of ophthalmic image as the base image P ₀ . As a result, the selected image is acquired by the ophthalmic image display device 10 via the communication unit 14 and stored in the image memory 16 as the moving image creation image _Pn .

  Note that the same kind of ophthalmic image refers to a photographed image photographed by the same type of device or an analysis image represented by the same expression technique based on the same index. The same type of device need not be the same device up to the manufacturer and model number, as long as the imaging principle is the same.

Figure 4 is a diagram showing an example of a moving image creating images P ₁ to create a video. Video creating image P ₁ is a fundus photographs of the same eye to be examined based image P _0.

Subsequently, the corresponding point detection unit 22 detects a plurality of feature points from the moving image creation image _Pn (step S6). Corresponding points can be detected by using, for example, template matching or phase only correlation method.

Here, as shown in FIG. 4, corresponding points B _1, the corresponding point as the corresponding point corresponding to the feature point A ₂ B _2, and the corresponding points corresponding to the characteristic point A ₃ as the corresponding point corresponding to the feature point A ₁ detects a corresponding point B ₃ are as.

Then, the image deforming unit 24, based on the image P ₀ by modifying the image P _n for movie created as a reference, the alignment of the base image P ₀ and the moving creating image P _n performs (step S7, positioning step Example).

Here, the coordinate position of each corresponding point B _n of video creating image P _n to deform the image P _n for moving created to match the coordinate position of each feature point A _n of the base image P _0. For example, when the position of the corresponding point B _n is moving parallel to the X direction, Y direction or XY directions relative to the position of each feature point A _n, the image deformation unit 24, the image P _n for creating video Is translated. Further, when the position of the corresponding point B _n is rotating moves the XY plane relative to the position of each feature point A _n, the image deformation unit 24 rotates moving the image P _n for creating video.

Further, if the positional relationship between the corresponding points B _n is shrinking relative positional relationship of each feature point A _n, the image deformation unit 24 expands the image P _n for creating video. Further, if the positional relationship between the corresponding points B _n is enlarged relative positional relationship of each feature point A _n, the image deformation unit 24 reduces the image P _n for creating video.

For these image deformations, the movement vector may be determined so that the sum of the distances between the feature points _An and the corresponding points _Bn is minimized. Note that these parallel movement processing, rotational movement processing, enlargement processing, and reduction processing can be performed by affine transformation.

The video creating image P _n which is modified so becomes the match image position relative to the base image P _0.

Subsequently, the brightness change unit 26, the brightness of the moving image creating image P _n to change the brightness of the moving image creating image P _n to match the brightness of the base image P ₀ (step S8).

Here, the brightness correction is performed by matching the average luminance value of each moving image creation image P _{n with} the average luminance value of the base image P ₀ . Further, the dynamic range of the moving image creating images P _n, may be modified to match the dynamic range of the base image P _0.

Note that, although change the brightness of the moving image creating images P _n in accordance with the brightness of the base image P ₀ is, if it is possible to match the base image P ₀ and the brightness of the moving image creating images P _n, The observer can arbitrarily determine the reference brightness.

Next, the control unit 12 determines whether or not the image selection unit 25 has finished selecting the movie creation image _Pn for creating a movie (step S9).

The server 100, the base image P ₀ and an ophthalmologic image of the same eye to be examined, and in the base image P ₀ and the same type of ophthalmologic image, if there is an ophthalmologic image not yet been selected, the process proceeds to step S4, Similar processing is repeated. That is, n is incremented (step S4), the moving image creation image _Pn is selected (step S5), and the same processing is performed (steps S6 to S9).

Note that the moving image creation image _Pn may be manually selected by the observer using the operation unit 30.

When the selection of the image is completed, the moving image creating unit 28 sorts the base image P ₀ and the moving image creating images P _{1 to} P _n in chronological order (step S10).

  In the header portion of the ophthalmic image stored in the server 100, the shooting date / time is recorded if it is a captured image, and the shooting date / time of the captured image that is the basis of analysis is recorded if it is an analysis image.

FIG. 5 is a diagram illustrating the shooting dates of the base image P ₀ and the moving image creation images P _{1 to} P ₅ and the order when arranged in time series. Here, they are arranged in chronological order from the oldest.

As shown in FIG. 5, the shooting dates of the base image P ₀ and the moving image creation images P _{1 to} P ₅ are June 22, 2013 for the base image P ₀ and July 2014 for the moving image creation image P ₁ , respectively. 1 day, moving creating image P ₂ December 15, 2013, the moving image creating image P ₃ January 12, 2016, the moving image creating image P ₄ July 21, 2016, and moving creating image P ₅ is a January 31, 2015. Accordingly, when the images are arranged in order from the oldest shooting date, the base image P ₀ , the movie creation image P ₂ , the movie creation image P ₁ , the movie creation image P ₅ , the movie creation image P ₃ , and the movie creation image. made in the order of P _4.

Subsequently, the moving picture creating unit 28 creates a time-lapse moving picture by combining the base image P ₀ and the moving picture creating images P _{1 to} P _n sorted in time series in time series (step S11, moving picture creating step). One case).

  Finally, the control unit 12 displays the moving image created by the moving image creating unit 28 on the display unit 32 (step S12, an example of a display process). The control unit 12 may record this moving image on the server 100.

  In this way, by using a moving image that displays a plurality of ophthalmologic images successively in chronological order, the observer can easily grasp the temporal change of the same eye to be examined. Although an example in which the front image of the fundus is converted into a moving image has been described here, the same effect can be obtained by converting the fundus 2D tomographic image, 3D tomographic image, fluorescence image, thickness map, thickness change map, and the like into a moving image. Can be obtained.

<Second Embodiment>
[Configuration of ophthalmic image display device]
FIG. 6 is a block diagram illustrating a configuration of an ophthalmologic image display apparatus 40 according to the second embodiment. In addition, the same code | symbol is attached | subjected to the part which is common in the block diagram shown in FIG. 1, and the detailed description is abbreviate | omitted.

  As shown in FIG. 6, the ophthalmic image display device 40 includes an interpolation image creation unit 34.

  The interpolated image creation unit 34 creates an interpolated image that interpolates between the two input images in time series. Here, the interpolation image creation unit 34 detects a motion vector from the two images by block matching processing, and creates an interpolation image based on the detected motion vector.

[Video creation method]
FIG. 7 is a flowchart showing a moving image display process executed by the ophthalmologic image display apparatus 40 according to the present embodiment. Parts common to the flowchart shown in FIG. 2 are denoted by the same reference numerals, and detailed description thereof is omitted.

  In the present embodiment, the temporal change of the ophthalmologic image is more easily grasped visually by interpolating the shooting date of the image for creating the moving image to create the morphing.

The processing from step S1 to step S10 is the same as in the first embodiment. Base image P ₀ and the image P _n for the moving image creating selected in the present embodiment is assumed to be the same as the base image P ₀ and the moving image creating images P ₁ to P ₅ shown in FIG.

In step S10, when the moving image creating unit 28 sorts the base image P ₀ and the moving image creating images P _{1 to} P _n in time series, the interpolated image creating unit 34 performs time series between two adjacent images in time series. An interpolation image to be interpolated is created (step S21). Here, the number of interpolated images corresponding to the shooting interval between two adjacent images in time series order is created.

FIG. 8 is a diagram illustrating the shooting interval between two images that are arranged in the time series in which the base image P ₀ and the moving image creation images P _{1 to} P ₅ are arranged in the time series. As shown in FIG. 8, the shooting interval between the base image P ₀ and the moving image creation image P ₂ is about 6 months, and the shooting interval between the moving image generation image P ₂ and the moving image generation image P ₁ is about 6 months. shooting interval between creating image P ₁ and the moving image shooting interval between creating image P ₅ is from about 6 months, moving creating image P ₅ and video creating image P ₃ is about 12 months, and moving creating image P ₃ shooting interval of the video created for the image P ₄ and is about 6 months.

Here, it is assumed that an interpolation image is created every about three months. Therefore, the number of necessary interpolation images is one interpolation image for interpolating between the base image P ₀ and the moving image creating image P _2, and between the moving image creating image P ₂ and the moving image creating image P _1. One interpolated image to be interpolated, one interpolated image to interpolate between the moving image creating image P ₁ and the moving image creating image P _5, and between the moving image creating image P ₅ and the moving image creating image P _3. interpolation image for interpolating three, and the interpolation image for interpolating between the moving image creating image P ₃ and moving creating image P ₄ is one.

The interpolation image creation unit 34 creates a necessary number of interpolation images. For example, in the case of an interpolated image of the base image P ₀ and the moving image creating image P ₂ , an interpolated image that divides the motion vector detected from these two images into two equal parts is created. Further, if the interpolated image and the moving image creating image P ₅ and video creating image P _3, generating an interpolation image to 4 equally dividing the detected motion vector from the two images.

After creating the interpolated image, the moving image creating unit 28 creates a time-lapse moving image by combining the base image P ₀ and the moving image creating images P _{1 to} P _n sorted in time series order and the respective interpolated images in time series. (Step S11). Moreover, the control part 12 displays the moving image produced in the moving image production part 28 on the display part 32 (step S12).

  The moving image created in this manner smoothly reproduces the change from the previous image to the subsequent image in the time series order. Therefore, the observer can visually grasp the temporal change of the same eye to be examined more easily.

  In addition, by creating the number of interpolated images corresponding to the imaging interval between two images that are adjacent in time series order, the temporal change of the same eye to be examined can be accurately reproduced. Note that the number of interpolation images to be created is not particularly limited, and a number that is generally easy to visually recognize as a moving image may be created as appropriate.

<Third Embodiment>
[Configuration of ophthalmic image display device]
FIG. 9 is a block diagram illustrating a configuration of an ophthalmologic image display apparatus 50 according to the third embodiment. In addition, the same code | symbol is attached | subjected to the part which is common in the block diagram shown in FIG. 6, and the detailed description is abbreviate | omitted.

The alignment unit 18 of the ophthalmic image display device 50 calculates a rotational movement amount between the base image P ₀ and the moving image creation image P _n at a subpixel level, and based on the rotational movement amount, the base image P ₀ and Positioning in the rotational direction is performed with the moving image creation image _Pn . Thereafter, the alignment unit 18 calculates a parallel movement amount between the aligned base image P ₀ and the moving image creation image P _n at the sub-pixel level, and the base image P ₀ based on the parallel movement amount. And the moving image creation image P _n are aligned in the parallel direction.

  In order to realize such processing, the alignment unit 18 includes a rotational movement amount calculation unit 52, an image rotation unit 54, a parallel movement amount calculation unit 56, and an image parallel movement unit 58, as shown in FIG. ing.

The alignment unit 18 calculates a rotational movement amount and a parallel movement amount between the base image P ₀ and the moving image creation image P _n by performing a phase-only correlation process described later. Here, the alignment unit 18 calculates in advance POC (Phase Only Correlation) data (first data) for the base image used for the phase-only correlation process. Hereinafter, the POC data for the base image P ₀ may be referred to as base POC data, and the POC data for the moving image creation image P _n may be referred to as target POC data. When the target image POC data (second data) is calculated, the alignment unit 18 continues the phase-only correlation process using the target POC data and the base POC data obtained in advance. As a result, the processing load is reduced and the processing time is shortened, and a minute positional deviation amount is calculated.

(Phase only correlation processing)
In the phase only correlation process in the present embodiment, a known phase only correlation function is used.

First, base image P ₀ having an image size of N ₁ × N ₂ (N ₁ and N ₂ are positive integers) is represented by f (n ₁ , n ₂ ), and moving image creation image P _n is represented by g (n ₁ , n ₂ ). Here, n ₁ = −M ₁ ,..., M ₁ on the discrete space, N ₁ = 2M ₁ +1 (M ₁ is a positive integer), and f (n ₁ , n ₂ ) two-dimensional discrete Assuming that the calculation result of the Fourier transform (hereinafter referred to as DFT) is F (k ₁ , k ₂ ), F (k ₁ , k ₂ ) is expressed as equation (1).

In equation (1), AF (k ₁ , k ₂ ) is the amplitude component of f (n ₁ , n ₂ ), and e ^{jθF (k1, k2)} is the phase component of f (n ₁ , n ₂ ). .

Similarly, in a discrete space, n ₂ = −M ₂ ,..., M ₂ , N ₂ = 2M ₂ +1 (M ₂ is a positive integer), and a two-dimensional DFT of g (n ₁ , n ₂ ) If G (k ₁ , k ₂ ) is G (k ₁ , k ₂ ), then G (k ₁ , k ₂ ) is expressed as in equation (2).

In Expression (2), A _G (k ₁ , k ₂ ) is an amplitude component of g (n ₁ , n ₂ ), and e ^{jθG (k1, k2)} is a phase component of g (n ₁ , n ₂ ). is there.

  The phase only synthesis function used in the phase only synthesis process is defined as in equation (3) using equations (1) and (2).

  The phase-only correlation function in the present embodiment is obtained by applying a two-dimensional inverse discrete Fourier transformation (hereinafter referred to as IDFT) to the phase-only synthesis function of Equation (3) as shown in Equation (4). expressed.

The 2-dimensional images _s c of the continuous space is defined _(x 1, _{x 2),} by a minute amount of movement [delta] ₁ in _{x 1} direction and the image obtained by shifting the _{x 2} direction by a minute amount of movement [delta] ₂ is , S _c (x ₁ −δ ₁ , x ₂ −δ ₂ ). A two-dimensional image f (n ₁ , n ₂ ) on the discrete space sampled at the sampling interval T ₁ is defined as in Expression (5).

Similarly, a two-dimensional image g (n ₁ , n ₂ ) on the discrete space sampled at the sampling interval T ₂ is defined as in Expression (6).

In the formula (5) and _{(6), n 1 = -M} 1, ···, a _{M 1, n 2 = -M 2} , ···, a _{M 2.} Thereby, the phase-only correlation function regarding the two-dimensional images f (n ₁ , n ₂ ) and g (n ₁ , n ₂ ) on the discrete space is expressed in a general form as in Expression (7). In equation (7), α = 1.

  The alignment unit 18 that performs the phase-only correlation process will be described. FIG. 10 is a block diagram illustrating an example of the alignment unit 18.

(Rotational movement amount calculation unit)
The rotational movement amount calculation unit 52 calculates the rotational movement amount between the base image P ₀ and the moving image creation image P _n . Specifically, the rotational movement amount calculating section 52, by contrast with the base image P ₀ and the moving creating image P _n subjected to phase-only correlation processing between the base image P ₀ and the moving creating image P _n The amount of rotational movement is calculated. As shown in FIG. 10, such a rotational movement amount calculation unit 52 includes a first conversion processing unit 60, a logarithmic conversion unit 62, a polar coordinate conversion unit 64, a second conversion processing unit 66, A first phase only synthesis unit 68 and a first inverse transform processing unit 70 are provided.

First conversion processing unit 60, the base image _{P 0} to performing two-dimensional DFT processing. The first conversion processing unit 60 performs a two-dimensional DFT process on the moving image creation image _Pn . The two-dimensional DFT process performed by the first conversion processing unit 60 includes a two-dimensional DFT and a known shift process for shifting a quadrant with respect to the calculation result of the two-dimensional DFT. Hereinafter, this shift process may be referred to as “shift”. Note that the two-dimensional DFT performed by the first transformation processing unit 60 may be a two-dimensional FFT (Fast Fourier Transformation).

Logarithmic conversion unit 62, calculation result of the first conversion processing section 60 for the base image P ₀ to performing logarithmic transformation. In addition, the logarithmic conversion unit 62 performs logarithmic conversion on the calculation result by the first conversion processing unit 60 for the moving image creation image _Pn . The logarithmic transformation by the logarithmic transformation unit 62 has an effect of compressing an amplitude spectrum that tends to concentrate in a low frequency region of a spatial frequency in a natural image.

Polar coordinate conversion section 64, the operation result by the logarithmic transformation unit 62 for the base image P ₀ to performing polar coordinate conversion. In addition, the polar coordinate conversion unit 64 performs polar coordinate conversion on the calculation result of the logarithmic conversion unit 62 for the moving image creation image P _n . When logarithmic conversion by the logarithmic conversion unit 62 is not performed, the polar coordinate conversion unit 64 performs polar coordinate conversion on the calculation result of the first conversion processing unit 60 for the base image P ₀ , and the moving image creation image P _n is obtained. Polar coordinate conversion is performed on the calculation result by the first conversion processing unit 60. Polar conversion by polar coordinate transformation unit 64 is a process of converting the movement amount of the rotational direction to the amount of movement of the parallel direction (n ₁ direction and n ₂ direction) in the formula (1) to (7).

The second conversion processing unit 66 performs a two-dimensional DFT process (two-dimensional DFT + shift) on the calculation result by the polar coordinate conversion unit 64 for the base image P ₀ as shown in Expression (1). The processing result by the second conversion processing unit 66 for the base image P ₀ is, for example, controlled as base POC data (first data) normalized by the amplitude component prior to the arithmetic processing by the first phase only combining unit 68. It is stored in advance in a ROM (not shown) of the unit 12. Further, as shown in Expression (2), the second conversion processing unit 66 performs a two-dimensional DFT process (two-dimensional DFT + shift) on the calculation result by the polar coordinate conversion unit 64 for the moving image creation image _Pn . Note that the two-dimensional DFT performed by the second conversion processing unit 66 may also be a two-dimensional FFT.

As shown in Expression (3), the first phase only synthesis unit 68 is a second conversion processing unit for the base POC data (first data) obtained in advance for the base image P ₀ and the moving image creation image P _n. Based on the calculation result of 66, a phase-only combining process for combining the target POC data (second data) normalized with the amplitude component is performed.

  The first inverse transform processing unit 70 performs a two-dimensional IDFT process on the calculation result by the first phase only synthesis unit 68. The two-dimensional IDFT process performed by the first inverse transform processing unit 70 includes a two-dimensional IDFT and a known shift process for shifting the quadrant with respect to the calculation result of the two-dimensional IDFT. The two-dimensional IDFT may be calculated by a two-dimensional inverse fast Fourier transformation (hereinafter referred to as IFFT).

  The rotational movement amount calculation unit 52 calculates the rotational movement amount based on the calculation result by the first inverse conversion processing unit 70. Specifically, the rotational movement amount calculation unit 52 specifies the peak position based on the calculation result by the first inverse conversion processing unit 70, thereby moving the rotational direction movement amount (rotational movement amount, displacement amount) at the pixel level. ) Is calculated. Thereafter, the rotational movement amount calculation unit 52 specifies the pixel position when the correlation value of the phase-only correlation function shown in Expression (7) is maximum in the vicinity of the peak position specified at the pixel level. The amount of movement in the rotation direction (rotation movement amount, displacement amount) is calculated at the pixel level.

Rotational movement amount calculating section 52, based on the base image P _0, it is described that calculates a movement amount of the rotation direction of the image P _n for creating video, but is not limited thereto. The rotational movement amount calculation unit 52 may calculate the movement amount in the rotation direction of the base image based on the target image.

  Further, the rotational movement amount calculation unit 52 does not have to calculate the rotational movement amount by the phase-only correlation process. The rotational movement amount is calculated by a known method, and the calculated rotational movement amount is used as the image rotation unit 54. May be output.

(Image rotation part)
The image rotation unit 54 aligns the rotation direction between the base image P ₀ and the moving image creation image P _n based on the rotation movement amount calculated by the rotation movement amount calculation unit 52. Specifically, the image rotation unit 54 uses the rotation movement amount calculated by the rotation movement amount calculation unit 52 to make the moving image creation image P so that the rotation movement amount becomes zero with reference to the base image P _0. Alignment in the rotation direction with respect to _n .

(Translation amount calculation unit)
The parallel movement amount calculation unit 56 calculates the parallel movement amount between the base image P ₀ that has been aligned by the image rotation unit 54 and the moving image creation image P _n . Specifically, the parallel movement amount calculation unit 56 performs phase-only correlation processing on the base image P ₀ and the moving image creation image P _n that have been aligned by the image rotation unit 54, thereby performing the base image P _0. And the parallel movement amount between the moving image creation image _Pn . As shown in FIG. 10, the parallel movement amount calculation unit 56 includes a third conversion processing unit 72, a second phase only combining unit 74, and a second inverse conversion processing unit 76.

The third conversion processing unit 72 performs two-dimensional DFT processing (two-dimensional DFT + shift) on the base image P ₀ as shown in Expression (1). The processing result of the third conversion processing unit 72 for the base image P ₀ is stored, for example, as base POC data (third data) normalized by the amplitude component prior to the arithmetic processing by the second phase only combining unit 74. Stored in the unit 212 in advance. Further, as shown in Expression (2), the third conversion processing unit 72 performs a two-dimensional DFT process (two-dimensional DFT + shift) on the moving image creation image _Pn . Note that the two-dimensional DFT performed by the third conversion processing unit 72 may be a two-dimensional FFT.

As shown in Expression (3), the second phase only synthesis unit 74 is a third conversion processing unit for base POC data (third data) obtained in advance for the base image P ₀ and the moving image creation image P _n. Based on the calculation result by 72, a phase only synthesis process for synthesizing the target POC data (fourth data) normalized by the amplitude component is performed.

  The second inverse transform processing unit 76 performs a two-dimensional IDFT process (two-dimensional IDFT + shift) on the calculation result by the second phase only synthesis unit 74. Note that the two-dimensional IDFT performed by the second inverse transform processing unit 76 may be a two-dimensional IFFT.

  The parallel movement amount calculation unit 56 calculates the parallel movement amount based on the calculation result by the second inverse transformation processing unit 76. Specifically, the parallel movement amount calculation unit 56 specifies the peak position based on the calculation result by the second inverse conversion processing unit 76, thereby moving the movement amount in the parallel direction (parallel movement amount, positional deviation amount) at the pixel level. ) Is calculated. After that, the parallel movement amount calculation unit 56 specifies the pixel position when the correlation value of the phase-only correlation function shown in Expression (7) is maximum in the vicinity of the peak position specified at the pixel level. The amount of movement in the parallel direction (parallel movement amount, displacement amount) is calculated at the pixel level.

Translation amount calculating section 56, based on the base image P _0, it is described that calculates a movement amount in the parallel direction of the image P _n for creating video, but is not limited thereto. The parallel movement amount calculation unit 56 may calculate the movement amount in the parallel direction of the base image P ₀ with reference to the moving image creation image P _n .

  The translation amount calculated by the translation amount calculation unit 56 is output to the image translation unit 58.

(Image translation part)
The image translation unit 58 performs alignment in the parallel direction between the base image P ₀ and the moving image creation image P _n based on the translation amount calculated by the translation amount calculation unit 56. Specifically, the image translation unit 58 uses the translation amount calculated by the translation amount calculation unit 56 to create a moving image creation image so that the translation amount becomes zero with reference to the base image P _0. Position alignment in the parallel direction with respect to _Pn .

[Video creation method]
FIG. 11 is a flowchart showing a moving image display process executed by the ophthalmologic image display apparatus 50 according to the present embodiment. In addition, the same code | symbol is attached | subjected to the part which is common in the flowchart shown in FIG. 7, and the detailed description is abbreviate | omitted.

  In the present embodiment, phase-only correlation processing is used for image alignment when creating a moving image.

Like the second embodiment, and initializes the variable n to 0 (step S1), the observer selects a base image P ₀ (step S2).

Next, the variable n is incremented (step S4), and a moving image creation image _Pn for creating a moving image is selected (step S5).

Subsequently, the rotational movement amount calculating section 52 calculates a rotational movement amount between the base image P ₀ and the moving creating image P _n (step S31). Further, the image rotation unit 54 performs alignment in the rotation direction between the base image P ₀ and the moving image creation image P _n based on the rotation movement amount calculated by the rotation movement amount calculation unit 52 (step S32). .

Next, the parallel movement amount calculation unit 56 calculates the parallel movement amount between the base image P ₀ and the moving image creation image P _n that have been aligned by the image rotation unit 54 (step S33). Furthermore, the image translation unit 58 performs alignment in the parallel direction between the base image P ₀ and the moving image creation image P _n based on the translation amount calculated by the translation amount calculation unit 56 (step S34). ).

Hereinafter, as in the second embodiment, by changing the brightness of the moving image creating images P _n as the brightness of the moving image creating images P _n coincides with the brightness of the base image P ₀ (step S8), and The same processing is repeated until the selection of the moving image creation image P _n for creating a moving image is completed (step S9).

  Thus, according to the present embodiment, a more appropriate moving image can be created by using the phase-only correlation process for image alignment when creating a moving image.

<Others>
In the embodiment described so far, for example, the control unit 12, the alignment unit 18, the brightness change unit 26, the moving image creation unit 28, the interpolation image creation unit 34, the rotational movement amount calculation unit 52, the image rotation unit 54, the parallel movement. An amount calculation unit 56, an image translation unit 58, a first conversion processing unit 60, a logarithmic conversion unit 62, a polar coordinate conversion unit 64, a second conversion processing unit 66, a first phase only synthesis unit 68, and a first inverse conversion processing unit 70. The hardware structure of the processing unit (processing unit) that executes various processes such as the third conversion processing unit 72, the second phase-only combining unit 74, and the second inverse conversion processing unit 76 is as follows. There are various kinds of processors. For various processors, the circuit configuration can be changed after manufacturing a CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), which is a general-purpose processor that functions as various processing units by executing software (programs). Includes a dedicated electrical circuit that is a processor having a circuit configuration specifically designed to execute a specific process such as a programmable logic device (PLD) or ASIC (Application Specific Integrated Circuit) It is.

  One processing unit may be configured by one of these various processors, or may be configured by two or more processors of the same type or different types (for example, a plurality of FPGAs or a combination of CPUs and FPGAs). May be. Further, the plurality of processing units may be configured by one processor. As an example of configuring a plurality of processing units with one processor, first, as represented by a computer such as a server and a client, one processor is configured with a combination of one or more CPUs and software. There is a form in which the processor functions as a plurality of processing units. Second, as represented by a system-on-chip (SoC) or the like, there is a form of using a processor that realizes the functions of the entire system including a plurality of processing units with a single integrated circuit (IC) chip. is there. As described above, various processing units are configured using one or more various processors as a hardware structure.

  Further, the hardware structure of these various processors is more specifically an electric circuit (circuitry) in which circuit elements such as semiconductor elements are combined.

  The ophthalmologic image display method according to the present embodiment is configured as a program for causing a computer to execute each of the above steps, and is a non-temporary method such as a CD-ROM (Compact Disk-Read Only Memory) that stores the configured program. It is also possible to configure a recording medium.

  The technical scope of the present invention is not limited to the scope described in the above embodiment. The configurations and the like in the embodiments can be appropriately combined between the embodiments without departing from the gist of the present invention.

DESCRIPTION OF SYMBOLS 10 Ophthalmological image display apparatus 12 Control part 14 Communication part 16 Image memory 18 Positioning part 20 Feature point extraction part 22 Corresponding point detection part 24 Image deformation part 25 Image selection part 26 Brightness change part 28 Movie creation part 30 Operation part 32 Display Unit 34 Interpolated Image Creation Unit 40 Ophthalmic Image Display Device 50 Ophthalmic Image Display Device 52 Rotation Movement Amount Calculation Unit 54 Image Rotation Unit 56 Parallel Movement Amount Calculation Unit 58 Image Parallel Movement Unit 60 First Conversion Processing Unit 62 Logarithmic Conversion Unit 64 Polar Coordinate Conversion Unit 66 second transformation processing unit 68 first phase only synthesis unit 70 first inverse transformation processing unit 72 third transformation processing unit 74 second phase only synthesis unit 76 second inverse transformation processing unit 100 server A ₁ feature point A ₂ feature Point A ₃ Feature point _An Feature point B ₁ Corresponding point B ₂ Corresponding point B ₃ Corresponding point B _n Corresponding point S1 to S34 Step of moving image display processing

Claims (13)

An image acquisition unit that acquires a plurality of captured images related to the same eye to be examined or a plurality of analysis images related to the shape of the same eye to be examined;
An alignment unit for aligning the acquired plurality of images;
A moving image creating unit that creates a moving image by joining the plurality of aligned images in chronological order;
A display unit for displaying the created video;
An ophthalmic image display device. The alignment unit is
A feature point extraction unit that extracts a plurality of feature points from a first image that is one of the plurality of images;
A corresponding point detection unit that detects a plurality of corresponding points corresponding to the plurality of feature points from a second image other than the first image of the plurality of images;
An image deformation unit that deforms the second image to match the positions of the plurality of corresponding points with the positions of the corresponding feature points, respectively;
The ophthalmic image display device according to claim 1, comprising:   The ophthalmologic image display apparatus according to claim 2, wherein the corresponding point detection unit detects the plurality of corresponding points using template matching.   The ophthalmic image display device according to claim 2, wherein the image deforming unit matches the positions of the plurality of corresponding points with the corresponding positions of the plurality of feature points using affine transformation. The alignment unit is
A rotational movement amount calculation unit that calculates a rotational movement amount between a first image that is one of the plurality of images and a second image other than the first image;
An image rotation unit that performs alignment in the rotation direction between the first image and the second image based on the rotation movement amount calculated by the rotation movement amount calculation unit;
A phase-only correlation process is performed on the first image and the second image that have been aligned by the image rotation unit, thereby calculating a parallel movement amount between the first image and the second image. A translation amount calculation unit;
An image translation unit that performs alignment in the parallel direction between the first image and the second image based on the translation amount calculated by the translation amount calculation unit;
The ophthalmic image display device according to claim 1, comprising:   The rotational movement amount calculation unit calculates a rotational movement amount between the first image and the second image by performing phase-only correlation processing on the first image and the second image. 5. An ophthalmologic image display device according to 5. The rotational movement amount calculation unit
A first transformation processing unit that performs a discrete Fourier transform on the second image;
A polar coordinate conversion unit that performs polar coordinate conversion on a calculation result of the first conversion processing unit for the second image;
A second transformation processing unit that performs a discrete Fourier transform on the calculation result of the polar coordinate transformation unit for the second image;
A first phase for performing a phase-only combining process for combining the first data obtained in advance for the first image and the second data obtained based on the calculation result of the second conversion processing unit for the second image. Limited synthesis department,
A first inverse transform processing unit that performs an inverse discrete Fourier transform on the calculation result by the first phase only synthesis unit,
The ophthalmic image display apparatus according to claim 6, wherein the rotational movement amount is calculated based on a calculation result by the first inverse conversion processing unit.   The ophthalmic image display apparatus according to claim 1, further comprising a brightness changing unit that matches brightness of each of the plurality of images. An interpolation image creating unit that creates an interpolated image that interpolates between two images adjacent in the time series order among the plurality of images connected in the time series order;
The ophthalmic image display device according to claim 1, wherein the moving image creating unit creates a moving image including the interpolated image.   The ophthalmologic image display device according to claim 1, wherein the captured image is a front image of the fundus or a tomographic image of the fundus.   The ophthalmologic image according to any one of claims 1 to 10, wherein the analysis image is a thickness map representing a two-dimensional distribution of the layer thickness of the fundus oculi or a thickness change map representing a time-series variation of the layer thickness. Display device. An image acquisition step of acquiring a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the same eye to be examined;
An alignment step of aligning the plurality of acquired images;
A moving image creating step for creating a moving image by connecting the plurality of aligned images in time series; and
A display step of displaying the created video on a display unit;
An ophthalmologic image display method comprising: On the computer,
An image acquisition step of acquiring a plurality of captured images relating to the same eye to be examined or a plurality of analysis images relating to the shape of the same eye to be examined;
An alignment step of aligning the plurality of acquired images;
A moving image creating step for creating a moving image by connecting the plurality of aligned images in time series; and
A display step of displaying the created video on a display unit;
A program for running

Images