JPWO2019107342A1 - Panorama fundus image generator and program - Google Patents

Abstract

The panoramic fundus image generator corresponds to the target pixel of the first fundus image as the pixel value of the target pixel included in the overlapping region of the first fundus image and the second fundus image whose positional relationship is determined by the positioning unit. The sum of the value obtained by multiplying the pixel value of one pixel by the first coefficient and the value obtained by multiplying the pixel value of the second pixel corresponding to the target pixel of the second fundus image by the second coefficient is derived, and the derived pixel value is derived. It is provided with an image synthesizing unit that generates a panoramic fundus image by synthesizing a first fundus image and a second fundus image.

Description

The present invention relates to a panoramic fundus image generator and a program.

Conventionally, there is known an imaging device that injects light from the pupil and captures a fundus image based on the reflected light from the fundus. In this type of imaging device, the subject is gazed at the fixation lamp for imaging, and by changing the position of the fixation lamp, it is possible to image a different place of the fundus (for example, Patent Document). 1). Further, a method of acquiring a wide range of fundus images by stitching the images of the same eye to be taken in this way is known as a panoramic photographing method (see, for example, Patent Document 2). However, when a panoramic image is generated by connecting a plurality of fundus images in this way, it is known that distortion occurs in each image because the fundus is not flat, and the same blood vessel is not connected in the overlapping portion of the plurality of fundus images. ing. On the other hand, there is known that a panoramic image is generated using a fundus image that has been subjected to distortion correction processing (see, for example, Patent Document 3).

Japanese Unexamined Patent Publication No. 2004-135941 JP-A-2009-112617 JP-A-2017-55916

However, in the conventional technique, a line is inserted at the joint part of the panoramic image connecting the blood vessels, the blood vessel is cut or shifted at the joint part, it appears twice, and the thickness changes. , Or disappeared in some cases.

The present invention has been made in consideration of such circumstances, and is a panoramic fundus image generator capable of maintaining a natural appearance while maintaining necessary information of panoramic fundus images in which fundus images are connected to each other. One of the purposes is to provide a program.

One aspect of the present invention for solving the above problem is an acquisition unit that acquires a plurality of fundus images obtained by imaging different ranges of the same eye to be examined in the fundus of the eye, and a plurality of acquisition units acquired by the acquisition unit. A distortion correction unit that corrects distortion of the fundus image, and a first and second fundus images that are joined to each other among a plurality of fundus images that have been subjected to distortion correction processing by the distortion correction unit. Regarding the image, the positioning unit that determines the positional relationship when connecting the first fundus image and the second fundus image based on the feature points in each fundus image, and the positioning unit that determines the positional relationship by the positioning unit. As the pixel value of the target pixel included in the overlapping region of the first fundus image and the second fundus image, the value obtained by multiplying the pixel value of the first pixel corresponding to the target pixel of the first fundus image by the first coefficient. , The sum of the pixel value of the second pixel corresponding to the target pixel of the second fundus image multiplied by the second coefficient is derived, and the derived pixel value is used to obtain the first fundus image and the first fundus image. It is a panoramic fundus image generation device including an image compositing unit for generating a panoramic fundus image obtained by combining two fundus images.

According to the present invention, it is possible to obtain a natural appearance while maintaining the necessary information of the panoramic fundus images in which the fundus images are connected to each other.

It is a figure which shows the whole structure of the fundus imaging system 1 including the panoramic fundus image generator which concerns on embodiment. It is a front view of the image pickup apparatus 5. It is a figure which shows an example of the 1st fundus image F11. It is a figure which shows an example of the 2nd fundus image F21. It is a figure which shows an example of the image which performed the strain correction processing on the 1st fundus image F11. It is a figure which shows an example of the image which performed the strain correction processing on the 2nd fundus image F21. It is a figure which shows an example of the blood vessel structure image F13 which shows the blood vessel structure of the first fundus image F12. It is a figure which shows an example of the blood vessel structure image F23 which shows the blood vessel structure of the 2nd fundus image F22. It is a reference figure for demonstrating how to connect panoramic fundus images. It is a figure which shows an example of the relationship between the 1st coefficient and the 2nd coefficient. It is a figure which shows the specific example of the derivation method of the pixel value of the overlap area pixel. It is a flowchart which shows an example of processing by a panoramic fundus image generation apparatus 100. It is a flowchart which shows an example of the processing by the image composition part 129. It is a figure which shows an example which connected the five fundus images. It is a figure which shows an example of the case where three or more fundus images overlap in the same region. It is a figure for demonstrating duplication of fundus images after excision. It is a figure which shows another example of the relationship between a 1st coefficient and a 2nd coefficient.

Hereinafter, embodiments of the panoramic fundus image generator and program of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an overall configuration of a fundus imaging system 1 including a panoramic fundus image generator according to an embodiment. As shown in FIG. 1, the fundus imaging system 1 includes an imaging device 5 and a panoramic fundus image generating device 100. The image pickup device 5 and the panoramic fundus image generation device 100 may be connected by wire or by short-range wireless communication such as Bluetooth (registered trademark). Further, the image pickup device 5 and the panoramic fundus image generation device 100 may communicate with each other using the network NW. The network NW includes, for example, a part or all of WAN (Wide Area Network), LAN (Local Area Network), the Internet, a provider device, a radio base station, a dedicated line, and the like.

FIG. 2 is a front view of the image pickup apparatus 5. As shown in FIG. 2, the image pickup device 5 includes a jaw base 10, a base 15, and an image pickup unit 20. The chin rest 10 is a portion on which the subject's chin is placed. The base 15 supports the chin rest 10 and the imaging unit 20. The base 15 supports the imaging unit 20 so that it can move in the horizontal direction (arrows R and L in the drawing) within a predetermined range. As a result, the examiner adjusts the relative positional relationship between the jaw base 10 and the imaging unit 20 so that the imaging unit 20 can image the eyes of the subject with the jaw resting on the jaw base 10. be able to.

The imaging unit 20 is, for example, a scanning laser ophthalmoscope (SLO). The image pickup unit 20 may be a fundus camera that captures a fundus image using a CCD camera.

The imaging unit 20 images different ranges of the same eye to be inspected in the fundus of the eye to be inspected, and outputs image data of a plurality of captured fundus images. The ranges that differ from each other include those that partially include the same region. The imaging unit 20 images different ranges by changing the incident position and the incident angle of the light incident from the pupil. Further, the imaging unit 20 may image different ranges by changing the relative position of the objective lens of the fundus camera with respect to the eye to be inspected.

Next, returning to FIG. 1, the panoramic fundus image generation device 100 will be described. As shown in FIG. 1, the panoramic fundus image generator 100 includes, for example, a communication unit 111, an input unit 113, a display unit 115, a storage unit 117, an imaging control unit 121, an acquisition unit 123, and a distortion correction unit. It includes 125, a positioning unit 127, and an image composition unit 129.

The communication unit 111 transmits / receives information to / from the image pickup device 5 connected by wire or wirelessly. Further, the communication unit 111 may have hardware (for example, an antenna and a transmission / reception device) for connecting to the network NW using a cellular network, a Wi-Fi network, or the like.

The input unit 113 includes a part or all of various keys, buttons, dial switches, a mouse, and the like. Further, the input unit 113 may be, for example, a touch panel TP formed integrally with the display unit 115.

The display unit 115 is, for example, an LCD (Liquid Crystal Display), an organic EL (Electroluminescence) display device, or the like.

The storage unit 117 is, for example, a flash memory such as a RAM (Random Access Memory), a ROM (Read Only Memory), an SSD (Solid State Drive), an HDD (Hard Disk Drive), or the like. Information such as fundus image data 171 and coefficient data 173 is stored in the storage unit 117.

The fundus image data 171 is image data of the fundus image captured by the imaging device 5 (hereinafter, simply referred to as a fundus image). The fundus image data 171 may include image data processed by the distortion correction unit 125, the positioning unit 127, the image composition unit 129, and the like. Identification information (hereinafter referred to as an image ID) is assigned to the fundus image, and information indicating the position in the fundus where each fundus image is captured, information indicating the subject, and the like are associated with each other. ..

The coefficient data 173 is a coefficient used when a plurality of fundus images are connected to generate a panoramic fundus image. This coefficient will be described later.

The image pickup control unit 121, the acquisition unit 123, the distortion correction unit 125, the positioning unit 127, and the image composition unit 129 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). To. In addition, some or all of these components are hardware (circuits) such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), and GPU (Graphics Processing Unit). It may be realized by the part (including circuitry), or it may be realized by the cooperation of software and hardware. The program may be stored in advance in a storage device such as a computer's HDD (Hard Disk Drive) or flash memory, or is stored in a removable storage medium such as a DVD or CD-ROM, and the storage medium is a drive. It may be installed in the storage device by being attached to the device.

The image pickup control unit 121 controls the image pickup device 5 based on the inspector's instruction received by the input unit 113. For example, when an inspector instructs to capture a panoramic fundus image, the imaging control unit 121 controls the imaging device 5 to capture images in different ranges.

The acquisition unit 123 acquires the fundus image captured by the image pickup device 5 by using the communication unit 111. The acquisition unit 123 stores the acquired fundus image in the fundus image data 171 of the storage unit 117. Further, the acquisition unit 123 may directly output the acquired fundus image to the strain correction unit 125. Hereinafter, an example will be described in which the acquisition unit 123 acquires the first fundus image F11 shown in FIG. 3 and the second fundus image F21 shown in FIG. 4 as the fundus image.

FIG. 3A is a diagram showing an example of the first fundus image F11. FIG. 3B is a schematic view of a partial region F11a shown in FIG. 3A. FIG. 4A is a diagram showing an example of the second fundus image F21. FIG. 4B is a schematic view of a partial region F21a shown in FIG. 4A. Θ1 shown in FIG. 3B and θ2 shown in FIG. 4B are branch angles at the same branch point, respectively. The first fundus image F11 and the second fundus image F21 are fundus images obtained by capturing different ranges of the same eye to be examined in the fundus of the eye to be inspected. The first fundus image F11 and the second fundus image F21 capture different locations by changing the relative positions of the optical system of the device and the eye to be inspected (pupil) and changing the incident angle at which the illumination light is incident on the anterior segment of the eye. It is an image that was made. As shown in the figure, the first fundus image F11 and the second fundus image F21 include a fundus portion (bright area) and other outer edge portions (dark area). The fundus portion in the first fundus image F11 and the second fundus image F21 has a shape close to a circle.

The distortion correction unit 125 performs a distortion correction process (hereinafter, referred to as a distortion correction process) on the fundus image read from the storage unit 117 (or the fundus image input from the acquisition unit 123). The strain correction process is a process for correcting the strain that occurs when the fundus of the eye to be inspected, which is a sphere, is represented by a plane. Further, the distortion correction unit 125 may correct the distortion so that the shape of the image of the fundus portion included in the fundus image is approximated to a square. FIG. 5A is a diagram showing an example of an image (first fundus image F12) obtained by performing strain correction processing on the first fundus image F11 shown in FIG. 3A. FIG. 5B is a schematic view of a partial region F12a shown in FIG. 5A. Similarly, FIG. 6A is a diagram showing an example of an image (second fundus image F22) obtained by subjecting the second fundus image F21 shown in FIG. 4A to distortion correction processing. FIG. 6B is a schematic view of a partial region F22a shown in FIG. 6A. Θ3 shown in FIG. 5B and θ4 shown in FIG. 6B are branch angles at the same branch point, respectively.

An example of the effect of performing this distortion correction will be described. For example, at the bifurcation of a blood vessel in the upper direction of the optic nerve head in F11, the bifurcation angle of the blood vessel: θ1 is 44.2 °, but the bifurcation angle at the same location in F21: θ2 is 38.5 °. It has become. Therefore, since θ1 and θ2 are different by about 4 °, it can be expected that the blood vessels at the boundary portion will not be connected well even if the F11 and F21 are connected as they are. Therefore, when distortion correction processing is performed on both images, the branch angle at F12: θ3 is 42.7 °, and the branch angle at F22: θ4 is 43.0 °, so there is almost no difference, so a good connection can be achieved. it is conceivable that.

The positioning unit 127 determines the positional relationship between the first fundus image F12 and the second fundus image F22 (hereinafter, also referred to as the positional relationship of both images) that are joined to each other among the plurality of fundus images that have undergone distortion correction processing. decide. The positioning unit 127 determines the positional relationship when connecting the two images so that the same blood vessels overlap in the overlapping region of the first fundus image F12 and the second fundus image F22, based on the positions of the blood vessels in each fundus image. ..

For example, the positioning unit 127 acquires the feature amount of the branch point of the blood vessel included in the fundus image, and determines the positional relationship between the two images based on the acquired feature amount. Specifically, the positioning unit 127 acquires a feature amount indicating the angle and shape of the blood vessel at the bifurcation point by performing image processing on the fundus image so that the bifurcation points having similar feature amounts overlap each other. Determine the positional relationship between both images.

The positioning unit 127 acquires the blood vessel structure images F13 and F23 as shown in FIGS. 7 and 8 by performing image processing on the fundus image, for example. FIG. 7 is a diagram showing an example of a blood vessel structure image F13 showing a blood vessel structure of the first fundus image F12. FIG. 8 is a diagram showing an example of a blood vessel structure image F23 showing the blood vessel structure of the second fundus image F22. The positioning unit 127 uses the blood vessel structure images F13 and F23 to determine the positional relationship between the two images so that the bifurcation points having similar feature amounts overlap each other.

As described above, the distortion of the blood vessel structure is determined by determining the positional relationship between the two images using the blood vessel structure images F13 and F23 based on the fundus images F12 and F22 that have been subjected to the distortion correction processing by the distortion correction unit 125. It can be prevented and the possibility that the overlapping positions of the same blood vessels match can be increased.

The image synthesizing unit 129 derives the pixel values of the pixels included in the overlapping region (hereinafter referred to as overlapping region pixels) in the positional relationship determined by the positioning unit 127, and uses the derived pixel values of the overlapping region pixels. , A panoramic fundus image F31 is generated by combining the first fundus image F12 and the second fundus image F22. The pixel value is, for example, brightness, but may be a value indicating the intensity of each color.

Here, an example of how to connect the panoramic fundus images F31 will be described with reference to FIG. FIG. 9 is a reference diagram for explaining how to join the panoramic fundus images.
In the illustrated example, it is assumed that the X-axis and the Y-axis of each image coincide with each other and there is no distortion in the rotation direction.

As shown in FIG. 9, the first fundus image F12 includes an overlapping region R14 which is an overlapping portion with the second fundus image F22, and other non-overlapping regions R15 and R16. The second fundus image F22 includes an overlapping region R24 which is an overlapping portion with the first fundus image F11, and other non-overlapping regions R25 and R26. The non-overlapping regions R16 and R26 are outer edges cut by the image synthesizing unit 129. As shown in FIGS. 3 and 4, the outer edge of the fundus image is black. The image synthesizing unit 129 cuts out the portions (non-overlapping regions R16 and R26) including black (luminance value is 0) on the side overlapping with the other fundus screen.

The panoramic fundus image F31 includes an overlapping region R32 based on the overlapping region R14 and the overlapping region R24, a non-overlapping region R15 of the first fundus image F11, and a non-overlapping region R25 of the second fundus image F21. In the panoramic fundus image F31, a portion including black (luminance value is 0) is trimmed at the outer edge portion by the image synthesizing unit 129. The image synthesizing unit 129 may cut off a portion of the other side including black at the stage of cutting out the non-overlapping regions R16 and R26.

A center set at the center in the Y-axis direction in the overlapping region R14 in a state where the first fundus image F12 and the second fundus image F22 are positioned by the positioning unit 127 (hereinafter, both images are referred to as positioned states). The line L1 and the center line L2 set at the center in the Y-axis direction in the overlapping area R24 coincide with the center line L3 in the overlapping area R32. That is, the center line L3 is set at a position in the overlapping region R32 where the distance to the non-overlapping region R15 and the distance to the non-overlapping region R25 are equal.

In the example shown in FIG. 9, each pixel constituting the overlapping region R32 corresponds to one pixel included in the overlapping region R14 and one pixel included in the overlapping region R24, respectively. That is, each pixel constituting the overlapping region R32 corresponds to both overlapping pixels (one pixel included in the overlapping region R14 and one pixel included in the overlapping region R24) in a state where both images are positioned. In this way, one pixel included in the overlapping areas R14 and R24 corresponding to each pixel constituting the overlapping area R32 is referred to as a corresponding pixel.

The image synthesizing unit 129 sets the pixel value of each overlapping region pixel included in the overlapping region R32 to a value obtained by multiplying the pixel value of the corresponding pixel in the overlapping region R14 by the first coefficient (first pixel value) and the overlapping region R24. The sum of the pixel value of the corresponding pixel in (2) multiplied by the second coefficient (second pixel value) is derived.

The first coefficient and the second coefficient have a predetermined relationship. An example of the relationship between the first coefficient and the second coefficient will be described with reference to FIG. FIG. 10 is a diagram showing an example of the relationship between the first coefficient and the second coefficient.

As shown in FIG. 10A, the first coefficient C1 is a constant value in the non-overlapping region R15, and C1 = 1 (100%). Further, the first coefficient C1 is set so as to gradually become 0 (0%) in the overlapping region R14. For example, among the pixels included in the overlapping region R14, the first coefficient C1 corresponding to the pixel closest to the non-overlapping region R15 is 1. Further, among the pixels included in the overlapping region R14, the first coefficient C1 corresponding to the pixel farthest from the non-overlapping region R15 is 0. Further, the first coefficient C1 has a constant inclination and is 0.5 (50%) in the pixels that coincide with the center line L1 in the overlapping region R14.

As shown in FIG. 10B, the second coefficient C2 is a constant value in the non-overlapping region R25, and C2 = 1 (100%). Further, the second coefficient C2 is set so as to gradually become 0 (0%) in the overlapping region R24. For example, among the pixels included in the overlapping region R24, the second coefficient C2 corresponding to the pixel closest to the non-overlapping region R25 is 1. Further, among the pixels included in the overlapping region R24, the second coefficient C2 corresponding to the pixel farthest from the non-overlapping region R25 is 0. Further, the second coefficient C2 has a constant inclination and is 0.5 (50%) in the pixels corresponding to the center line L2 in the overlapping region R24.

That is, the image synthesizing unit 129 sets the first coefficient C1 to be larger than the second coefficient C2 for the pixels included in the overlapping area R32 that is closer to the non-overlapping area R15 of the first fundus image F12 than the center line L3. To do. The image synthesizing unit 129 sets the second coefficient C2 to be larger than the first coefficient C1 for the pixels included in the overlapping area R32 that is closer to the non-overlapping area R25 of the second fundus image F22 than the center line L3. The image synthesizing unit 129 increases the first coefficient C1 and decreases the second coefficient C2 as the overlapping region pixels closer to the non-overlapping region R15 of the first fundus image F12 in the overlapping region R32. The image synthesizing unit 129 increases the second coefficient C2 and decreases the first coefficient C1 as the overlapping region pixels closer to the non-overlapping region R25 of the second fundus image F22 in the overlapping region R32.

Further, in both coefficients, the relationship that the sum of the first coefficient and the second coefficient installed for each pixel included in the overlapping region R32 is 1 is established.

Next, with reference to FIG. 11, a specific example of a method for deriving the pixel values of the overlapping region pixels will be described.
FIG. 11 is a diagram showing a specific example of a method for deriving the pixel values of the overlapping region pixels. FIG. 11A shows an example of the arrangement of pixels in all the overlapping regions R32. For example, in the overlapping region R32, 6 pixels in the X-axis direction and 10 pixels in the Y-axis direction, for a total of 60 pixels, are arranged. In the illustrated example, the number of pixels is small for the sake of simplicity. It is assumed that each pixel is designated by P11 to P70. Although the illustration of all the symbols is omitted, the symbols of the values added one by one in the Y-axis direction are attached.

11 (b), (c), and (d) show a part of the first fundus image F12, the second fundus image F22, and the panoramic fundus image F31, respectively. As shown in the figure, the arrangement of the pixels in the overlapping areas R14 and R24 is also the same as the arrangement of the pixels in the overlapping area R32, and the pixels corresponding to the overlapping area pixels will be described using the same reference numerals.

The following first coefficient C1 is set in each pixel of the overlapping region R14.
Pixels P11 to P20 have a first coefficient C1 = 1.
Pixels P21 to P30 have a first coefficient C1 = 0.8.
Pixels P31 to P40 have a first coefficient C1 = 0.6.
Pixels P41 to P50 have a first coefficient C1 = 0.4.
Pixels P51 to P60 have a first coefficient C1 = 0.2.
Pixels P61 to P70 have a first coefficient C1 = 0.

The following second coefficient C2 is set in each pixel of the overlapping region R24.
Pixels P11 to P20 have a first coefficient C1 = 0.
Pixels P21 to P30 have a first coefficient C1 = 0.2.
Pixels P31 to P40 have a first coefficient C1 = 0.4.
Pixels P41 to P50 have a first coefficient C1 = 0.6.
Pixels P51 to P60 have a first coefficient C1 = 0.8.
Pixels P61 to P70 have a first coefficient C1 = 1.

In this way, with the first fundus image F12 and the second fundus image F22 positioned, the sum of the first coefficient and the second coefficient is 1 in the overlapping pixels (pixels having the same code). There is.

For example, the image synthesizing unit 129 executes the following calculation as the pixel value of the overlapping region pixel P35.
Pixel value (77) of pixel P35 of overlapping region R14 x first coefficient C1 (0.6)
+ Pixel value (26) of pixel P35 in overlapping region R24 x second coefficient C2 (0.4) = 57
In this way, the image synthesizing unit 129 derives the pixel values of the overlapping region pixels P11 to P70.

Next, a processing example of the panoramic fundus image generation device 100 will be described with reference to FIG. FIG. 12 is a flowchart showing an example of processing by the panoramic fundus image generator 100.

As shown in FIG. 12, the imaging control unit 121 determines whether or not an imaging instruction has been received from the inspector by the input unit 113 (step S11). The image pickup control unit 121 repeats the process of step S11 until it receives an image pickup instruction. When the image pickup instruction is received, the image pickup control unit 121 controls the image pickup apparatus 5 according to the instruction (step S12). Then, the fundus image is imaged by the image pickup device 5. The acquisition unit 123 acquires the fundus image captured by the imaging device 5 (step S13). Next, the image pickup control unit 121 determines whether or not the entire designated range has been imaged (step S14). If the entire range has not been imaged, the image pickup control unit 121 returns to step S12 and repeats the process. For example, the image pickup control unit 121 instructs the image pickup apparatus 5 to change the incident angle of the laser beam to image a different range.

When the entire designated range is imaged, the strain correction unit 125 performs the strain correction process on all the fundus images acquired by the acquisition unit 123 (step S15). Next, the positioning unit 127 acquires the feature amount of the branch point of the blood vessel included in the fundus image (step S16).
Based on the acquired features, the positioning unit 127 determines the positional relationship between the first fundus image and the second fundus image that are joined to each other among the plurality of fundus images that have undergone distortion correction processing (step S17). .. The image synthesizing unit 129 derives the pixel values of the overlapping region pixels in the positional relationship determined by the positioning unit 127, and uses the derived pixel values of the overlapping region pixels to obtain the first fundus image and the second fundus image. A composite panoramic fundus image is generated (step S18). Then, the image synthesizing unit 129 displays the generated panoramic fundus image on the display unit 115 (step S19).

Next, a processing example of the image synthesizing unit 129 will be described with reference to FIG. FIG. 13 is a flowchart showing an example of processing by the image synthesizing unit 129.

The image synthesizing unit 129 determines the overlapping region in each of the first fundus image and the second fundus image whose positional relationship is determined by the positioning unit 127 (step S171). Here, the image synthesizing unit 129 may determine each center line in the determined overlapping region.
Next, the image synthesizing unit 129 derives a first pixel value obtained by multiplying the pixel value of the pixel included in the overlapping region of the first fundus image by the first coefficient for each pixel corresponding to the overlapping region pixel (step S172). ), The second pixel value is derived by multiplying the pixel value of the pixel included in the overlapping region of the second fundus image by the second coefficient (step S173). The image synthesizing unit 129 derives the sum of the first pixel value derived in step S172 and the second pixel value derived in step S173 to obtain the pixel value of the overlapping region pixel (step S174). Then, the image synthesizing unit 129 returns to step S172 and repeats the process until the process is executed for the pixels corresponding to all the overlapping region pixels (step S175).

When the pixel values of all the overlapping region pixels are derived, the image synthesizing unit 129 determines the pixel values of the derived overlapping region pixels, the pixel values of the pixels included in the non-overlapping region of the first fundus image, and the second fundus image. A panoramic fundus image is generated based on the pixel values of the pixels included in the non-overlapping region of (step S176).

The panoramic fundus image generation device 100 may generate a panoramic fundus image by connecting not only two images but also three or more fundus images by the processing as described above. For example, as shown in FIG. 14, five fundus images may be connected to generate a panoramic fundus image. FIG. 14 is a diagram showing an example in which five fundus images are connected. As shown in FIG. 14, the fundus images F102 and F103 are connected in the left-right direction of the fundus image F101, respectively. Further, the fundus image F104 is connected to the fundus image F102, and the fundus image F105 is connected to the fundus image F103.

According to the panoramic fundus image generator 100 of the embodiment described above, both the influence of the pixel value of the first fundus image and the influence of the pixel value of the second fundus image are considered as the pixel values of the overlapping region pixels. Can be done. In addition, the coefficient can be used to adjust the influence of the pixel value of each fundus image on the overlapping region pixels. As a result, it is possible to obtain a natural appearance while maintaining the necessary information of the panoramic image in which the images are connected to each other.

On the other hand, for example, when two fundus images are joined without overlapping, the joint portion has an unnatural appearance. In addition, when two fundus images are overlapped and spliced together in order to make the appearance natural, and a panoramic fundus image is generated by adopting the brighter pixel value in the corresponding pixel, the necessary information cannot be maintained. There was a case. This is because the blood vessel is darker than the other parts, and if the brighter pixel value is adopted, the image looks like a part of the blood vessel disappears. As described above, the image synthesizing unit 129 can solve such a problem by determining the pixel values of the overlapping region pixels by using the pixel values of both images in a predetermined ratio by using the coefficients.

[When three or more fundus images overlap in the same area]
In the above, the description focused on the scene where two fundus images are joined when generating a panoramic fundus image, but the following describes a method for synthesizing a fundus image in which three or more fundus images overlap in the same region. To do.

FIG. 15 is a diagram showing an example of a case where three or more fundus images overlap in the same region. As shown in FIG. 15, the fundus images F112 and F113 are connected in the left-right direction of the fundus image F111, and the fundus images F114 and F115 are connected in the vertical direction of the fundus image F111, respectively. In the fundus image F111, three or more fundus images are overlapped at each vertex. For example, the fundus image F111 also overlaps with the fundus image F114 in a part of the region overlapping with the fundus image F113.

In this way, when three or more fundus images overlap in the same region, the image synthesizing unit 129 cuts a part of the fundus image so that only two fundus images overlap in the overlapping region where the three or more fundus images overlap. To do. For example, the image synthesizing unit 129 cuts a region including one apex of the fundus image F113 and a region including one apex of the fundus image F114 at the cutting line L4. The cutting line L4 is, for example, a line connecting two points where the sides of the fundus image F113 and the fundus image F114 intersect.

An example of fundus images F113 and F114 after excision is shown in FIG. FIG. 16 is a diagram for explaining duplication of fundus images after excision. In the example shown in FIG. 16, a gap is formed in the cut portion of the fundus images F113 and F114 for easy understanding, but the image synthesizing unit 129 cuts a part of the image so that there is no gap. By doing so, only the two fundus images overlap in the overlapping region, so that the image synthesizing unit 129 can generate a panoramic fundus image by performing the same processing as described above.

Although the embodiments for carrying out the present invention have been described above using the embodiments, the present invention is not limited to these embodiments, and various modifications and substitutions are made without departing from the gist of the present invention. Can be added.

For example, the positioning unit 127 may determine the positional relationship between the two images based on the ratio of the overlapping region to the non-overlapping region. For example, the ratio of the overlapping region is preferably about half of the non-overlapping region. Further, the size of the overlapping region may be determined in advance by the number of pixels. By securing the size of the overlapping region to some extent, the joint portion can be smoothed.

For example, the image synthesizing unit 129 may determine the first coefficient and the second coefficient based on the length of the overlapping region R32 in the X-axis direction and the number of pixels. For example, the image synthesizing unit 129 determines the step (interval) of the coefficient in the X-axis direction so that the coefficient satisfies the predetermined relationship as described above.

Further, the image synthesizing unit 129 may determine the first coefficient and the second coefficient based on the average value of the brightness in the overlapping region of each fundus image. For example, the overlapping region of the first fundus image has a higher average brightness value (that is, when it is bright overall) than the overlapping region of the second fundus image, and the first coefficient described with reference to FIG. 10 The value obtained by adding 0.2 to may be used as the adjusted first coefficient, and the value obtained by subtracting 0.2 from the first coefficient described with reference to FIG. 10 may be used as the adjusted second coefficient. The image synthesizing unit 129 derives the pixel values of the overlapping region pixels by using the adjusted first coefficient and the adjusted second coefficient as described above.

The first coefficient and the second coefficient are not limited to the relationship shown in FIG. For example, the slope in the non-overlapping region is not limited to a constant value, and may be a slope that draws a gentle curve as shown in FIG.

1 ... fundus imaging system, 5 ... imaging device 5, 10 ... jaw stand, 15 ... base, 20 ... imaging unit, 100 ... panoramic fundus image generator, 111 ... communication unit, 113 ... input unit, 115 ... display unit 117 ... Storage unit, 121 ... Imaging control unit, 123 ... Acquisition unit, 125 ... Distortion correction unit, 127 ... Positioning unit, 129 ... Image composition unit, 171 ... Fundus image data, 173 ... Coefficient data

Claims (7)

An acquisition unit that acquires a plurality of fundus images of the same eye to be inspected, which are images of different ranges in the fundus of the eye to be inspected.
A distortion correction unit that performs distortion correction processing on a plurality of fundus images acquired by the acquisition unit, and
The first fundus image and the second fundus image, which are joined to each other among the plurality of fundus images processed to correct the distortion by the distortion correction unit, are based on the feature points in the respective fundus images. A positioning unit that determines the positional relationship when connecting the image and the second fundus image, and
As the pixel value of the target pixel included in the overlapping region of the first fundus image and the second fundus image whose positional relationship is determined by the positioning unit, the first pixel corresponding to the target pixel of the first fundus image The sum of the pixel value multiplied by the first coefficient and the pixel value of the second pixel corresponding to the target pixel of the second fundus image multiplied by the second coefficient is derived, and the derived pixel value is obtained. An image synthesizing unit that generates a panoramic fundus image by synthesizing the first fundus image and the second fundus image.
A panoramic fundus image generator equipped with. The image synthesizing unit
Regarding the center line in which the distance to the non-overlapping region of the first fundus image and the distance to the non-overlapping region of the second fundus image are equal in the overlapping region.
For the pixels included in the overlapping region closer to the non-overlapping region of the first fundus image than the center line, the first coefficient is made larger than the second coefficient.
For the pixels included in the overlapping region closer to the non-overlapping region of the second fundus image than the center line, the second coefficient is made larger than the first coefficient.
The panoramic fundus image generator according to claim 1. The image synthesizing unit
The closer the pixel is closer to the non-overlapping region of the first fundus image in the overlapping region, the larger the first coefficient and the smaller the second coefficient.
The closer to the non-overlapping region of the second fundus image in the overlapping region, the larger the second coefficient and the smaller the first coefficient.
The panoramic fundus image generator according to claim 1. The sum of the first coefficient and the second coefficient set for each pixel included in the overlapping region is 1.
The panoramic fundus image generator according to claim 1. The image synthesizing unit
When the panoramic fundus image is generated by superimposing a part of three or more fundus images included in the plurality of fundus images, the said so that only the two fundus images overlap in the region where the three or more fundus images overlap. After excising a part of one or more fundus images included in three or more fundus images, the pixel value of the target pixel is derived.
The panoramic fundus image generator according to claim 1. The positioning unit is
The feature amount of the branch point of the blood vessel included in the fundus image is acquired, and the positional relationship between the first fundus image and the second fundus image is determined based on the acquired feature amount.
The panoramic fundus image generator according to claim 1. On the computer
For the same eye to be inspected, a plurality of fundus images in which different ranges of the fundus of the inspected eye are imaged are acquired.
Distortion correction processing is applied to the acquired multiple fundus images.
With respect to the first fundus image and the second fundus image which are joined to each other among the plurality of fundus images subjected to the processing for correcting the distortion, the first fundus image and the second fundus image are based on the feature points in the respective fundus images. Determine the positional relationship when connecting to the fundus image,
The pixel value of the target pixel included in the overlapping region of the first fundus image and the second fundus image whose positional relationship is determined is the pixel value of the first pixel corresponding to the target pixel of the first fundus image. The sum of the value obtained by multiplying by one coefficient and the value obtained by multiplying the pixel value of the second pixel corresponding to the target pixel of the second fundus image by the second coefficient is derived.
Using the derived pixel values, a panoramic fundus image obtained by synthesizing the first fundus image and the second fundus image is generated.
program.