JP5027624B2 - Image processing method and image processing apparatus - Google Patents

Description

  The present invention relates to an image processing method and an image processing apparatus, in particular, a stereo image of a subject's eye fundus with a predetermined parallax via a fundus imaging optical system, and a three-dimensional shape measurement process of a fundus blood vessel using the obtained parallax images. The present invention relates to an image processing method and an image processing apparatus.

  2. Description of the Related Art Conventionally, for the purpose of diagnosing glaucoma, an image processing apparatus such as a fundus camera that captures a three-dimensional image of the fundus of the eye to be examined is known in order to grasp the shape of the fundus of the eye to be examined. For example, there is a device that stereo-photographs the fundus of the eye to be examined by moving the diaphragm in a single optical system of the fundus camera to different positions that are decentered from the optical axis to the left and right (or up and down). Known (Patent Document 1 below).

  Image processing for three-dimensional measurement of the fundus can be performed as follows. For example, pattern matching is performed on left and right (or upper and lower) images taken in stereo to search for corresponding image points, that is, corresponding points. If the corresponding points can be searched, each of the object points conjugated with the corresponding points using the principle of triangulation along the xy axes (two orthogonal axes parallel to the image plane) and the z axis (one axis parallel to the optical axis). Coordinates can be calculated. If a three-dimensional coordinate value of a sufficient number of object points is obtained, a three-dimensional model of the fundus of the eye to be examined can be created.

Further, the captured left and right parallax images are three-dimensionally displayed (3D display) using a 3D monitor or the like using various methods, that is, the captured left and right parallax images are independently displayed on both the left and right sides of the observer (examiner). By observing each of the eyes, the state of the photographed eye fundus can be observed stereoscopically.
JP-A-10-75932

  It is thought that it is useful for diagnosis to grasp not only the shape of the fundus but also the three-dimensional shape of the fundus blood vessel, but the conventional three-dimensional measurement of the fundus and the three-dimensional construction technique have specialized in the fundus blood vessel. Image processing, for example, special image processing that is useful for displaying and measuring characteristics such as the shape, positional relationship, and thickness of the fundus blood vessels, which are actually tubular organs, is more prominent. Not.

  When 3D construction is performed from a fundus image that is usually taken in stereo, the 3D construction is performed without distinguishing the vascular part and the retinal part in the fundus image, so the fundus blood vessels that should be in front of the retina It is constructed in a state where it is buried in the retina, and the result is far from the actual situation and is perceived by the examiner. Therefore, the fundus blood vessels are simply displayed as irregularities of different colors on the fundus, and it is not possible to clearly see a state in which a plurality of blood vessels crawling in front and back.

  For example, even though the fundus blood vessel is actually in front of the retina, simply displaying a stereoscopic image of the fundus image is displayed in a state in which the blood vessel is buried in the retina. However, it is perceived by the examiner as a video far from the actual situation. In the conventional 3D display of the fundus, the fundus blood vessels are simply displayed as irregularities of different colors on the fundus, and it is not possible to clearly see how the blood vessels are crawling in front and back. .

  Therefore, it is desired to image the three-dimensional structure of blood vessels in a form that can be understood by anyone. Usually, in a stereo photographing optical system with a parallax of about 7 degrees used for this kind of photographing, the back side of the blood vessels is photographed. Therefore, it is considered impossible in principle to perform volume rendering from only the captured stereo image to the side and back of the blood vessel.

  In view of the above problems, an object of the present invention is to provide an image processing method and an image processing apparatus capable of forming a precise three-dimensional model of a fundus blood vessel from a stereo photographed image of the fundus and outputting information useful for ophthalmologic diagnosis. There is.

In order to solve the above problems, in the present invention, the fundus oculi blood vessel is stereo-photographed with a predetermined parallax via the fundus imaging optical system, and the fundus blood vessel three-dimensional shape measurement process is performed using the obtained parallax images. In an image processing method and an image processing apparatus to be performed,
A process of extracting a blood vessel image corresponding to the imaging region of the fundus blood vessel from the left and right parallax images;
A process of measuring a blood vessel center and a blood vessel diameter in the vicinity of the blood vessel center in either the left or right blood vessel image;
Setting a region of interest having a block size determined according to a blood vessel diameter in the vicinity of the blood vessel center around the blood vessel center in the left or right blood vessel image;
In the other right or left blood vessel image, a search region including the region of interest and wider than the region of interest is set, and a corresponding point having a high correlation with the blood vessel center in the left or right blood vessel image is set in the search region. Searching from the right or left other blood vessel image;
Calculating the three-dimensional coordinate value of the blood vessel center from the blood vessel center in the left and right blood vessel images and its corresponding points;
Rendering a three-dimensional model of the fundus blood vessel based on each of the plurality of three-dimensional coordinate values of the blood vessel center obtained from the left and right blood vessel images and information on the blood vessel diameter in the vicinity of the blood vessel center;
The structure including was adopted.

  By adopting the above configuration, in the present invention, it is possible to render a three-dimensional fundus blood vessel model having a shape approximating a photographed blood vessel, and form a precise three-dimensional model of the fundus blood vessel from a stereo photographed image of the fundus, It is possible to obtain an excellent effect that information useful for ophthalmic diagnosis can be output.

  Hereinafter, as an example of the best mode for carrying out the present invention, the fundus of the eye to be examined is photographed in stereo via a stereo photographing optical system, and three-dimensional measurement processing and three-dimensional display are performed on the obtained photographed image data. An embodiment relating to the ophthalmic measuring apparatus to be performed will be described.

<Configuration of ophthalmic measurement device>
In the fundus camera 10 illustrated in FIG. 1 surrounded by an alternate long and short dash line, an observation lamp 11 that emits infrared illumination light is disposed at the center of curvature of the spherical mirror 12. Is incident on the total reflection mirror 17 through the condenser lens 14, the visible light strobe 15 which is a photographing light source, and the condenser lens 16.

  The illumination light reflected by the total reflection mirror 17 passes through the relay lens 22 through the illumination diaphragm 19, is reflected by the perforated total reflection mirror 23, and passes through the objective lens 24 to the anterior eye part (pupil) Ep of the eye E to be examined. Incident. The illumination stop 19 is disposed in a position substantially conjugate with the anterior segment Ep (pupil) of the eye to be examined in the illumination optical system.

  The reflected light from the fundus Er illuminated with the illumination light is transmitted through the objective lens 24, the aperture 23a of the perforated total reflection mirror 23, the aperture of the 2 aperture aperture stop (2 aperture aperture) 28, the focusing lens 35, and the imaging lens 36. Then, the light passes through the zoom lens 38a and enters the return mirror 39. When the return mirror 39 is in the position shown in the figure, the reflected light from the fundus is incident on a CCD (imaging means) 40 that is substantially conjugate with the fundus and is sensitive to infrared light, and the fundus is imaged by the CCD 40, and the return mirror. When 39 leaves the optical path, the reflected light from the fundus is incident on a CCD (imaging means) 41 that is at a position almost conjugate with the fundus and is sensitive to visible light, and the fundus is photographed by the CCD 41.

  As shown in FIGS. 2A and 2B, the photographing aperture 28 is provided with two rectangular openings 28a and 28b. The photographing aperture 28 is disposed such that the openings 28a and 28b are decentered with respect to the optical axis 26 and are symmetric with respect to the optical axis 26, and are substantially conjugate with the anterior eye portion (pupil) of the eye to be examined. .

  The positions and sizes of the openings 28a and 28b are set so that the opening 23a of the perforated total reflection mirror 23 includes the entire openings 28a and 28b.

  The apertures 28a and 28b of the photographing aperture 28 are opened or closed by shutter plates 29 and 30 moved along guides 28c and 28d, respectively.

  For this opening and closing, switching means comprising rotary solenoids 31 and 32 is provided. When the rotary solenoids 31 and 32 are not energized, the shutter plates 29 and 30 are at the position of FIG. , 28b are opened.

  On the other hand, when the rotary solenoids 31 and 32 are energized, the rods 31a and 32a of the rotary solenoids 31 and 32 rotate, and the other ends of the rods 31a and 32a are connected to pins 29a and 30a provided on the shutter plates 29 and 30, respectively. Engagement causes the shutter plates 29 and 30 to move inward to close the openings 28a and 28b.

  Note that reference numeral 33 in FIGS. 2A and 2B denotes an optical fiber for forming a working distance index (WD index).

  In the above configuration, the fundus image captured by the CCD 40 is input to the control calculation unit 60 configured by a CPU or the like, and the image is displayed on the monitor 62 as a moving image. The examiner looks at the image displayed on the monitor 62 and can perform operations such as alignment and focus adjustment as will be described later. In addition, a stereo monitor 63 is provided as a stereoscopic display, and the examiner can observe the fundus in a stereoscopic manner by observing the left and right images through the stereo monitor 63.

  The display method of the stereo monitor 63 includes various methods such that the polarization direction and display color are different between the left and right stereo images, and the left and right stereo images are independently observed through an observation scope that separates the left and right visual fields. However, the display method of the stereo monitor 63 according to the present embodiment is arbitrary, and any display device can be used as long as the left and right parallax images can be observed independently by the left and right eyes of the examiner. A stereo display system can be used.

  Further, the fundus illuminated by the strobe 15 when the shutter switch 66 is operated can be photographed as a still image by the CCD 41. This fundus image is temporarily stored in the high-speed memory 61 and recorded on a recording means realized by a low-speed hard disk (HDD) 64 as an external recording device via the control calculation unit 60, or a monitor 62 and a stereo monitor 63. Is displayed.

  Also, input means such as a keyboard 67 and a mouse 68 are provided, and various data can be input via these input means.

  Further, the fundus camera is provided with a control unit 65 composed of a CPU or the like. The control unit 65 is connected to the control calculation unit 60 and exchanges signals with each other. When the shutter switch 66 is operated, the control unit 65 returns. The mirror 39 is removed from the optical path, and the strobe 15 is caused to emit light with an appropriate amount of light. The control unit 65 controls insertion / removal of the variable magnification lens 38a and the like into the optical path, and controls driving of the rotary solenoids 31 and 32 described above.

  In addition, the fundus camera is provided with an operation unit (operation panel) 69, and the operation unit 69 can select a photographing mode. Further, a left / right eye detection unit 70 for detecting whether the subject eye to be photographed is the left eye or the right eye is provided, and information on whether the left eye or the right eye is detected by the left / right eye detection unit 70 is input to the control unit 65. The

  Here, an outline of stereo imaging of the fundus of the eye to be examined in the above configuration will be described.

  First, at the time of observation, the observation lamp 11 is turned on, and the illumination stop 19 is inserted into the optical path. The rotary solenoids 31 and 32 are driven to the positions shown in FIG. 2A, whereby the two openings 28a and 28b of the photographing aperture 28 are in the open positions. Reflected light from the fundus of the subject's eye irradiated with infrared light through the illumination diaphragm 19 is received by the observation CCD 40 through the openings 28a and 28b of the imaging diaphragm 28, and the fundus image is displayed on the monitor 62. .

  In this state, alignment and focusing with the eye to be examined are performed. At this time, alignment and focusing operations of the examiner are supported by a target projection system (not shown).

  When the shutter operation is completed after the alignment and focusing, the examiner presses the shutter switch 66. In response to this operation, the control unit 65 drives the rotary solenoid 31 to move the shutter plate 29 to the right, and closes the opening 28a on the left side of the photographing aperture 28. The strobe 15 emits light in synchronization with the operation of the shutter switch 66, and the return mirror 39 leaves the optical path, so that the light flux from the fundus illuminated by the strobe passes through the opening 28b of the photographing aperture 28 that is open. The light enters the image receiving surface of the CCD 41, and the first fundus image for stereoscopic viewing is captured as a still image by the CCD 41 and stored in the memory 61.

  Subsequently, the rotary solenoids 31 and 32 are controlled to move the shutter plates 29 and 30 to the left, and when the opening 28a is opened and the opening 28b is closed, the strobe 15 is caused to emit light again. At this time, the second fundus image for stereoscopic viewing that has passed through the opening 28 a is captured as a still image by the CCD 41 and stored in the memory 61.

  In this way, two parallax images photographed from two left and right viewpoints in succession by one shutter operation, that is, stereo photography, are the positions of the aperture of the photographing aperture that is open, the left position, and the right position. (Information indicating at least one of the left and right parallax images) is stored in the HDD 64 from the memory 61.

<Image processing in this embodiment>
The two parallax images stored as described above are read from the HDD 64 and displayed using the stereo monitor 63, and the examiner observes the corresponding one of the fundus images with the left and right eyes, thereby three-dimensionalizing the fundus. Can be seen. However, this stereoscopic observation image often depends on the subjectivity of the observer, and is insufficient for the desire to objectively grasp the blood vessel strike.

  Therefore, in this embodiment, a portion of the fundus blood vessel is extracted from the left and right parallax images obtained by photographing the fundus, and is rendered as three-dimensional stereoscopic image data based on the parallax image of the fundus blood vessel region. 3D image data can be displayed on the stereo monitor 63.

  In the rendering of the three-dimensional stereoscopic image data, the fundus blood vessel is assumed to be a tube having a substantially circular cross section, and the stereoscopic image data in which the diameter and coordinates in the three-dimensional space change depending on the part is generated. The basic color of the fundus blood vessel in the stereoscopic image data is determined based on, for example, color data obtained from the original blood vessel parallax image, or an appropriately determined color such as a red color system is used. Of course, the color used is not a single color, and shading or the like can be applied as appropriate so that the blood vessel rendered as a tube looks three-dimensional.

  Further, when displaying the three-dimensional stereoscopic image data on the stereo monitor 63, the fundus blood vessel rendered as a three-dimensional solid can be arbitrarily rotated around the XYZ three axes (three-dimensional display coordinate axes). . The display mode of such control is based on the assumption that the fundus blood vessel is a tube having a substantially circular cross section as described above, and the fundus blood vessel is modeled as stereoscopic image data in which the diameter and coordinates in a three-dimensional space change depending on the part. This is possible for the first time. For example, it is impossible to simply display a stereo image of the captured fundus parallax (or the extracted image of the blood vessel site).

  Hereinafter, rendering of the three-dimensional stereoscopic image data of the fundus blood vessel in the present embodiment will be described with reference to FIG. In particular, FIG. 7 shows the flow of rendering processing of the three-dimensional stereoscopic image data of the fundus blood vessel in the present embodiment, and hereinafter, the three-dimensional stereoscopic image of the fundus blood vessel along the control flow of FIG. Data rendering processing will be described. 7 can be stored in the ROM unit of the control calculation unit 60 or the HDD 64 as a program executed by the control calculation unit 60, for example.

  Reference numerals 301 and 302 in FIG. 3 indicate parallax images (stereo fundus images) of the fundus of the eye to be inspected and stored in the HDD 64 with the configuration in FIGS. 1 and 2.

  In this embodiment, these parallax images 301 and 302 are read from the HDD 64 (step S101 in FIG. 7), and only the blood vessel site is extracted as shown in FIG. 4 (step S102). Extraction of a blood vessel part can be performed by, for example, extracting a region having color characteristics peculiar to blood vessels, for example, lightness and saturation within a specific range from the parallax images 301 and 302.

  In the parallax images 401 and 402 of the fundus blood vessel in FIG. 4, reference numerals 403 and 404 indicate the extracted blood vessel sites.

  At this stage, the fundus blood vessel parallax images (blood vessel images) 401 and 402 of FIG. 4 can be binarized in order to facilitate the processing described later, particularly steps S104 to S106. For example, the image data is converted into binary image data in which the regions occupied by blood vessels (403 and 404 in FIG. 4) are “1” and the other regions are “0” (step S103). Each processing of the following steps S104 to S106 is performed on the binary blood vessel image data converted into the binary image thus obtained.

  Subsequently, using one of the left and right parallax images 401 and 402 (in the present embodiment, the left parallax image 402 is used hereinafter), as shown in FIG. (Diameter) is measured (step S104). In the configuration of FIG. 1, the blood vessel diameter can be measured by measuring the distance on the screen by photographing the fundus at a predetermined magnification at a predetermined imaging distance.

  The blood vessel diameter measurement is performed on the range of all blood vessel sites by sequentially searching the region of the blood vessel site 402 at a predetermined density on the screen. For example, when a blood vessel exists in a block obtained by dividing the entire screen with a grid of several dots (pixels), an algorithm such as measuring at least one blood vessel diameter can be used.

  Reference numeral 404 in FIG. 5 denotes a binarized blood vessel site, and its vicinity or the number shown above is a measured blood vessel diameter (for example, in mm). This blood vessel diameter measurement process is performed independently for the left and right images. At this stage, the measurement result may be displayed on the monitor 62 in the manner shown in FIG.

  When the blood vessel diameter is measured as shown in FIG. 5, if it is assumed that the blood vessel is a tube having a circular cross section, the center position (blood vessel point Bc) of the measured blood vessel diameter can also be specified. Therefore, when the measurement of the blood vessel diameter is completed, the central position of the blood vessel (blood vessel point Bc) is measured (step S105).

  The measurement processing of the blood vessel center (blood vessel point Bc) and the blood vessel diameter (Br: described later) is merely an example. Either the blood vessel center (blood vessel point Bc) or the blood vessel diameter (Br) may be measured first, and the blood vessel center does not necessarily have to be located on the straight line where the blood vessel diameter is measured. The blood vessel diameter in the vicinity of the specific range may be measured.

  Further, as described later, the three-dimensional coordinate value of each blood vessel center (blood vessel point Bc) is measured, and this three-dimensional coordinate value and the blood vessel diameter (Br) constitute an important element of the three-dimensional fundus blood vessel model. become. Therefore, the density at which each blood vessel center (blood vessel point Bc) is acquired from the screen is an important factor that determines the accuracy of the rendered three-dimensional fundus blood vessel model. The method for acquiring the blood vessel center (blood vessel point Bc) or blood vessel diameter (Br) from the region divided by the grid having a predetermined density as described above is a method that can easily control the acquisition density of the blood vessel center (blood vessel point Bc). It is an example.

  In the above description, the blood vessel center (blood vessel point Bc) or blood vessel diameter (Br) is acquired using only one of the binarized blood vessel images (left parallax image 402). As for, a method of obtaining both the left and right parallax images (401, 402) and taking an arithmetic average is also conceivable. Thereby, the improvement of the measurement accuracy of the blood vessel diameter can be expected. However, in that case, first, the blood vessel center (blood vessel point Bc) and the blood vessel diameter (Br) in the vicinity thereof are obtained using only one of the blood vessel images binarized as described above (left parallax image 402), The blood vessel center corresponding to the blood vessel center (blood vessel point Bc) is obtained in the other blood vessel image (right parallax image 401) by the processing described later, and the corresponding blood vessel center or the blood vessel diameter (Br) in the vicinity thereof is acquired. Processing is required, and the processing is somewhat complicated. In consideration of this point, the above method for acquiring the blood vessel center (blood vessel point Bc) and the blood vessel diameter (Br) in the vicinity thereof from only one parallax image based on the assumption that the blood vessel has a substantially circular cross-sectional tube shape. Can be executed at a high speed with a small processing load, and sufficient measurement accuracy can be obtained.

  FIG. 6 shows the right pupil position Pa and the left pupil position Pb corresponding to the apertures 28a and 28b of the imaging aperture 28, and the captured blood vessels B1, B2,... Bn, and the right and left parallax images on the screens a and b. , Corresponding points Ia1, Ib1, Ia2, Ib2,... Ian, Ibn. Note that FIG. 6 shows the screens a and b of the right and left parallax images moved to the conjugate plane on the object point side for the sake of convenience. Further, in order to facilitate understanding of the drawing, the right and left parallax images are displayed without matching the screens a and b.

  The blood vessel diameter Br in FIG. 6 corresponds to the blood vessel diameter (diameter) measured as shown in FIG. 5 (step S104 in FIG. 7). If this blood vessel diameter Br is known, the blood vessel center (blood vessel point) Bc (assuming the blood vessel is a tube) at this site can also be calculated. For example, the blood vessel center (blood vessel point) Bc can be obtained as the midpoint of the shortest distance between the blood vessel walls on the screen.

  In FIG. 6, although the blood vessel image is not shown, the corresponding points Ia1, Ib1, Ia2, Ib2 are used as the centers from the blood vessel images whose diameters are known on the screens a and b of the right and left parallax images. ... Ian and Ibn can be obtained. The corresponding points Ia1, Ib1, Ia2, Ib2,... Ian, Ibn correspond to the blood vessel centers (blood vessel points) Bc of the blood vessels B1, B2,.

  As described above, the blood vessel diameter Br and the blood vessel point Bc at the blood vessel center can be measured for the parallax images of the two blood vessels.

  Furthermore, rendering of three-dimensional stereoscopic image data of the fundus blood vessel from the blood vessel diameter Br at each blood vessel point Bc and the three-dimensional coordinate information obtained from the three-dimensional measurement of each blood vessel point Bc is performed as follows. (Steps S106 to S111 in FIG. 7).

  First, in steps S106 to S108 and S109, a parallax amount calculation and a depth amount (Z coordinate) calculation process between the left and right parallax images of a stereo fundus blood vessel are performed. This parallax amount calculation processing is as follows.

  First, in the left parallax image, a region of interest of vertical and horizontal n × n pixels (15 × 15 pixels in the example of FIG. 8) around the attention point (the blood vessel point Bc obtained as described above) for obtaining the parallax amount is set. (Step S106, FIGS. 8A and 8B). This process may be performed using the blood vessel image data binarized as described above.

  Here, the block size (n) of the region of interest of n × n pixels is determined according to the diameter of the blood vessel that is actually imaged. For example, it is about 5 times the blood vessel diameter Br at the blood vessel point Bc.

  Subsequently, a region having the highest correlation with the region of interest (FIG. 8C) of the left parallax image is searched from the right parallax image (FIG. 8D) (step S107). The processes after the search process (steps S107 to S110) are performed using the blood vessel image data before binarization.

  At that time, since it takes a calculation time to search from the entire image, the search area is limited to a range of (n + α) × (n + α) pixels (in the example of FIG. 8, α = 5 and 20 × 20 pixels). That is, the search is performed in the range (search area) of (n + α) × (n + α) pixels in the right parallax image with the point at which the parallax amount of the left parallax image is obtained as the center.

  When creating a three-dimensional model of the entire fundus, processing such as setting a fixed block size region of interest and a search region from the edge of the screen may be performed in advance. In this embodiment using a blood vessel image in which only a blood vessel part is extracted in stages, there is no blood vessel at the center of the region of interest and the search region, or different depth information is included in the left and right regions of interest and the search region. There is a possibility that a plurality of blood vessels may enter.

  Therefore, in the present embodiment, the selection of the region of interest and the search region (FIG. 8A) is performed centering on a certain point of interest in the left or right image, for example, the left image, that is, a certain blood vessel point Bc. And First, a region of interest of n × n pixels is set based on the blood vessel diameter as described above around the blood vessel point Bc in the left parallax image, and the coordinates corresponding to the blood vessel point Bc in the right parallax image are set as the center (n + α ) × (n + α) pixel search area is set, and it is confirmed whether or not only one blood vessel is included in the left region of interest and the right search region, and if two or more blood vessels are in the left region of interest. If it is included in the region and the right search region, the block size of the region of interest and the search region is corrected by correcting the value of n or further α, and then the search is performed in the search region.

  The attention points for obtaining the amount of parallax are all blood vessel points in the left parallax image, and the region having the highest correlation in the right parallax image is searched for each blood vessel point. At this time, instead of searching for each pixel, for example, the search may be performed with an interval of about 4 pixels.

  Then, the position shift of the region in the right parallax image having the highest correlation with the region of interest in the left parallax image is obtained as parallax (number of parallax pixels). In this way, one amount of parallax is calculated for each point of interest (blood vessel point) (step S108).

  By the above processing, it is possible to obtain the number of parallax pixels of a specific blood vessel point, that is, how many pixels the specific blood vessel point is captured in the left and right parallax images.

  If the conditions of the magnification (or shooting distance) of the optical system are recorded, the number of parallax pixels on a specific pixel or specific part on the shooting screen is converted into the actual amount of parallax on the retina (fundus) can do.

  If the actual amount of parallax on the retina (fundus) of a specific pixel or a specific part is obtained, the depth of the specific pixel or the specific part (or the coordinate value in the z-axis direction parallel to the optical axis) is obtained. It can be obtained as follows (step S109).

Here, in specific imaging, the parallax angle determined by the imaging distance and the positions of the stops 28a and 28b is 8 degrees, and the parallax amount on the fundus of a specific site is set to 0. 0 by the above calculation (S162). When calculated as 1 mm, the depth of a specific part (depth or coordinates in the z-axis direction) is calculated from this parallax using the principle of triangulation as shown in FIG.
Depth amount = (parallax amount / 2) / tan (parallax angle / 2) (Formula 1)
Is calculated to be 0.715 mm (relative amount from a part having a parallax amount of 0).

  In the case of a fundus camera, since the shooting distance is adjusted to a specific working distance by alignment before shooting, the parallax angle (8 degrees in the above example) determined by the positions of the stops 28a and 28b is assumed to be constant. A three-dimensional measurement process can be performed.

  As described above, the depth amount (Z coordinate) of a certain blood vessel point Bc can be obtained. Further, as is clear from the conjugate relationship of FIG. 6, since the right pupil position Pa, the left pupil position Pb, the focal length f of the eye to be examined, and the like are known, the correspondence on the screen of the left and right parallax images of a certain blood vessel point Bc is known. From the points Ibn and Ian, the X coordinate of the blood vessel point Bc and the Y coordinate can be obtained in the same manner (step S110).

  As described above, the three-dimensional coordinate values (x, y, z) indicating the positions of the many blood vessel points Bc constituting the photographed blood vessel and the blood vessel diameter Br at the positions can be calculated.

  As is clear from the configuration of rendering processing of the following blood vessel model, a blood vessel diameter for obtaining a blood vessel diameter and three-dimensional coordinates and a blood vessel point obtained on the same blood vessel adjacent to the blood vessel point are the diameter of the blood vessel. More than Br (preferably radius Br / 2) is determined so as not to be separated or within a distance range. For example, when the next blood vessel point is selected following a certain blood vessel point, the next blood vessel point is selected within a range within the blood vessel diameter Br (preferably the radius Br / 2) at that blood vessel point. In this way, the fundus blood vessel can be rendered as a continuous tube-shaped three-dimensional object. If blood vessel points are obtained only at a density that allows only spaced spheres to be arranged, the space between them may be complemented using a shape such as a cylinder.

  As described above, if all of the three-dimensional coordinate value (x, y, z) indicating the position of the blood vessel point Bc and the information regarding the blood vessel diameter Br at the position are prepared, a three-dimensional model of the fundus blood vessel can be created. Yes (step S111).

  In creating the three-dimensional model of the fundus blood vessel, for example, the blood vessel point Bc is arranged at the three-dimensional coordinates (x, y, z) in the three-dimensional model space, the blood vessel point Bc is the center, and the blood vessel diameter Br is the diameter. Place a sphere. If a sufficient number of the spheres can be arranged in the three-dimensional model space, a three-dimensional fundus blood vessel model having a shape approximating the photographed blood vessel can be rendered.

  Note that when rendering a blood vessel model by connecting two spheres separated from each other, a rule for linking is necessary. For this purpose, for example, when the distance between the target blood vessel point and the candidate blood vessel point is shorter than the blood vessel diameter and the difference in blood vessel diameter between the two blood vessel points is 50% or less, the connection is made. Blood vessel points that do not satisfy this condition are not considered as one blood vessel and are not connected. There are two types of blood vessel intersections (branch points): one blood vessel branching and two blood vessel intersections. Therefore, using the brightness information of the blood vessel image (refer to the lightness information of the blood vessel image before binarization (for example, FIG. 4)), the blood vessel points having greatly different brightness are not connected and the brightness is close. By using the condition that the objects are connected, it is possible to cope with the case where two blood vessels overlap.

  Once such a 3D fundus blood vessel model has been rendered, once the viewpoint information is set, a method such as ray tracing can be used to set the viewpoint information from any viewpoint without being limited to the shooting position performed by the optical system. The shape of the viewed three-dimensional fundus blood vessel model can be displayed on the monitor 62. In addition, by moving the viewpoint using an operation device such as a trackball or a mouse, the rendered three-dimensional fundus blood vessel model can be rotated or enlarged / reduced with an arbitrary rotation axis.

  Further, such display can be performed using the stereo monitor 63. In that case, if different viewpoint information is set for the image visually recognized by the examiner's left eye and the image visually recognized by the right eye, the fundus blood vessel model rendered as a stereo image is displayed using a method such as ray tracing. be able to. Of course, even in the case of this stereo display, the rendered three-dimensional fundus blood vessel model is rotated or enlarged / reduced by an arbitrary rotation axis by moving the viewpoint using an operation device such as a trackball or a mouse. Display control can be performed.

  In addition to displaying on the monitor 62, the stereo monitor 63, etc., the display information of the fundus blood vessel model can be recorded and output using a (color) printer or the like.

  The rendered fundus blood vessel model information includes, for example, a blood vessel diameter Br of each blood vessel point Bc, three-dimensional coordinate information of each blood vessel point Bc, or bibliographic information such as a subject's name and photographing date / time. The file can be stored in the HDD 64 in a file format such as a format, or can be transmitted to an external device (such as another ophthalmic measurement device or a computer) via a network interface or the like.

  The present invention provides an image processing such as a fundus camera that performs stereo imaging of the fundus of a subject's eye via a stereo imaging optical system and performs three-dimensional measurement processing or further display / recording processing of the captured image on the obtained captured image data. It can be implemented in the device.

It is explanatory drawing which showed the structure of the retinal camera as an image processing apparatus which employ | adopted this invention. (A) is a front view when the photographing aperture is viewed from the photographing side, and (b) is a cross-sectional view taken along X-X ′ of (a). It is explanatory drawing which showed the parallax image of the fundus | photograph image | photographed with the apparatus of FIG. It is explanatory drawing which showed the blood-vessel image extracted from the parallax image of FIG. It is explanatory drawing which showed the blood vessel diameter calculated from the blood vessel image of FIG. It is explanatory drawing which showed the conjugate relationship of the corresponding point of the blood vessel on the screen of a fundus blood vessel, and the right and left parallax images. FIG. 3 is a flowchart showing rendering processing of three-dimensional stereoscopic image data of a fundus blood vessel in the apparatus of FIG. 1. It is explanatory drawing which showed the mode of the parallax amount measurement process in the apparatus of FIG. It is explanatory drawing which showed the mode of depth amount (z-axis coordinate) measurement in the apparatus of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Fundus camera 24 Objective lens 28 Imaging stop 28a, 28b Aperture 35 Focusing lens 36 Imaging lens 62 Monitor 63 Stereo monitor Pa, Pb Pupil position B1, B2 ... Bn Blood vessel Bc Blood vessel point Br Blood vessel diameter

Claims (4)

In an image processing method of performing stereo imaging of a fundus of a subject's eye with a predetermined parallax via a fundus imaging optical system and performing three-dimensional shape measurement processing of a fundus blood vessel using the obtained left and right parallax images,
A process of extracting a blood vessel image corresponding to the imaging region of the fundus blood vessel from the left and right parallax images;
A process of measuring a blood vessel center and a blood vessel diameter in the vicinity of the blood vessel center in either the left or right blood vessel image;
Setting a region of interest having a block size determined according to a blood vessel diameter in the vicinity of the blood vessel center around the blood vessel center in the left or right blood vessel image;
In the other right or left blood vessel image, a search region including the region of interest and wider than the region of interest is set, and a corresponding point having a high correlation with the blood vessel center in the left or right blood vessel image is set in the search region. Searching from the right or left other blood vessel image;
Calculating the three-dimensional coordinate value of the blood vessel center from the blood vessel center in the left and right blood vessel images and its corresponding points;
Rendering a three-dimensional model of the fundus blood vessel based on each of the plurality of three-dimensional coordinate values of the blood vessel center obtained from the left and right blood vessel images and information on the blood vessel diameter in the vicinity of the blood vessel center;
An image processing method comprising:   The image processing method according to claim 1, wherein the size of the search area is determined so that only one blood vessel is included in the search area.   2. The image processing method according to claim 1, wherein a plurality of spheres having a diameter of a blood vessel diameter in the vicinity of the blood vessel center around the three-dimensional coordinate value of the blood vessel center are arranged in a three-dimensional model space. An image processing method characterized by rendering a dimensional model.   An image processing apparatus comprising: a fundus photographing optical system for performing the image processing method according to claim 1; and image processing means.

Images