JP5160823B2 - Image processing device - Google Patents

Description

  The present invention relates to an image processing device, and in particular, image processing for performing stereo imaging with a predetermined parallax through a fundus imaging optical system and performing three-dimensional shape measurement processing and three-dimensional display using the obtained parallax image. It relates to the device.

  2. Description of the Related Art Conventionally, for the purpose of diagnosing glaucoma, an image processing apparatus such as a fundus camera that captures a three-dimensional image of the fundus of the eye to be examined is known in order to grasp the shape of the fundus of the eye to be examined. For example, there is a device that stereo-photographs the fundus of the eye to be examined by moving the diaphragm in a single optical system of the fundus camera to different positions that are decentered from the optical axis to the left and right (or up and down). Known (Patent Document 1 below).

  Image processing for three-dimensional measurement of the fundus can be performed as follows. For example, pattern matching is performed on left and right (or upper and lower) images taken in stereo to search for corresponding image points, that is, corresponding points. If the corresponding points can be searched, each of the object points conjugated with the corresponding points using the principle of triangulation along the xy axes (two orthogonal axes parallel to the image plane) and the z axis (one axis parallel to the optical axis). Coordinates can be calculated. If a three-dimensional coordinate value of a sufficient number of object points is obtained, a three-dimensional model of the fundus of the eye to be examined can be created.

Further, the captured left and right parallax images are three-dimensionally displayed (3D display) using a 3D monitor or the like using various methods, that is, the captured left and right parallax images are independently displayed on both the left and right sides of the observer (examiner). By observing each of the eyes, the state of the photographed eye fundus can be observed stereoscopically.
JP-A-10-75932 JP 2002-34924 A

  Incidentally, an optical system of a photographing apparatus such as a fundus camera has image distortion. For example, when a grid pattern such as graph paper placed on a plane is captured, an image such as reference numeral 151 in FIG. 5 is captured. This distortion includes distortion that is caused by a manufacturing error that occurs in each optical system, in addition to distortion that is preliminarily designed in the optical system.

  If stereo imaging and three-dimensional measurement image processing as described above are performed with an optical system having such image distortion, the z-axis coordinates obtained on the P-P ′ line, for example, are square patterns placed on a plane. Even if it should be measured linearly as indicated by reference numeral 152, there is a possibility of deformation as indicated by reference numeral 153.

  Generally, in the three-dimensional measurement using a stereo image, it is necessary to correct the image distortion as described above, and there is a possibility that a three-dimensional shape different from the actual condition of the subject may be measured without correction. In view of this point, a configuration is used in which image data is affine transformed using parameters corresponding to the distortion of the optical system estimated in advance. In addition, in the case of fundus photography, there is also known a configuration in which a stereo photographed image is corrected on the assumption that the fundus is an ideal spherical surface in advance (Patent Document 2 described above).

  However, it is difficult to say that such conventional correction processing accurately reflects the characteristics of the optical system actually used for photographing, and there is a high possibility that some error will occur in the three-dimensional measurement result of the fundus. In clinical applications such as fundus retinal shape analysis and optic nerve head shape analysis, depending on the degree of error, there is a risk of misdiagnosis, and image processing that can accurately correct optical distortion from stereo images is required. Yes.

  For example, in the case of a grid pattern placed on the plane of FIG. 5, it is desirable that a three-dimensional shape that is distorted and measured as indicated by reference numeral 153 on the line P to P ′ can be accurately corrected as indicated by reference numeral 154. It is.

  In addition to correcting the shape data of the three-dimensional model of the fundus oculi to be examined so as to eliminate the influence of the distortion of the optical system in this way, it is considered that the photographed fundus image should also be corrected. For example, when the captured left and right parallax images are displayed in 3D using a 3D monitor or the like, and the examiner observes the captured fundus of the eye to be examined in a stereoscopic view, the fundus of the eye to be examined has a shape corresponding to the correction of the shape data. It is desirable that the 3D display image is also corrected so that the examiner can observe (perceive) the shape. In addition, even when the photographed fundus image is used for printing on paper or for recording that is a target of data recording on an HDD or the like, it is desirable that the image be corrected similarly. Needless to say.

  In view of the above problems, an object of the present invention is to provide a distortion of an optical system that is actually used for photographing in an image processing apparatus that performs stereo photographing of the fundus of the eye to be examined and performs three-dimensional measurement processing on the obtained photographed image data. Therefore, accurate 3D measurement processing can be performed on the fundus image, or an accurate fundus image corresponding to the 3D measurement result can be provided, and the examiner can accurately evaluate the three-dimensional shape of the fundus There is in doing so.

In order to solve the above problems, in the present invention, in an image processing apparatus that performs stereo imaging of a subject's eye fundus with a predetermined parallax via a fundus imaging optical system, and performs a three-dimensional shape measurement process using the obtained parallax image , Subjecting a subject having a predetermined shape in stereo with the predetermined parallax via the fundus imaging optical system, measuring a three-dimensional shape of the predetermined portion of the subject based on image data of the obtained parallax image, Calibration data represented by the amount of parallax to be corrected in the three-dimensional shape measurement process is created, corresponding to the difference between the actual three-dimensional shape and the three-dimensional shape of the predetermined part measured based on the image data. , Correction of shape distortion is performed on the three-dimensional shape data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data, and the eye fundus of the eye to be examined is used by using the calibration data. By adding the shape distortion correction on the parallax image data obtained by the stereo imaging employs a configuration that creates a corrected image data.
Alternatively, in an image processing apparatus that performs stereo imaging with a predetermined parallax through a fundus imaging optical system and performs a three-dimensional shape measurement process using the obtained parallax image, the fundus imaging optics Stereo shooting with the predetermined parallax through the system, the three-dimensional shape of the predetermined part of the subject is measured based on the image data of the obtained parallax image, the actual three-dimensional shape of the predetermined part and the image data A calibration data corresponding to a difference from the three-dimensional shape of the predetermined portion measured based on the three-dimensional shape and expressed by a depth amount to be corrected in the three-dimensional shape measurement process is created, and the eye to be examined is used using the calibration data The shape distortion correction is performed on the three-dimensional shape data obtained by stereo imaging of the fundus, and the parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data. Adopting a structure that creates a corrected image data by adding the shape distortion correction on data.
Alternatively, in an image processing apparatus that performs stereo imaging with a predetermined parallax through a fundus imaging optical system and performs a three-dimensional shape measurement process using the obtained parallax image, the fundus imaging optics Stereo shooting with the predetermined parallax through the system, the three-dimensional shape of the predetermined part of the subject is measured based on the image data of the obtained parallax image, the actual three-dimensional shape of the predetermined part and the image data The table format corresponds to a difference from the three-dimensional shape of the predetermined part measured based on the abscissa of the three-dimensional shape data or a specific part in a parallax image that is stereo-photographed by the fundus photographing optical system. When the calibration data created in step 1 is created and shape distortion correction is performed on the three-dimensional shape data obtained by stereo imaging of the fundus oculi to be examined using the calibration data. The adopted a structure that creates a corrected image data by adding the shape distortion correction on the parallax image data obtained by the stereo imaging of the fundus by using the calibration data.

  In particular, as the subject, a model eye that simulates the shape and optical characteristics of the human eye and has a simulated fundus surface with a predetermined shading pattern as the predetermined portion can be used.

  Further, left and right corrected image data is created by applying the shape distortion correction to the left and right parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data, and three-dimensional correction is performed using a stereo monitor. It can be set as the structure which performs a display.

  Further, the calibration data is expressed by a depth amount to be corrected in the three-dimensional shape measurement process, and the shape of the parallax image data obtained by stereo imaging of the fundus of the eye to be examined by geometric calculation using the calibration data. It can be set as the structure which produces correction | amendment image data by adding distortion correction.

  According to the above configuration, in the image processing apparatus that performs the three-dimensional measurement process on the obtained fundus image data in stereo through the stereo imaging optical system, the optical system actually used for imaging is used. Distortion can be corrected and accurate three-dimensional measurement processing can be performed on the fundus image, or an accurate fundus image corresponding to the three-dimensional measurement result can be provided, so that the three-dimensional shape of the fundus can be accurately evaluated, There is an excellent effect that problems such as misdiagnosis can be avoided.

  Further, by using a model eye that simulates the shape and optical structure of the eye to be examined, calibration measurement and calculation can be easily performed without changing other imaging conditions and calculation conditions.

  Further, by creating left and right corrected image data corresponding to the shape distortion correction and performing stereoscopic display using a stereo monitor, the examiner can easily grasp the state of the fundus of the subject through stereoscopic vision.

  Further, the shape distortion correction is applied to the parallax image data obtained by stereo imaging of the fundus of the eye to be examined by geometric calculation using the calibration data expressed by the depth amount to be corrected in the three-dimensional shape measurement process. Corrected image data can be created, and the corrected image data can be used for display, recording, and the like.

  Hereinafter, as an example of the best mode for carrying out the present invention, the fundus of the eye to be examined is photographed in stereo via a stereo photographing optical system, and three-dimensional measurement processing and three-dimensional display are performed on the obtained photographed image data. An embodiment relating to the ophthalmic measuring apparatus to be performed will be described.

  In the fundus camera 10 illustrated in FIG. 1 surrounded by a one-dot chain line, an observation lamp 11 that emits infrared light and visible illumination light is disposed at the center of curvature of the spherical mirror 12. The light from the mirror 12 is incident on the total reflection mirror 17 through the condenser lens 14, the strobe 15 that is a photographing light source, and the condenser lens 16.

  The illumination light reflected by the total reflection mirror 17 passes through the relay lens 22 through the illumination diaphragm 19, is reflected by the perforated total reflection mirror 23, and passes through the objective lens 24 to the anterior eye part (pupil) Ep of the eye E to be examined. Incident. The illumination stop 19 is disposed in a position substantially conjugate with the anterior segment Ep (pupil) of the eye to be examined in the illumination optical system.

  The reflected light from the fundus Er illuminated with the illumination light is transmitted through the objective lens 24, the aperture 23a of the perforated total reflection mirror 23, the aperture of the 2 aperture aperture stop (2 aperture aperture) 28, the focusing lens 35, and the imaging lens 36. Then, the light passes through the zoom lens 38a and enters the return mirror 39. When the return mirror 39 is in the position shown in the figure, the reflected light from the fundus is incident on a CCD (imaging means) 40 that is substantially conjugate with the fundus and is sensitive to infrared light, and the fundus is imaged by the CCD 40, and the return mirror. When 39 leaves the optical path, the reflected light from the fundus is incident on a CCD (imaging means) 41 that is at a position almost conjugate with the fundus and is sensitive to visible light, and the fundus is photographed by the CCD 41.

  The photographing aperture 28 is provided with two rectangular openings 28a and 28b as shown in FIG. The photographing aperture 28 is disposed such that the openings 28a and 28b are decentered with respect to the optical axis 26 and are symmetric with respect to the optical axis 26, and are substantially conjugate with the anterior eye portion (pupil) of the eye to be examined. .

  The positions and sizes of the openings 28a and 28b are set so that the opening 23a of the perforated total reflection mirror 23 includes the entire openings 28a and 28b.

  The apertures 28a and 28b of the photographing aperture 28 are opened or closed by shutter plates 29 and 30 moved along guides 28c and 28d, respectively.

  For this opening and closing, switching means comprising rotary solenoids 31 and 32 is provided. When the rotary solenoids 31 and 32 are not energized, the shutter plates 29 and 30 are at the position of FIG. , 28b are opened.

  On the other hand, when the rotary solenoids 31 and 32 are energized, the rods 31a and 32a of the rotary solenoids 31 and 32 rotate, and the other ends of the rods 31a and 32a are connected to pins 29a and 30a provided on the shutter plates 29 and 30, respectively. Engagement causes the shutter plates 29 and 30 to move inward to close the openings 28a and 28b.

  The fundus image captured by the CCD 40 is input to a control calculation unit 60 constituted by a CPU or the like, and the image is displayed on the monitor 62 as a moving image. The examiner looks at the image displayed on the monitor 62 and can perform operations such as alignment and focus adjustment as will be described later. In addition, a stereo monitor 63 is provided as a stereoscopic display, and the examiner can observe the fundus in a stereoscopic manner by observing the left and right images through the stereo monitor 63.

  The display method of the stereo monitor 63 includes various methods such that the polarization direction and display color are different between the left and right stereo images, and the left and right stereo images are independently observed through an observation scope that separates the left and right visual fields. However, the display method of the stereo monitor 63 according to the present embodiment is arbitrary, and any display device can be used as long as the left and right parallax images can be observed independently by the left and right eyes of the examiner. A stereo display system can be used.

  Further, the fundus illuminated by the strobe 15 when the shutter switch 66 is operated can be photographed as a still image by the CCD 41. This fundus image is temporarily stored in the high-speed memory 61 and recorded on a recording means realized by a low-speed hard disk (HDD) 64 as an external recording device via the control calculation unit 60, or a monitor 62 and a stereo monitor 63. Is displayed.

  Also, input means such as a keyboard 67 and a mouse 68 are provided, and various data can be input via these input means.

  Further, the fundus camera is provided with a control unit 65 composed of a CPU or the like. The control unit 65 is connected to the control calculation unit 60 and exchanges signals with each other. When the shutter switch 66 is operated, the control unit 65 returns. The mirror 39 is removed from the optical path, and the strobe 15 is caused to emit light with an appropriate amount of light. The control unit 65 controls insertion / removal of the variable magnification lens 38a and the like into the optical path, and controls driving of the rotary solenoids 31 and 32 described above.

  In addition, the fundus camera is provided with an operation unit (operation panel) 69, and the operation unit 69 can select a photographing mode. Further, a left / right eye detection unit 70 for detecting whether the subject eye to be photographed is the left eye or the right eye is provided, and information on whether the left eye or the right eye is detected by the left / right eye detection unit 70 is input to the control unit 65. The

  Here, an outline of stereo imaging of the fundus of the eye to be examined in the above configuration will be described.

  First, at the time of observation, the observation lamp 11 is turned on, and the illumination stop 19 is inserted into the optical path. The rotary solenoids 31 and 32 are driven to the positions shown in FIG. 2A, whereby the two openings 28a and 28b of the photographing aperture 28 are in the open positions. Reflected light from the fundus of the subject's eye irradiated with non-infrared light through the illumination diaphragm 19 is received by the observation CCD 40 through the openings 28a and 28b of the imaging diaphragm 28, and the fundus image is displayed on the monitor 62. The

  In this state, alignment and focusing with the eye to be examined are performed. At this time, alignment and focusing operations of the examiner are supported by a target projection system (not shown).

  When the shutter operation is completed after the alignment and focusing, the examiner presses the shutter switch 66. In response to this operation, the control unit 65 drives the rotary solenoid 31 to move the shutter plate 29 to the right, and closes the opening 28a on the left side of the photographing aperture 28. The strobe 15 emits light in synchronization with the operation of the shutter switch 66, and the return mirror 39 leaves the optical path, so that the light flux from the fundus illuminated by the strobe passes through the opening 28b of the photographing aperture 28 that is open. The light enters the image receiving surface of the CCD 41, and the first fundus image for stereoscopic viewing is captured as a still image by the CCD 41 and stored in the memory 61.

  Subsequently, the rotary solenoids 31 and 32 are controlled to move the shutter plates 29 and 30 to the left, and when the opening 28a is opened and the opening 28b is closed, the strobe 15 is caused to emit light again. At this time, the second fundus image for stereoscopic viewing that has passed through the opening 28 a is captured as a still image by the CCD 41 and stored in the memory 61.

  In this way, two parallax images photographed from two left and right viewpoints in succession by one shutter operation, that is, stereo photography, are the positions of the aperture of the photographing aperture that is open, the left position, and the right position. And the like are added to the HDD 64 from the memory 61. The two parallax images stored in this way are read from the HDD 64 and displayed using the stereo monitor 63, and the examiner observes one corresponding fundus image with the left and right eyes. The fundus can be viewed stereoscopically.

  Here, before describing the calibration process according to the present invention, an outline of a process for three-dimensionally measuring the three-dimensional shape of the fundus of the eye to be examined from the left and right parallax images taken in stereo as described above will be described.

  FIG. 6 shows an outline of this fundus three-dimensional measurement process. In step S161 in FIG. 6, the fundus of the eye to be examined is photographed in stereo as described above.

  Subsequently, a parallax amount calculation process between the left and right parallax images taken in stereo is performed in step S162. This parallax amount calculation processing is as follows.

  First, a region of interest of 15 × 15 pixels is set in the left parallax image with the focus point for obtaining the parallax amount as the center (FIGS. 7A and 7B). Then, a region having the highest correlation with the region of interest (15 × 15 pixels: FIG. 7C) of the left parallax image is searched from the right parallax image (FIG. 7D).

  At this time, since it takes a calculation time to search from the entire image, the search area is limited to 20 × 20 pixels. That is, the search is performed in the range of 20 × 20 pixels in the right parallax image with the point for obtaining the parallax amount of the left parallax image as the center.

  The attention points for obtaining the amount of parallax are all points in the left parallax image, and an area having the highest correlation in the right parallax image is searched for each point, but instead of searching for each pixel, for example, four pixels The search may be performed with a certain interval.

  Then, the position shift of the region in the right parallax image having the highest correlation with the region of interest in the left parallax image is obtained as parallax (number of parallax pixels). In this way, one amount of parallax is calculated for each attention point.

  Through the above processing, it is possible to obtain the number of parallax pixels of a specific pixel or a specific part, that is, how many pixels the specific pixel or the specific part is imaged in the left and right parallax images.

  If the conditions of the magnification (or shooting distance) of the optical system are recorded, the number of parallax pixels on a specific pixel or specific part on the shooting screen is converted into the actual amount of parallax on the retina (fundus) can do.

  If the actual amount of parallax on the retina (fundus) of a specific pixel or a specific part is obtained, the depth of the specific pixel or the specific part (or the coordinate value in the z-axis direction parallel to the optical axis) is obtained. It can be obtained as follows (S163 in FIG. 6).

Here, in specific imaging, the parallax angle determined by the imaging distance and the positions of the stops 28a and 28b is 8 degrees, and the parallax amount on the fundus of a specific site is set to 0. 0 by the above calculation (S162). When calculated as 1 mm, the depth of the specific part (depth or coordinates in the z-axis direction) is calculated from this parallax amount using the principle of triangulation as shown in FIG.
Depth amount = (parallax amount / 2) / tan (parallax angle / 2) (Formula 1)
Is calculated to be 0.715 mm (relative amount from a part having a parallax amount of 0).

  In the case of a fundus camera, since the shooting distance is adjusted to a specific working distance by alignment before shooting, the parallax angle (8 degrees in the above example) determined by the positions of the stops 28a and 28b is assumed to be constant. A three-dimensional measurement process can be performed.

  Now, the imaging optical system of the fundus camera in FIG. 1, that is, the objective lens 24... Variable magnification lens 38a has the image distortion as shown in FIG. to be influenced.

  Therefore, in this embodiment, a subject having a predetermined shape, in particular, a model eye 300 as shown in FIG. 3 is photographed in stereo, and an inherent shape distortion caused by the optical system of the fundus camera is calculated in advance, and this distortion is dealt with. That is, calibration data that can correct this distortion is acquired.

  Based on this calibration data, the depth amount (depth or coordinates in the z-axis direction) as the three-dimensional measurement data measured as shown in FIGS. 6 to 8 is corrected. Further, using the same distortion data, the stereo image that the examiner observes in stereoscopic view is corrected on the stereo monitor 63.

  The model eye 300 is created in a shape that simulates (simulates) the optical characteristics of the human eye as shown in FIG. For example, the model eye 300 is created by a technique such as cutting and polishing optical glass having a refractive index n = 1.52. The curvature of the cornea simulated surface (incident surface) 302 is a spherical surface of about R7.7 mm, and the curvature of the simulated retina surface 301 corresponding to the retina is a spherical surface of about R15 mm. The distance between these two surfaces is about 22.6 mm, for example.

  In the later-described calibration imaging, a fundus simulating surface that simulates the fundus of the model eye 300 is imaged as a predetermined part for acquiring calibration data. More specifically, this fundus simulating surface is configured as a surface 301 that simulates the retina in the model eye 300 of this embodiment. The surface 301 simulating the retina is processed into a three-sided (semi-transparent) surface, and a certain shade pattern (for example, a square pattern or the like) is given by a technique such as engraving + coloring or printing. It should be noted that when creating calibration data, which will be described later, a specific pattern may be set to a specific color so as to facilitate processing.

  The model eye 300 is attached to and detached from the chin rest 310 of the fundus camera of FIG. 1 through a jig 311 calibrated to a predetermined shape as a mounting member (for example, using a structure such as latch coupling or screwing). It is convenient to configure the optical system so that it is positioned at substantially the same position as the (ideal) position of the eye E when mounted.

  Using the above model eye 300, calibration data for correcting image data used for three-dimensional measurement can be acquired as follows at any timing before or after shipment of the fundus camera of FIG. it can.

  First, the model eye 300 is positioned at substantially the same position as the (ideal) position of the eye E using the jig 311 as described above, and right and left parallax images are captured as described above.

  The photographing distance and the magnification are also controlled so as to be constant values for the calibration process. For this reason, it is convenient to provide a notch in the position of the optical system, the variable magnification ring (dial), etc. so that the photographing distance and magnification for calibration processing can be easily selected.

  In this embodiment, the calibration data can be obtained by photographing the model eye 300 and calculating the three-dimensional shape as described above. When photographing is performed under different photographing magnification conditions by the magnification lens 38a of the fundus camera, calibration data is obtained for each magnification condition.

  As shown in FIG. 10A, the calibration data includes a three-dimensional shape (curve of depth information (z coordinate)) 1002 obtained by photographing the model eye 300, and a shape 1001 of the actual model eye 300. This is equivalent to the difference 1003. A specific calculation example of the amount corresponding to the difference 1003 will be described later.

  In FIG. 10A, the three-dimensional shape 1002 obtained by photographing the model eye 300 is affected by image distortion, and the fundus shape of the model eye 300 must be obtained as 1001 originally. . Here, for ease of explanation, the fundus shape 1001 of the model eye 300 is illustrated in a substantially flat shape.

  The calibration data of the present embodiment is recorded with the expression of the difference 1003 in the depth amount (coordinate in the z-axis direction) as shown in FIG. 10A, or by the measurement of the above three-dimensional shape. It can be expressed by the difference in the amount of parallax used (if the amount of parallax is determined, the depth amount is uniquely determined as apparent from the above equation 1).

  The calibration data can be used to correct the three-dimensional shape of the fundus obtained by actually photographing the eye to be examined to obtain the three-dimensional shape of the fundus of the actual eye to be examined.

  For example, as shown in FIG. 10B, when the shape 1004 (depth information (z coordinate) curve) of the fundus of the eye to be examined is measured, the calibration data expressing the difference 1003 is applied to act as shown in FIG. The fundus shape 1005 of the eye to be examined can be corrected as in c) (the fundus shape 1005 of the eye to be examined is shown in a substantially flat shape for the sake of easy explanation).

  The calibration data obtained as described above can be recorded in an HDD or a non-volatile memory in a table format.

  For example, in the case of calibration data representing a depth amount (coordinate in the z-axis direction), since it is applied after obtaining the captured image depth amount (coordinate in the z-axis direction) by the above three-dimensional measurement process, the three-dimensional The shape data is recorded in association with the abscissa (xy coordinate or polar coordinate expression from the image center (optical axis)). Then, if calibration data is read from the table using the abscissa of the predetermined part in the fundus shape data and applied, the corrected depth amount (coordinate in the z-axis direction) of the part can be obtained.

  In addition, when the calibration data is a representation of the amount of parallax used in the measurement of the three-dimensional shape, the parallax amount measurement processing is performed with attention points at specific intervals in the parallax image (about 4 pixels in the above example). Therefore, the calibration data representing the parallax amount may be recorded in a table format for each point of interest. Then, at the stage of the parallax amount calculation process (step S162) in FIG. 6, based on the attention point (point number or pixel address) being processed, the table data is referred to, the calibration data representing the parallax amount expression is read, and the depth amount What is necessary is just to apply to the amount of parallax used for calculation (step S163).

  By using the calibration data having the table configuration as described above, it is possible to perform image distortion correction at a high speed with an extremely small calculation load.

  FIG. 9 shows a flow of the above-described shape distortion correction and actual fundus stereo photography.

  Steps S901 and S902 in FIG. 9 show calibration photographing and calibration data acquisition processing performed at any timing before or after shipment of the fundus camera in FIG.

  In step S901, the model eye 300 is photographed in stereo as described above, and in step S902, calibration data is obtained as described above.

  That is, in step S902, the amount of parallax is calculated for each point of interest from the left and right parallax images of the model eye 300, and the shape data of the model eye 300 (1002 in FIG. 10A) is obtained. Since the fundus shape of the model eye 300 (1001 in FIG. 10A) is known, calibration data can be acquired as the difference between them (representation by the depth amount or parallax amount). The acquired calibration data is recorded in the HDD or nonvolatile memory as described above.

  Thereafter, actual stereo imaging of the fundus is performed after step S161. Steps S161, S162 ′, and S163 ′ in FIG. 9 are substantially the same as the stereo shooting (S161), parallax pixel number measurement (S162), and depth amount calculation (S163) in FIG. In S162 ′ or S163 ′, correction processing is performed using the calibration data recorded in the table format as described above.

  That is, when the calibration data is expressed by the amount of parallax as described above, correction is performed at the stage of measuring the number of parallax pixels in step S162. Alternatively, when the calibration data is expressed by a depth amount, correction is performed at the depth amount calculation stage in step S163.

  In this way, it is possible to accurately measure the number of parallax pixels and calculate the depth amount in steps S162 and S163 after applying the shape distortion correction inherent to the optical system to the left and right captured images.

  Furthermore, using the same calibration data, in step S170, a captured image, for example, a stereo image that the examiner observes in stereoscopic view on the stereo monitor 63 can be corrected. The 3D display by the stereo monitor 63 is not necessarily performed sequentially following the calibration processing (steps S162 ′ and S163 ′) as shown in FIG. 9, but using calibration data prepared as table data, for example, calibration processing and Can be executed at the same time.

  11 and 12 show an outline of processing for calibrating a 3D display image by the stereo monitor 63 in step S170. Here, for ease of explanation, the relationship between the direction of depth direction calibration and the z-axis direction is opposite to that in FIG.

  FIG. 11 and FIG. 12 show the imaging relationship when the model eye 300 is photographed with the optical system of FIG. In FIGS. 11 and 12, the a-plane and the b-plane correspond to the right and left parallax image planes captured by the CCD 41, respectively, and are illustrated at conjugate positions on the object point side. Here, in order to facilitate understanding of the drawing, the a and b surfaces that are actually coincident with each other on the straight line are slightly shifted up and down in the drawing.

  The c-plane is a model eye construction plane, that is, a calibration reference plane corresponding to the original shape of the model eye, which is a three-dimensional shape of the model eye constructed from the amount of parallax photographed by the optical system of FIG. A three-dimensional shape to be corrected to the d-plane.

  Here, in order to facilitate the following description, the d-plane, that is, the original model eye shape is a plane, and the c-plane and d-plane are correction reference points on the optical axis (coincident with the z-axis). Assume that they match at C0.

  That is, at the correction reference point C0, the correction amount in each of the xyz axial directions is 0. In the following, the coordinates on the c plane other than the correction reference point C0 are moved to the d plane (for example, the following points (Xc, Zc) This shows how the pixels in the a and b planes should be translated in accordance with the movement from the point to the point (Xd, Zd).

  It is assumed that the coordinates of the image forming point on the a-plane (right parallax image plane) of the correction reference point C0 are (a0, f).

  Points (Pa, 0) and (Pb, 0) correspond to the conjugate positions of the stops 28a and 28b that coincide with the anterior eye part (pupil) of the eye to be examined at the right pupil position and the left pupil position, respectively. In addition, the distance between the left and right pupils and the above-described a-plane and b-plane (right and left parallax image planes) matches the eyeball focal length f of the eye to be inspected by the setting of the photographing distance.

  Here, the xz plane will be used to explain the correction of the a-plane and b-plane images in the x-axis direction. However, the following explanation is valid for the yz plane if x is replaced with y. Correction regarding the y-axis direction of the image of the surface and b-plane can also be performed.

  Here, when the model eye 300 is photographed with the optical system of FIG. 1, the c-plane (model eye construction surface) of FIG. 11 is measured as the above-described depth amount. This shape is calculated as a shape in which the peripheral portion is far (dented) reflecting the distortion of the optical system in FIG.

  That is, from the corresponding points photographed as (Xa, f) and (Xb, f) on the a-plane and b-plane (right and left parallax image planes), the point (Xc) on the c-plane (model eye construction plane) , Zc).

  As described above, for example, if the depth amount difference Δz is recorded as the table data, the process of correcting the point (Xc, Zc) to the point (Xd, Zd) becomes possible.

Here, from the calculation of the similar triangle, the x coordinate value Xc and the z coordinate value Zc of the point (Xc, Zc) are: Xc = (Pa · Xb−Pb · Xa) / ((Pa−Pb) − (Xa− Xb)) (Formula 2)
Zc = f · (Pa−Pb) / ((Pa−Pb) − (Xa−Xb)) (Formula 3)
Next, this point (Xc, Zc) is shifted in the z-axis direction to calculate a point (Xd, Zd) on the d-plane (calibration reference plane). The x and y coordinate values Xd and Zd of the point (Xd, Zd) are respectively Xd = Xc (Formula 4)
Zd = f · Pa / (Pa−a0) (Formula 5)
Can be associated.

Therefore, the depth difference Δz is expressed as: Δz = Zd−Zc (Formula 6)
Can be obtained as Here, Zd and Zc already use values obtained by Equation 5 and Equation 3. This Δz can also be stored in the table as the calibration data described above.

  Furthermore, in order to correct the 3D display image by the stereo monitor 63, the positions of (Xa, f) and (Xb, f) on the a and b planes (right and left parallax image planes) shown in FIG. As shown in FIG. 12, correction must be made in the a and b planes. That is, in order to make the point (Xd, Zd) on the d-plane which is the calibration reference plane as a new construction point, the corresponding points to be photographed on the a-plane and the b-plane are (Xa ′, f) in FIG. , (Xb ′, f).

  The points (Xa ′, f) and (Xb ′, f) in FIG. 12 are photographed on the a-plane and b-plane (right and left parallax image planes) by photographing the corrected point (Xd, Zd). The corresponding points are shown.

Here, the x coordinates Xa ′ and Xb ′ of the points (Xa ′, f) and (Xb ′, f) are
Xa '= (Zd.Pa-f.Pa-f.Xd) / (Zd-2f) (Formula 7)
Xb ′ = (Zd · Pb−f · Pb−f · Xd) / (Zd−2f) (Formula 8)
It can be asked.

  As described above, if the point (Xc, Zc) is corrected to the point (Xd, Zd) (Xc = Xd), that is, if correction in the depth direction (z-axis direction) can be performed, a in the x-axis direction The positions of corresponding points in the right and left parallax images of the plane and b plane can be corrected. The correction of the position of the corresponding point can actually be performed by moving a pixel in the right and left parallax images.

  11 and 12 show the correction (calibration) in the x-axis direction in the xz plane, the correction (calibration) in the y direction in the yz plane can be performed in the same manner. In this case, illustration of the correction (calibration) in the y-axis direction in the yz plane is omitted, but in this case, the processing shown in FIG. 11 and FIG. Is equivalent to the correction (calibration) in the y direction in the yz plane.

  In this way, it is possible to obtain corrected image data in which the positions of corresponding points (pixels) in the right and left parallax images on the a and b planes are corrected in the xy plane. The obtained corrected image data can be displayed, recorded and output (for example, printed), or transferred as a database record to an external storage device or another processing device (such as a computer).

  For example, if the left and right corrected image data corrected as described above is displayed in 3D on the stereo monitor 63, the appearance of the 3D display image is corrected. That is, on the 3D display via the stereo monitor 63, before correction, the shape perceived by the examiner as the c-plane in FIGS. 11 and 12 can be calibrated so as to be perceived as the d-plane.

  In the description of FIG. 11 and FIG. 12, an example of calculation in the case where the d-plane is a plane that coincides with (is approximated to) the c-plane at the correction reference point C0 is shown. In this case, the correction in the depth direction (calculation of Δz) from the point (Xc, Zc) to (Xd, Zd) and the correction of the parallax image can be performed by a simple geometric operation as described above. However, for example, depth data (z-axis data) obtained by actually measuring a model eye in advance (or obtained from design data of the model eye) can also be used for the d-plane representation. In this case, for example, a point (Xc, Zc) on the c plane constructed from the photographing data is associated with a point (Xd, Zd) having the same x coordinate on the d plane, and the point (Xd, Zd) Thus, as is clear from Equations 7 and 8, the corrected pixel coordinates on the left and right parallax images corresponding to the point (Xc, Zc) can be easily obtained (the same applies to the y-axis).

  As described above, according to the present embodiment, in the image processing apparatus that performs the three-dimensional measurement processing on the obtained photographed image data by actually photographing the fundus of the eye to be examined through the stereo photographing optical system. It is possible to correct distortion of the optical system used for photographing, perform accurate three-dimensional measurement processing on the fundus image, accurately evaluate the three-dimensional shape of the fundus of the eye to be examined, and avoid problems such as misdiagnosis.

  For example, in the case of the fundus image 141 as shown in FIG. 4, by performing the correction as described above, a three-dimensional shape that accurately reflects the shape of the nipple portion 143 on the line P-P ′ is represented by depth information. (Z-axis coordinate value).

  Furthermore, according to the present embodiment, it is possible to provide an accurate fundus image corresponding to the three-dimensional measurement result in which the distortion of the optical system is corrected, in particular, to allow the examiner to observe through a stereo monitor. According to the above configuration (correction) processing, the captured right and left parallax images are displayed in 3D, and when the photographed eye fundus is observed by the examiner in stereoscopic view, the fundus eye to be examined has a shape corresponding to the correction of the shape data. 3D display images (left and right parallax images) can be corrected so that the examiner can observe (perceive) the shape of the 3D display image. The shape can be accurately evaluated.

  As described above, by performing the calibration process shown in FIGS. 11 and 12 along the x-axis and the y-axis, it is possible to correct the shape of the 3D display image on the xy plane. However, the calibration processing shown in FIGS. 11 and 12 is not intended only for 3D display, and is sufficiently effective when performed only when the left or right parallax image is used as a measurement image. is there.

  This is because the left and right parallax images photographed by the above-described optical system are photographed through a diaphragm aperture that is decentered from the optical axis of the optical system, so that the image distortion occurs unevenly. In principle, the calibration process shown in FIGS. 11 and 12 can correct the distortion of the left-right unequal image. For example, when the image is photographed through an aperture on the optical axis, Distortion can be corrected to a substantially equivalent image, and an accurate fundus measurement image without distortion can be obtained.

  Furthermore, the calibration process of the present embodiment can be easily performed using calibration data obtained by photographing a model eye that simulates the shape and optical structure of the eye to be examined. Calibration data is recorded as a difference expressed by depth or parallax, and fundus depth information (three-dimensional shape data) can be easily corrected with a light calculation load. The fundus image for display or recording can be corrected by a simple geometric operation as shown in FIGS.

  In addition, the subject to be stereo-photographed to obtain the calibration data may theoretically be a subject such as graph paper. However, by using a model eye that simulates the shape and optical structure of the eye to be examined as in the above embodiment, calibration measurement and calculation can be easily performed without changing other imaging conditions and calculation conditions. .

  Further, by providing a model eye wearing adapter such as the jig 311 described above, calibration measurement and calculation can be easily performed, and the model eye, the jig 311, the model eye wearing adapter, etc. This product can be supplied and sold to users and maintenance companies as calibration kits and expansion parts.

  The present invention provides an image processing such as a fundus camera that performs stereo imaging of the fundus of a subject's eye via a stereo imaging optical system and performs three-dimensional measurement processing or further display / recording processing of the captured image on the obtained captured image data. It can be implemented in the device.

It is explanatory drawing which showed the structure of the retinal camera as an image processing apparatus which employ | adopted this invention. It is explanatory drawing which showed the structure around the imaging | photography diaphragm of FIG. It is explanatory drawing which showed the model eye used in the apparatus of FIG. It is explanatory drawing which showed the three-dimensional shape measurement result of the fundus oculi obtained in the apparatus of FIG. It is explanatory drawing which showed the problem of the image distortion in the conventional stereo photography. It is the flowchart figure which showed the stereo imaging | photography process and three-dimensional measurement process in the apparatus of FIG. It is explanatory drawing which showed the mode of the parallax amount measurement process in the apparatus of FIG. It is explanatory drawing which showed the mode of depth amount (z-axis coordinate) measurement in the apparatus of FIG. It is the flowchart figure which showed the calibration and imaging | photography process in the apparatus of FIG. It is explanatory drawing which showed the calibration process of the depth (z-axis) direction in the apparatus of FIG. It is explanatory drawing which showed the calibration process of the stereo display image in the apparatus of FIG. It is explanatory drawing which showed the calibration process of the stereo display image in the apparatus of FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Fundus camera 24 Objective lens 28 Imaging stop 28a, 28b Aperture 300 Model eye 301 Retina simulation surface 302 Cornea simulation surface 311 Jig

Claims (8)

In an image processing apparatus that performs stereo imaging of the fundus of a subject's eye with a predetermined parallax via a fundus imaging optical system and performs a three-dimensional shape measurement process using the obtained parallax image.
Stereo shooting of a subject having a predetermined shape with the predetermined parallax through the fundus imaging optical system, and measuring a three-dimensional shape of a predetermined portion of the subject based on image data of the obtained parallax image,
A calibration that corresponds to the difference between the actual three-dimensional shape of the predetermined part and the three-dimensional shape of the predetermined part measured based on the image data, and that is expressed by the amount of parallax to be corrected in the three-dimensional shape measurement process Create data,
While performing shape distortion correction on the three-dimensional shape data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data,
An image processing apparatus, wherein corrected image data is created by applying the shape distortion correction to parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data. In an image processing apparatus that performs stereo imaging of the fundus of a subject's eye with a predetermined parallax via a fundus imaging optical system and performs a three-dimensional shape measurement process using the obtained parallax image.
Stereo shooting of a subject having a predetermined shape with the predetermined parallax through the fundus imaging optical system, and measuring a three-dimensional shape of a predetermined portion of the subject based on image data of the obtained parallax image,
This corresponds to the difference between the actual three-dimensional shape of the predetermined portion and the three-dimensional shape of the predetermined portion measured based on the image data, and is expressed by the depth amount to be corrected in the three-dimensional shape measurement process. Create calibration data,
While performing shape distortion correction on the three-dimensional shape data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data,
An image processing apparatus, wherein corrected image data is created by applying the shape distortion correction to parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data. In an image processing apparatus that performs stereo imaging of the fundus of a subject's eye with a predetermined parallax via a fundus imaging optical system and performs a three-dimensional shape measurement process using the obtained parallax image.
Stereo shooting of a subject having a predetermined shape with the predetermined parallax through the fundus imaging optical system, and measuring a three-dimensional shape of a predetermined portion of the subject based on image data of the obtained parallax image,
This corresponds to the difference between the actual three-dimensional shape of the predetermined part and the three-dimensional shape of the predetermined part measured based on the image data, and is stereo-photographed using the abscissa of the three-dimensional shape data or the fundus photographing optical system. Create calibration data created in table format in association with a specific part in the parallax image
While performing shape distortion correction on the three-dimensional shape data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data,
An image processing apparatus, wherein corrected image data is created by applying the shape distortion correction to parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data. 4. The image processing apparatus according to claim 1, wherein the subject has a simulated fundus oculi surface having a predetermined shading pattern as the predetermined portion while simulating the shape and optical characteristics of a human eye. An image processing apparatus characterized by being a model eye. The image processing apparatus according to claim 1, wherein the calibration data is created for each zooming condition of the fundus photographing optical system. 5. The image processing apparatus according to claim 4 , further comprising a mounting member that positions the model eye at substantially the same position as the eye to be examined with respect to the fundus photographing optical system. 4. The image processing device according to claim 1, wherein the shape distortion correction is applied to left and right parallax image data obtained by stereo imaging of the fundus of the eye to be examined using the calibration data. 5. An image processing apparatus, wherein left and right corrected image data are generated by the three-dimensional display using a stereo monitor. 3. The image processing apparatus according to claim 2 , wherein corrected image data is created by applying the shape distortion correction to parallax image data obtained by stereo imaging of the fundus of the eye to be examined by geometric calculation using the calibration data. An image processing apparatus.

Images