KR20120024842A - Image processing apparatus, control method thereof, and computer program - Google Patents

Abstract

The present invention is an image processing apparatus for processing a tomogram image obtained by imaging an eye to be examined by a tomography imaging device, and using layer candidate detection means and layer candidates to detect layer candidates of the retina of the eye to be examined from the tomography image. Artifact region determination means for determining an artifact region on a tomographic layer based on the obtained image characteristic, and image correction for correcting a luminance value in the artifact region based on the determination result of the artifact region determination means and the image characteristic in the region. Means;

Description

Image processing apparatus, control method thereof, and computer program {IMAGE PROCESSING APPARATUS, CONTROL METHOD THEREOF, AND COMPUTER PROGRAM}

The present invention relates to an image processing apparatus, a control method thereof, and a computer program.

Examination of ophthalmology is widely conducted for the purpose of early diagnosis of various diseases that occupy the top causes of illness and blindness related to lifestyle. A tomography apparatus for eye portion, such as optical coherence tomography (OCT), allows for three-dimensional observation of the state inside the retinal layers, thus making it more appropriate to diagnose the disease. It is expected to be effective in doing so. From this tomogram, for example, glaucoma and macular edema are measured by measuring a change in layer thickness such as a nerve fiber layer or a retina and a change in layer shape such as irregularities in the retinal pigment epithelial layer. ), And the extent of disease progression, such as age-related macular degeneration, and the level of recovery after treatment. In order to quantitatively measure the thickness of these layers, a technique for detecting each layer of the retina from a single layer using a computer and measuring the thickness of these layers has been proposed (see Japanese Laid-Open Patent Publication No. 2008-073099). ).

On the other hand, on the OCT tomography, when the measurement light is strongly reflected or absorbed by the object, artifacts due to attenuation or loss of the signal often occur behind the object. Note that the object includes, for example, tissue such as blood vessels and lesion sites such as alum and bleeding. As shown in Fig. 1A, when a luminance value is typically displayed in the depth direction of the retina (hereinafter referred to as the z-axis direction or the A-scan direction), the maximum luminance appears in the vicinity of the retinal pigment epithelium 2. However, as shown in FIG. 1B, when the artifact region 5 occurs on the positive side of the z-axis of the retinal vessel 4, the luminance near the retinal pigment epithelium 6 in the artifact region 5 is reduced. It is attenuated or missing. Therefore, depending on the degree of luminance attenuation of the layer in the region where the artifact has occurred, it is often difficult to extract the layer and to measure the layer thickness and the layer shape. To solve this problem, the vascular area is extracted from the surface image of the fundus, the vascular area is backprojected onto the OCT tomography, and the layers in the artifact area caused by the vascular are interpolated by interpolating the layers in the vicinity of the backprojection area. A technique for estimating is proposed (see Japanese Unexamined Patent Publication No. 2007-325831).

However, in the technique described in Japanese Unexamined Patent Publication No. 2008-073099, there is no disclosure about the method of calculating the layer position in the region where the artifact has occurred.

In addition, in the technique described in Japanese Unexamined Patent Application Publication No. 2007-325831, the luminance signal of the layer attenuated in the reverse projection area is merely an interpolation of the layers near the reverse projection area on the tomographic layer as the area where artifacts are likely to occur. Does not calculate the original floor position. In addition, even when image correction is performed for layer detection, the characteristics of luminance, such as histogram and contrast level, are different in the artifact region and other regions, so it is necessary to determine the artifact region and apply image correction appropriately. There is.

Therefore, the present invention determines an area where brightness is attenuated due to the influence of tissues such as blood vessels or lesion sites such as alum and bleeding, and performs image correction for making it easier to detect layers in the area.

An aspect of embodiments of the present invention is an image processing apparatus for processing a tomographic image obtained by imaging an eye to be examined by a tomographic imaging device, comprising: layer candidate detection means for detecting layer candidates of the retina of the eye to be examined from a tomography layer; Artifact area determining means for determining an artifact area on a tomographic layer based on image features obtained by using the candidates, and correcting luminance values in the artifact area based on the determination result of the artifact area determining means and the image characteristics in the area. It relates to an image processing apparatus including an image correction means.

Another aspect of embodiments of the present invention is an image processing apparatus for processing a tomographic image obtained by imaging an eye to be examined by a tomographic imaging apparatus, wherein layer candidate detection for detecting layer candidates of the retina of the eye to be examined is performed from the tomography. Means, an artifact region determining means for determining an artifact region on the tomographic layer based on an image characteristic obtained by using the layer candidates, and a layer of the retina in the artifact region based on a determination result of the artifact region determining means. Means for determining the position of the layer determining means, using the shape model defined by a plurality of control points and used to specify the layer shape included in the artifact region; An evaluation function associated with the shape and a flatness associated with the luminance value in the vicinity of the control point On the basis of the function it is related to the image processing apparatus for determining a position of the layer.

Other features of the present invention will become apparent from the following description of the embodiments (with reference to the accompanying drawings).

1A and 1B are diagrams illustrating an example of attenuation of an image signal in the retinal pigment epithelial layer due to the presence of retinal blood vessels.
2 is a block diagram showing the configuration of an apparatus connected to the image processing apparatus 10 according to the first embodiment.
3 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the first embodiment.
4 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the first embodiment.
5 is a view for explaining a method for determining an artifact region according to the first embodiment.
6 is a flowchart showing an image processing procedure necessary for determining an artifact area according to the first embodiment.
7 is a flowchart showing an image processing procedure for an area determined as an artifact area according to the first embodiment.
8 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the second embodiment.
9 is a flowchart showing an image processing procedure for an area determined as an artifact area according to the second embodiment.
10 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the third embodiment.
11 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the third embodiment.
12 is a diagram for explaining a method for determining an artifact region according to the third embodiment.
13 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the fourth embodiment.
14 is a block diagram showing the structure of an apparatus connected to the image processing apparatus 10 according to the fifth embodiment.
15 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the fifth embodiment.
16 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the fifth embodiment.
17A and 17B are views for explaining a method for determining an artifact region according to the fifth embodiment.
18 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the sixth embodiment.
19 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the seventh embodiment.
20 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the seventh embodiment.
21 is a flowchart showing an image processing procedure for an area determined as an artifact area according to the seventh embodiment.
It is a figure explaining the measuring method of the uneven | corrugated shape of the layer which concerns on 7th Example.
23 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the eighth embodiment.
24 is a flowchart showing an image processing procedure for an area determined as an artifact area according to the eighth embodiment.
25 is a view for explaining an image processing method determination method and a layer determination method according to the eighth embodiment.
26 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the ninth embodiment.
27 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the ninth embodiment.
28 is a diagram for explaining an artifact area determining method according to the ninth embodiment.
29 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the tenth embodiment.
30 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the eleventh embodiment.
31 is a flowchart showing the processing procedure of the image processing apparatus 10 according to the eleventh embodiment.
32 is a block diagram showing the functional configuration of the image processing apparatus 10 according to the twelfth embodiment.
33 is a block diagram showing the basic configuration of a computer that can realize respective units of the image processing apparatus 10 by software.

Hereinafter, with reference to the accompanying drawings will be described in detail preferred embodiments of the image processing apparatus and method according to the present invention. However, the scope of the present invention is not limited to the illustrated examples.

[First Embodiment]

When imaging a subject's eye (eye as an object to be examined) with an OCT device, if a blood vessel, alum, or the like is present on the retina, the intensity of the measurement light decreases, so that the retinal pigment epithelial layer is attenuated in the obtained image, making it difficult to detect the layer. Lose. Therefore, in the present embodiment, the artifact region is determined from the tomogram of the eye to be examined, and the image correction is performed on the region according to the statistics in the region.

2 is a block diagram showing the configuration of an apparatus connected to the image processing apparatus 10 according to the present embodiment. As shown in FIG. 2, the image processing apparatus 10 is connected to the tomographic imaging apparatus 20 via an optical fiber and an interface such as, for example, USB or IEEE1394. The tomographic image pickup device 20 is connected to the data server 40 via a local area network (LAN) 30 based on Ethernet® or the like, for example. Note that the image processing apparatus 10 can be connected to these apparatuses via an external network such as the Internet. The tomographic image pickup device 20 acquires a tomographic image of an eye portion and includes, for example, a time domain OCT or a Fourier domain OCT. The tomographic image pickup device 20 images three-dimensionally the tomographic image of the eye to be examined (not shown) in response to an operation by an operator (not shown). The tomographic image pickup device 20 transmits the acquired tomographic image to the image processing device 10. The data server 40 maintains the tomogram of the eye to be examined, the image feature amount of the eye to be examined, and the like. The data server 40 stores the tomographic image of the eye to be output from the tomographic imaging device 20 and the analysis result output from the image processing device 10. In response to a request from the image processing apparatus 10, the data server 40 transmits past data regarding the eye to be examined to the image processing apparatus 10.

Note that the present embodiment describes a case where a retinal pigment epithelial layer candidate is obtained. However, the candidate to be acquired is not limited to the outer boundary 2 of the retinal pigment epithelial layer. Another layer boundary (inner boundary (not shown) of the retinal pigment epithelial layer, boundary 3 between the photoreceptor segment and the outside, or outer boundary membrane (not shown)) may be detected. When acquiring a layer candidate, determining an artifact region, and performing image correction on the optic nerve papilla instead of the macula, an area where a layer does not exist, such as the center of the papilla (concave), is subjected to processing using a known site recognition method in advance. Can be excluded from the region.

In the following, the functional configuration of the image processing apparatus 10 according to the present embodiment will be described with reference to FIG. 3 is a functional block diagram of the image processing apparatus 10. As shown in FIG. 3, the image processing apparatus 10 includes a tomographic image acquisition unit 310, a storage unit 320, an image processing unit 330, a display unit 340, a result output unit 350, and The instruction acquisition unit 360 is included. The image processing unit 330 also includes a layer candidate detection unit 331, an artifact area determination unit 332, and an image correction unit 333. The function of each block constituting the image processing apparatus 10 will be described below in relation to the specific processing performed by the image processing apparatus 10 of the present embodiment with reference to the flowchart shown in FIG. 4.

In step S410, the tomographic image acquisition unit 310 requests the tomographic image pickup device 20 to transmit the tomographic image, and acquires the tomographic image transmitted from the tomographic image capture device 20. The unit 310 then transmits the acquired information to the storage unit 320. The storage unit 320 stores the tomographic image. In step S420, the layer candidate detection unit 331 acquires a tomographic image from the storage unit 320, and detects the inner boundary film 1 and the retinal pigment epithelial layer candidate point sequence {P _i } from the tomographic image. The unit 331 then outputs these results to the storage unit 320.

Since the retinal layers have different luminance values for each layer, contrast (edge) of density values occurs at the boundary between two adjacent layers. Therefore, paying attention to this contrast, the boundary of the layer is extracted. There are various methods of extracting the region containing such contrast. For example, considering the contrast as an edge, the layer position can be extracted by detecting the edge. More specifically, edge components are detected by applying an edge detection filter on a single layer, and edges are searched in the depth direction of the fundus from the vitreous side. The first peak position is then detected as the boundary between the vitreous body and the retinal layer, and the maximum peak position as the retinal pigment epithelial layer boundary.

In addition, a layer boundary may be detected by applying a deformable model such as a snakes or a level set method. In the level set method, a level set function that is one dimension higher than the dimension of the area to be detected is defined, and the layer boundary to be detected is regarded as its zero contour line. The contour is controlled by updating the level set function to detect the boundary of the layer. In addition, layer boundaries may be detected using graph theory such as GraphCut. In this case, a node corresponding to each pixel of the image, and a terminal called a sink and a source are set, and an edge (n-link) connecting between the nodes and an edge connecting between the terminals are set. Set (t-link). The layer boundary is detected by calculating a minimum cut based on a graph created by weighting the edges.

The above-described layer position extraction method can be applied three-dimensionally by treating the entire three-dimensional (3D) tomographic phase, or each 2D tomographic layer while considering the input 3D tomographic image as a set of two-dimensional (2D) tomographic layers. It may be applied independently with respect to the phase. Note that the method of detecting the layer boundary is not limited to these methods, and any other method may be used as long as the boundary of the layer can be detected from the tomogram of the eye area.

In step S430, the artifact region determining unit 332 determines whether or not an artifact occurs in the vicinity of each layer candidate point based on the degree of connection of the candidate point sequence {P _i } of the retinal pigment epithelium detected in step S420 (artifact) Whether an area occurs). And, when the artifact region is determined, the unit 332 calculates a statistic associated with the luminance value in the artifact region. In addition, the unit 332 outputs the determination result to the storage unit 320. The artifact area determination processing in this step will be described later in detail using the flowchart shown in FIG. 6.

In step S440, the artifact area determination unit 332 branches the processing according to the determination result obtained in step S430. That is, the unit 332 transmits a signal instructing the image correction unit 333 to perform a predetermined process with respect to the layer candidate point determined that artifacts have occurred (the process proceeds to step S450). On the other hand, when the unit 332 determines that the region other than the artifact region where artifacts do not occur (hereinafter referred to as a true image region), a signal instructing the execution of a predetermined process is transmitted to the display unit 340. (The process proceeds to step S455).

In addition, in step S450, the image processing unit 330 executes an analysis process when an artifact occurs in the vicinity of the candidate point of the predetermined layer. The processing of this step will be described later in detail using the flowchart shown in FIG. On the other hand, in step S455, the display unit 340 performs normal image display processing of displaying a tomographic image in association with the true image region as a process in which no artifacts occur in the vicinity of the candidate point of the predetermined layer. .

In step S460, the instruction acquisition unit 360 acquires, from the outside, an instruction of whether or not to store in the data server 40 the result of the current process associated with the examination target. This instruction is input by the operator, for example, via a keyboard and a mouse (not shown). If the operator has instructed to save the result of this time, the process proceeds to step S470, otherwise, the process proceeds to step S480. Then, in step S470, the result output unit 350 associates the inspection date and time, the information used to identify the eye to be examined, the tomogram of the eye to be examined, and the analysis result obtained by the image processing unit 330 as information to be stored. The information is sent to the data server 40.

In step S480, the instruction acquisition unit 360 acquires, from the outside, an instruction of whether or not to complete the tomographic analysis process by the image processing apparatus 10. This instruction is input by the operator, for example, via a keyboard and a mouse (not shown). When the instruction for terminating the process is acquired, the image processing apparatus 10 ends the process. On the other hand, when an instruction to continue the process is obtained, the process returns to step S410 to execute a process (or reprocessing for the same eye) for the next eye. In this manner, the processing of the image processing apparatus 10 is performed.

The procedure of the artifact area determination processing executed in step S430 will be described below using FIG. 5. FIG. 5 shows an example of a tomographic layer comprising an artifact region, and shows the region 5 enclosed by the dotted lines as the artifact region. The following two characteristics are known as the characteristics of the area | region where these artifacts generate | occur | produce. Note that in the tomogram shown in FIG. 5, the longitudinal direction of the tomogram corresponding to the depth direction of the retina is defined as the z-axis, and the transverse direction orthogonal to the depth direction is defined as the x-axis. The z-axis direction corresponds to the A-scan direction.

(1) The average, variance, and maximum values of the luminance values in these regions 5 are all smaller than those in the true image regions.

(2) When calculating the retinal pigment epithelial candidate points in the region including these regions 5, the points P _i and the high brightness regions other than the retinal pigment epithelial layer, for example, the retinal vascular region are incorrectly extracted. Non-connected parts like P _{i +1} are likely to occur.

Therefore, in this embodiment, using these features, each artifact area is determined as follows.

(i) Detect pairs of unconnected layer candidate points.

(ii) Examine the artifact-generating side among each pair of layer candidate points determined as unconnected points.

(iii) The layer candidate point sequence on the side where the artifact occurs is tracked until the next unconnected point is found.

(iv) The statistical value (average, variance, maximum value, etc.) of the luminance value at the deep side of the layer candidate point is calculated on the A-scan line passing through the tracked layer candidate point. Note that the deeper side means that the coordinate value in the z-axis direction is larger among two points in FIG. 5.

(v) The area having the unconnected candidate point as the edge point and having a low luminance statistic is determined as an artifact area. In addition, since the luminance statistic is considered to be an amount reflecting the degree of luminance attenuation in each artifact region, it is used when determining the image processing method in the artifact region in step S450.

A specific method of performing artifact determination based on the candidate point sequence {P _i } of the retinal pigment epithelium is described below with reference to the flowchart shown in FIG. In this case, the 3D tomographic image to be processed is considered to be a set of 2D tomographic images, and the following 2D image processing is performed on each 2D tomographic image.

In step S610, the connection degree C between neighboring layer candidate points is calculated for all layer candidate points. Connection diagram C is given by

Here, i is a layer candidate point number, and S _i is a statistic of luminance values of the pixels on the curve obtained by interpolating between the layer candidate points P _i and P _{i + 1} . In addition, S is a statistic of luminance values of pixels on a curve defined by the entire layer candidate point sequence {P _i }, and T _s is a threshold value. In this case, as the statistics S _i and S of the luminance values, the average of the luminance values of the pixels on the defined curves is used. In Equation 1, if | S _i -S | is equal to or less than the threshold value T _s , it is determined that neighboring points are connected.

Note that the statistics of luminance are not limited to the above-described amounts, and other statistics, for example, a maximum value, a variance, a mode value, a median value, and the like may be used. Alternatively, the degree of connectivity may be determined based on the combination of these statistics. In addition, as S, the statistical value related to the brightness | luminance value of the predetermined | prescribed layer computed previously about each imaging device or a test subject may be used, or the preset standard value may be used. Note that a statistic associated with the luminance value on the curve connecting the layer candidate points is used as an indicator used for the determination of the degree of connection. However, the present invention is not limited to such specific indicators. For example, incorrect detection of blood vessels or alum results in an edge parallel to the z-axis direction (indicated by reference numeral 7 in FIG. 5) in the vicinity of the unconnected candidate point, and as an index for capturing such characteristics. The degree of difference between the plurality of luminance profiles associated with the scan direction may be used. The luminance profile shows a graph showing the relationship between the spatial position in the A-scan direction and the luminance value at that position, as shown on the right side of FIG. 1A or 1B, and the difference between neighboring luminance profiles is typical. Note that it is small.

A specific calculation method of the indicator related to the degree of difference between the luminance profiles is expressed as, for example, as follows.

(1) Differences in luminance values in the x-axis direction with respect to the pixels in the local area surrounded by four points of P _i , P _{i +1} ', P _i ", and P _{i +1} " shown in FIG. Calculate D. Note that P _i "has the same x coordinate value as P _i and has a maximum z coordinate value. Also, P _{i + 1} " has the same x coordinate value as P _{i +1} and a maximum z coordinate value. It has a point.

(2) The sum ΣD is calculated by adding the difference D that is equal to or larger than the threshold Td1. The ΣD value is assumed to be a larger value as the edge appears clearer over a wider range.

(3) When the sum value ΣD is larger than the threshold Td2, disconnection points are determined.

In addition, the connection degree may be determined by combining a plurality of indices.

In step S620, it is checked which side of the pair of layer candidate points determined as non-connected points an artifact occurs to specify an edge portion of the artifact region. Specific processing of the artifact region edge portion is performed for each pair of unconnected layer candidate points. More specifically, in the case of the pair of P _i and P _{i +1} of FIG. 5,

(i) calculates, statistic of z-axis positive direction side of the luminance value of P _i of the phase A- scan line passing through the P _i.

(ii) The statistical value of the luminance value on the positive direction side of the z-axis of P _{i + 1} 'on the A-scan line passing through P _{i + i} is calculated. Note that P _{i +1} ′ has a x coordinate equal to P _{i +1} and a z coordinate equal to P _i .

(iii) Compare the two statistics and determine that artifacts occur on the candidate point side with the smaller statistics. The candidate point with the determined small statistic is defined as the edge portion of the artifact area.

In this example, since the statistics of the luminance values in a small P _{i +1} side, it is determined that artifacts are generated in the P _{i +1} side. Note that the method of selecting pairs of points used when calculating the statistic of the luminance value is not limited to the above-described method. For example,, P _i 'is (x has the same coordinates as the P _i, points having a z-coordinate, such as the P _{i +1)} and may be determined by using the P _{i + 1} as shown in Fig.

In step S630, the layer candidate point on the side where the artifact occurs is tracked until the next unconnected point is found, and the range of the artifact area is calculated. For example, with respect to P _{i + 1} of FIG. 5, up to P _{i +3} is determined as the artifact region.

In step S640, in the area determined as the artifact area, the average, variance, or maximum value of the luminance values of the areas on the positive direction side of the z-axis of each candidate point is calculated. However, the spatial range for calculating the statistics associated with the luminance value is not limited to that particular range. For example, the artifact region may be divided into arbitrary local regions, and statistics may be calculated for each local region. In this case, the tracking process of the layer candidate point and the calculation process of the statistics associated with the luminance signal are not necessarily executed as separate steps, and the statistics of the luminance signal may be calculated whenever the layer candidate point is tracked for an arbitrary range. do.

By the above-described processing, the statistics of the range of the artifact region having the non-connected portion as an end portion and the luminance value in the artifact region are calculated. Note that the determination method of the artifact area is not limited to the above-described method. For example, the artifact area determination process can be similarly performed not only on the B-scan image (tomographically perpendicular to the y-axis) but also on tomographicly perpendicular to the x-axis, and determined in both determination processings. The artifact region can be determined as the artifact region. Alternatively, the determination process may be applied three-dimensionally on the 3D tomography.

Referring to Fig. 7, the procedure of the processing executed in step S450 will be described below.

In step S710, the image correction unit 333 corrects the luminance value in each artifact region based on the statistics associated with the luminance value in the artifact region calculated by the artifact region determination unit 332. As a method of correcting the luminance value, a method based on the conversion of the histogram in the artifact region will be described. More specifically, the luminance average and the dispersion in the artifact region are adjusted to be the same as the luminance average and the dispersion in the true image region, respectively. In other words, the signal before correction is x, the signal after correction is y, and the standard deviation and average value of the luminance value in the artifact region are S _f and A _f , respectively, and the luminance value in the entire image except the artifact region. If the standard deviation and average value as S _t a and _t, respectively, and performs the calibration, as follows:

Note that the image correction method is not limited to this. For example, the image correction method may be achieved by the method described in (i) to (iii) below.

Alternatively, any image correction may be applied as long as it is a correction capable of establishing a relationship of the increment function between the luminance value before correction and the luminance value after correction in the artifact region.

(i) For example, the luminance is linearly transformed so that the maximum luminance in the artifact region matches the maximum luminance of the true image region. In this case, the luminance value of the artifact region after image correction is y, the luminance value of the artifact region before image correction is x, the maximum luminance in the artifact region is called I _maxF , and the minimum luminance in the region is called I _minF . If the maximum luminance in the true image area is I _maxT , correction is performed as follows.

(ii) For example, edge enhancement processing such as Sobel filter or Laplacian filter, or spatial frequency filter processing for passing only high frequency components are performed.

(iii) A layer structure enhancement filter process is performed based on the eigenvalues of a Hessian matrix that emphasizes the layer structure. This filter emphasizes the secondary local structure of the 3D concentration distribution based on the relationship between the three eigenvalues λ ₁ , λ ₂ , λ ₃ of the Hesse matrix. The Hesse matrix is a square matrix constituted by the second partial derivative of the density value I of the image, and is given as follows.

The relationship between the eigenvalues of the Hesse matrix is expressed as

The conditional expression of the eigenvalues necessary to emphasize the layer structure is expressed as follows.

By calculating the following equation from the three eigenvalues calculated on the basis of these equations, the layer structure of the retina can be emphasized,

Where ω (λ _s ; λ _t ) is a weighting function, which is given by

Here, γ and α are weights.

Note that the above-described image processing methods are not necessarily executed alone, but may be executed in combination. In step S640, when the artifact area is divided into a plurality of local areas and a statistical value associated with the luminance value is calculated for each area, image correction may also be applied for each local area.

In step S720, the display unit 340 superimposes the result of the correction of the image in step S710 on the tomography layer. When the boundary of each artifact region is represented by a line, a line of a predetermined color may be used for each boundary, or the layer may be represented by a translucent color without explicitly indicating the boundary. For example, the image before correction | amendment and the image after correction | amendment may be selectively displayed with respect to the area | region specified using GUI, and the information of the statistics associated with the luminance value in each artifact area computed in step S640 may be displayed. As described above, the process of step S450 is executed.

According to the above-described configuration, the image processing apparatus 10 easily detects a layer included in the artifact region by specifying an artifact region and performing image correction based on, for example, a statistic or the like associated with a luminance value in the region. A picture can be obtained.

Second Embodiment

In this embodiment, not only image correction is performed in the artifact region as in the first embodiment, but also a predetermined layer is detected from the corrected image. In this embodiment, even in an area where artifacts are generated and luminance is attenuated, image correction is performed so that the remaining edge information is easily detected, and the edge information is detected, thereby calculating a more accurate layer position.

Since the configuration of the device connected to the image processing apparatus 10 according to the present embodiment is the same as that of the first embodiment, the description thereof is omitted. 8 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. Referring to FIG. 8, the image processing unit 801 of the present embodiment has a structure of the image processing unit 330 of the image processing apparatus 10 of the first embodiment in that the layer determination unit 334 is added. It is different.

In the following, the image processing procedure of the present embodiment will be described. The processing procedure of this embodiment is as shown in the flowchart of FIG. 4, except for step S450 and step S455. Therefore, in this embodiment, only steps S450 and S455 are described, and description of other steps is omitted.

In step S450, image processing in the artifact region is performed. Below, the detail of the process in this step is demonstrated using FIG.

In step S1010, the image correction unit 333 corrects the luminance value in the area based on the statistics associated with the luminance value in the artifact area calculated by the artifact area determination unit 332. Note that since this processing is the same as the image correction processing in step S710, the detailed description thereof is omitted. In step S1020, the layer determination unit 334 acquires the image feature of the layer to be extracted based on the luminance information of the region corrected by the image correction unit 333, and connects these feature points to the layer position. .

For example, the retinal pigment epithelial layer is originally the region with the highest luminance on each A-scan line, and the luminance is likely to increase even within the artifact region. Therefore, in each A-scan line of the image-corrected area, the layer position is determined by connecting pixels of maximum luminance on the positive direction side of the z-axis of the layer candidate point in the x-axis direction. However, the method of detecting a layer from the image correction result is not limited to this specific method.

For example, in each A-scan line, the layer may be extracted by connecting pixels having a luminance value equal to or greater than a predetermined value on the positive direction side of the z-axis of the layer candidate point and having the maximum z coordinate in the x-axis direction. Alternatively, for each pixel on each A-scan line in the artifact region, a linear sum of the luminance values before and after image correction is calculated, and the pixels corresponding to the maximum sum are connected in the x-axis direction to extract the layer. You may also

Alternatively, a plurality of candidate points of the retinal pigment epithelial layer are selected in each A-scan line of the artifact region, and all of the above layer candidate points are defined as a set of layer candidate points set in the artifact region. For each combination of the layer candidate point sequences obtained from the set of layer candidate points,

(i) the magnitude of the sum of the luminance values of the layer candidate point sequences, and

(ii) the smoothness of the layer candidate point sequence shape

The evaluation functions associated with may be set, and a combination of the layer candidate points at which the linear sum of the two evaluation function values is maximum may be determined as the floor position. Note that when selecting a combination of the layer candidate points, the evaluation function value may be calculated using not only the layer candidate point sequence in the artifact region but also the layer candidate point sequence including the layer candidate viscosity in the vicinity of the region.

In step S1030, the retinal layer thickness is measured by calculating, in each x and y coordinate, the distance between the layer candidate point sequence corresponding to the calculated retinal pigment epithelial layer and the internal boundary film 1 calculated in step S420. However, the measurement content is not limited to this. For example, in order to investigate layer-shaped unevenness, you may calculate angle distribution between layer candidate points. The layer thickness to be measured is not limited to the retinal layer thickness. For example, you may analyze another layer shape, such as a photoreceptor cell layer. Information including the calculated layer thickness and layer shape is output to the storage unit 320.

In step S1040, the display unit 340,

(i) Result of Determination of Layers

(ii) the measurement result of the layer shape, and

(iii) the range of the artifact area and the result of image correction in that area

Superimpose on the fault.

For (i), the display unit 340 superimposes the layer determination result in step S1020 on the single layer. When the boundary of the layer is represented by a line, a line of a predetermined color may be used for each boundary, or the region of the layer may be presented in a translucent color without explicitly indicating the boundary. Note that when performing such a display, a configuration in which a section to be selected is selected using, for example, a GUI or the like is preferably adopted. Moreover, you may display the result three-dimensionally using a well-known volume rendering technique.

For (ii), the display unit 340 displays the distribution map of the layer thickness with respect to the entire 3D tomographic layer (x-y plane) as the measurement result of the layer shape. However, the present invention is not limited to such a display method, and may display the area of each layer in the cross section of interest, or may display the volume of the entire predetermined layer. Alternatively, the operator may calculate and display the volume in the area designated by the operator on the x-y plane.

For (iii), in step S1020, when the reliability of the detected layer is low (for example, when the signal of the detected layer is weak), the image correction result is superimposed on the single layer without detecting the layer. However, the present invention is not limited to such a specific display method of the image correction result, and even when the layer detection result is good, the image correction result may be displayed on the display unit 340.

As described above, the image processing for the artifact area is executed in step S450. The image processing for the true image area in step S455 will be described below. In step S455, as the processing in the true image area,

(i) the layer candidate point sequence obtained in step S420 is defined as the layer position, and the layer shape is measured from the layer position;

(ii) The measurement result of the layer position and the layer shape is displayed on the display unit 340.

Therefore, below, the detail of the process in this step is demonstrated.

The layer shape is measured similarly to step S1030 of FIG. However, in the case of the image processing for the true image region, it is different from the case of step S1030 that the image to be analyzed is not corrected. Then, similarly to step S1040 of FIG. 10, the display unit 340 displays the result. However, in the case of the image processing for the true image region, not displaying the image correction result is different from the case of step S1040.

According to the above-described configuration, the image processing apparatus 10 can more accurately calculate the layer position in the artifact region by performing image correction on a specific artifact region and detecting an image feature corresponding to the layer position from the correction result. .

Third Embodiment

In the present embodiment, instead of determining the artifact region using only the tomographic image as in the first and second embodiments, the projection image is created from the tomographic image of the eye to be examined, and the positional information of the tissue or lesion site extracted from the projection image is obtained. Backprojection onto the tomogram narrows the artifact candidate region in advance. In general, it is easier to calculate the positional information of the artifact area caused by, for example, blood vessels (or bleeding) from the projected image than from the tomographic image alone. Therefore, in the present embodiment, the blood vessel (bleeding) region is extracted from the projected image, the positional information is mapped onto the tomographic image, and the end of the artifact region is searched and specified around the mapped region, thereby providing high accuracy of the artifact region. The case of calculating a range is demonstrated.

Since the configuration of the apparatus connected to the image processing apparatus 10 according to the present embodiment is the same as that of the first embodiment, the description thereof is omitted. 10 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. Referring to FIG. 10, the image processing unit 1001 includes the projection image generating unit 335 and the feature extraction unit 336, so that the image processing unit 330 of the image processing apparatus 10 of the first embodiment is provided. Is different. Since the remaining units are the same as the units of FIG. 3, the description thereof is omitted.

With reference to the flowchart shown in FIG. 11, the content of the image processing which concerns on this embodiment is demonstrated. Most of the steps of this flowchart are common to the flowchart of Fig. 4, and common processing includes the same step number. The description of the common processing is omitted. This embodiment differs from the first embodiment in that projection image generation processing S1110 and feature extraction processing S1120 are performed between the layer candidate detection process in step S420 and the artifact region determination process in step S430.

In step S1110, the projection image generating unit 335 generates an image by projecting a tomographic image. More specifically, the projection image is defined as a pixel value of the projection image as a value obtained by simply adding the luminance value in each pixel on the tomography in the positive direction of the z-axis. However, the pixel values of the projection image are not limited to these values, and the sum of the luminance values may be divided by the added number of pixels. Alternatively, the maximum or minimum value of the luminance value at each depth position may be used as each pixel value of the projection image. In addition, it is not necessary to add the luminance values of all the pixels in the z-axis direction, and may add only in an arbitrary range or only between specific layers.

In step S1120, the feature region in which the biological tissue, the lesion site, etc. in the eye to be examined, such as the retinal blood vessel, is extracted from the projection image generated by the projection image generating unit 335. Since the retinal vessels have a thin linear structure, the retinal vessels are extracted using a filter that emphasizes the linear structure. In this case, for example, using a line segment enhancement filter based on contrast, such as a filter for calculating a difference between an average value of image density values in a line segment defined as a structural element and an average value in a local area surrounding the structural element, for example. do. Note that the multi-valued region obtained as a result of the filter treatment may be used as the blood vessel extraction result, or the region binarized using a specific threshold value may be used as the extraction result.

However, the method of emphasizing the linear structure is not limited to this. For example, a differential filter such as a Sobel filter or a Laplacian filter may be used. The eigenvalue of the Hesse matrix can be calculated for each pixel of the density value image, and the linear region may be extracted from the combination of the two eigenvalues obtained as a result. Moreover, you may use arbitrary well-known blood vessel extraction methods, such as the top hat calculation which uses a line segment simply as a structural element.

In this way, if the feature regions (x, y) on the projected image calculated in step S1120 are back projected on the tomogram, a back projection region as shown in the dotted line region 8 in Fig. 12 is obtained. In general, attenuation of luminance tends to occur on the positive side of the z axis of the retinal blood vessel. Thus, when backprojecting the position (in the x-y direction) of the extracted feature onto a tomogram, the backprojected dotted area 8 is more likely to include the artifact area 5. However, in the case where the wrongly extracted region is back projected, the luminance attenuation does not occur in the back projected region. Even in the case of backprojecting the correctly extracted retinal vascular region, the luminance attenuation under the backprojection region is slight, so that there is little effect on layer extraction.

Thus, it is determined whether artifacts occur in the reverse projection area and near the boundary of the area. When an artifact region occurs, a statistic associated with a luminance value within that region is calculated. Therefore, the determination method of the artifact area in step S430 is basically the same as in the case of steps S610 to S640 of the first embodiment except for the range of the layer candidate point as the calculation target of the connection degree. More specifically, the linkage calculation processing is not performed for all the layer candidate points, but is performed inside the reverse projection area and in the vicinity of the x-y direction of the area.

According to the above-described configuration, the image processing apparatus 10 of the present embodiment specifies an artifact region from a tomographic image and a projected image, and performs artifact correction by performing image correction in the region based on, for example, a luminance statistic or the like. It is possible to obtain an image that can more easily detect the layer region included in the region.

[Example 4]

Unlike the third embodiment, this embodiment not only performs image correction in the artifact area after determining the artifact area, but also detects a predetermined layer from the corrected image. In this embodiment, the following matters are processed.

(i) When an artifact occurs due to blood vessel (or bleeding) or the like, the position information of the blood vessel (bleeding) region calculated from the projection image is mapped onto the tomogram, and the edge portion of the artifact region in the peripheral region of the mapped region is By searching and specifying, a range of artifact regions can be calculated with higher precision.

(ii) Even in an area where artifacts occur and the luminance is attenuated, in order to make it easier to detect the remaining edge information, image correction is performed on the area, and edge information is detected from the correction area, so that more accurate floor positions can be calculated. have.

Since the configuration of the device connected to the image processing apparatus 10 according to the present embodiment is the same as that of the third embodiment, the description thereof is omitted. 13 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. The image processing unit 1301 of the present embodiment differs from the image processing unit 1001 of the third embodiment in that it includes the layer determination unit 334. The contents of the image processing in this embodiment are the same as the image processing contents in FIG. 13 except for the processing in steps S450 and S455. Therefore, only the processing in steps S450 and S455 of the present embodiment is described, and description of other steps is omitted.

In step S450, image correction, layer determination, layer shape measurement, and result display processes are performed as image processing in the artifact region. Since the processing of this step is the same as in the case of steps S1010 to S1040 in the second embodiment, detailed description thereof is omitted. In step S455, as the processing performed when no artifacts occur, the layer shape is measured from the layer position acquired in step S420, and the layer position and the layer shape measurement result are superimposed on the single layer. Details of the superimposition method are the same as those in the steps S1110 to S1120 in the second embodiment, and thus detailed description thereof is omitted.

According to the above-described configuration, the image processing apparatus 10 of the present embodiment specifies an artifact region from a tomographic image and a projected image, and performs image correction in that region. By detecting the image characteristic of the layer from the above correction result, the position of the layer in the area can be calculated more accurately.

[Fifth Embodiment]

In the present embodiment, in addition to the third embodiment, the positional information of the tissue or the lesion site extracted from at least one of the surface image and the projection image of the eye to be examined is reversely projected on a tomogram to narrow the artifact candidate region in advance. This can be achieved by calculating the alum area using surface images and searching for and specifying the edge portion of the artifact area in the peripheral area of the extracted lesion site, especially when a lesion site such as alum appears to be easily extracted from the surface image. This is because the range of the artifact region can be calculated with high precision.

14 shows the configuration of an apparatus connected to the image processing apparatus 10 according to the present embodiment. In the present embodiment, in addition to the tomographic imaging device 20, the surface image imaging device 50 is also included in the configuration, which is different from the third embodiment. The surface image pickup device 50 picks up a surface image of an eye portion and includes, for example, a fundus camera or a scanning laser ophthalmoscope (SLO). 15 is a functional block diagram of the image processing apparatus 10 of this embodiment. The image processing apparatus 10 of the present embodiment includes a surface image acquisition unit 315, and the image processing unit 1501 includes a registration unit 337. It is different from the configuration.

Image processing in the image processing unit 1501 of this embodiment will be described below with reference to the flowchart shown in FIG. The image processing procedure of this embodiment is almost the same as that shown in FIG. 11 except for the processing of steps S1610 to S1650. Therefore, the following describes the processing in these steps.

In step S1610, in addition to the tomographic image acquisition by the tomographic image acquisition unit 310, the surface image acquisition unit 315 requests the surface image image pickup device 50 to transmit the surface image, and from the surface image image pickup device 50. Acquire a surface image to be sent. Assume that a fundus camera image is input as the surface image. The unit 315 transmits the acquired information to the storage unit 320.

In step S1620 subsequent to the projection image generation processing in step S1110, the feature extraction unit 336 extracts tissue such as blood vessels or lesion area such as alum from the surface image acquired by the surface image acquisition unit 315. Since the retinal vessels have a linear structure, they are extracted using a filter that emphasizes the linear structure. Since the extraction method of the linear structure is the same as that of step S1120, the description is abbreviate | omitted. Since alum exists as a granular high brightness area | region, it is computed by morphology calculations, such as a top hatch | transformation. In this case, the alum region may be obtained as a multi-value region of high brightness by morphology calculation, and the multi-value region itself may be used as an extraction result, or a region binarized using a specific threshold value may be used as an extraction result. However, the extraction method of alum is not limited to this, and an identifier such as a Support Vector Machine or Ada Boost, which uses, as a feature amount, the luminance value of the surface image and the output result of a known filter that emphasizes contrast, is used. You may identify alum by an identifier ensemble such as Ada Boost.

In step S1630, in order to associate the coordinates of the projected image with the coordinates of the surface image, the registration unit 337 performs registration between the projected image and the surface image. In registration, an evaluation function expressing the similarity between two images is defined in advance, and the image is modified to obtain an optimum evaluation value. As a similarity evaluation method, a method of evaluating similarity based on pixel values using mutual information amounts is used. However, the present invention is not limited to such a specific method, and the mean square error, correlation coefficient, or overlapping area between blood vessel regions extracted from the surface image and the projection image by the feature extraction unit 336, or branched portions of the vessel You may use the distance between them. Image deformation is realized by translating or rotating an image by assuming an affine transformation, or by changing an enlargement ratio.

In step S1640, the blood vessel or alum extraction result from the surface image calculated in step S1620 is back projected onto the tomographic phase using the registration parameter calculated in step S1630. As the reverse projection area, the area shown by the dotted line area 8 in Fig. 17A is obtained. In general, since attenuation of luminance often occurs on the positive side of the z-axis of the retinal blood vessels, if the (xy direction) coordinates and luminance of the region from which the features are extracted are backprojected onto a tomogram, each backprojected dotted region ( 8) is more likely to include artifacts. However, when the alum is extracted from the surface image as in the present embodiment, there is a possibility that a high-brightness region of a granular particle such as druse is extracted out of the alum candidates. In this case, as shown in Fig. 17B, luminance attenuation does not occur in the layer region. Even reverse projection of the correctly extracted retinal blood vessel region may result in a slight luminance attenuation below the reverse projection region, thus having little effect on layer extraction. Thus, it is determined whether artifacts occur in the reverse projection area and near the boundary of the area. When an artifact occurs, a statistical amount of luminance values in the area is calculated.

The determination method of the artifact region is basically the same as in the case of steps S610 to S640 of the first embodiment, but the range of the layer candidate points as the calculation target of the connection degree is different from that of the first embodiment. More specifically, the calculation of the degree of connection is not performed for all the layer candidate points, but in the inside of the reverse projection area and in the vicinity of the x-y direction of the area. Note that, in addition to the information obtained only from the tomographic image, the artifact region may be determined with reference to the information obtained from the projection image and the fundus image. For example, when the retinal vascular area obtained from the projection image and the retinal vascular area obtained from the fundus image overlap, it may be considered that the area is more likely to be a blood vessel, and the edge portion of the area may be determined as unconnected. . Alternatively, the linear sum of the value of the degree of connection calculated from the tomographic layer and the value of the overlapping degree of the blood vessel region can be calculated and binarized using a threshold value, so that the degree of connection can be determined.

In the image processing for artifact area in step S1650, since the correction result after image correction is displayed, i.e., basically the same procedure as in the first embodiment is adopted, the detailed description thereof is omitted. However, in this embodiment, the information obtained from the fundus image can also be referred to at the time of image correction of the artifact region. For example, when the luminance of the alum is very high on the fundus image, the luminance is more likely to be attenuated on the positive side of the z-axis of the layer candidate point even on the tomogram, so that the luminance value is proportional to the luminance signal value of the alum region. Amplify or highlight. In this case, the pixel value of the area on the fundus image is directly referred to as the luminance value of the alum area. However, the luminance value of the alum area is not limited to these values, and may refer to, for example, the value (value data) of the processing result obtained by morphology calculation or the like.

According to the above-described configuration, the image processing apparatus 10 of the present embodiment, in the artifact region specified by using the surface image and the projection image, corrects the image based on the statistical value of the luminance value, thereby presenting the layer present in the region. It is possible to obtain an image in which the area can be detected more easily.

[Sixth Embodiment]

In this embodiment, not only the image correction of the artifact region of the fifth embodiment is performed, but also the detection of a predetermined layer from the corrected image is performed. This embodiment uses the following matters, especially when artifacts occur due to alum.

(i) The position information of the alum area calculated from the surface image is mapped onto the tomogram, and the range of the artifact area is calculated with higher precision by searching and specifying the edge portion of the artifact area in the peripheral area of the area.

(ii) Even in an area where artifacts occur and the luminance decays, the more accurate layer position is calculated by image correcting the area so that the remaining edge information is easier to detect and detecting the edge information from the corrected area.

The configuration of the device connected to the image processing apparatus 10 according to the present embodiment is the same as that of the fifth embodiment. 18 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. The image processing unit 1801 of the present embodiment is different from the image processing unit 1501 of the fifth embodiment in that it includes the layer determination unit 334. The image processing procedure in this embodiment is the same as the image processing procedure shown in FIG. 16 except for the processing in steps S1650 and S455. Therefore, only the processing in steps S1650 and S455 is described, and description of other steps is omitted.

In step S1650, image correction, layer determination, layer shape measurement, and result display processes are performed as image processing in the artifact region. Since the processing in this step is the same as in the case of steps S1010 to S1040 in the second embodiment, detailed description thereof is omitted. However, in this embodiment, in step S1010, image correction can also be performed using information obtained from the fundus image. Since the specific procedure which references the information obtained from a fundus image at the time of image correction is the same as that of the case of step S1650 in 5th Example, the description is abbreviate | omitted.

In step S455, as a process performed when artifacts do not occur, the layer shape is measured from the layer position acquired in step S420, and the layer position and the layer shape measurement result are superimposed on the single layer. Since the superposition method in this case is the same as that in the second embodiment, detailed description thereof is omitted.

According to the above-described configuration, the image processing apparatus 10 performs image correction in the artifact region specified from the surface image and the projection image. By detecting the image characteristic of the layer from the correction result, the position of the layer in the area can be calculated more accurately.

[Example 7]

In the present embodiment, the artifact region is determined from the tomogram of the eye to be examined, and the layer position in the artifact region is calculated by using both information in consideration of the luminance value in the artifact region and the layer shape around the region.

Since the configuration of the device connected to the image processing apparatus 10 according to the present embodiment is the same as that shown in FIG. 2 of the first embodiment, the description thereof is omitted. 19 is a functional block of the image processing apparatus 10 according to the present embodiment. Referring to FIG. 19, the image processing unit 1901 of the present embodiment includes an image processing method determination unit 1910 instead of the image correction unit 333, and further includes a layer determination unit 334. The configuration of the image processing unit 330 of the image processing apparatus 10 of the first embodiment is different. The image processing method determination unit 1910 includes a luminance use determination unit 1911 and an evaluation function setting unit 1912. The function of each block constituting the image processing apparatus 10 will be described with reference to the flowchart shown in FIG. 20 in association with the procedure of specific processing executed by the image processing apparatus 10 of the present embodiment.

The image processing procedure of this embodiment will be described below. Note that the processing procedure of this embodiment is the same as the flowchart shown in FIG. 4 except for the processing in steps S2010 to S2030. Therefore, only the descriptions of S2010 to S2030 will be described below, and the description of other steps will be omitted.

In step S2010, the artifact area determination unit 332 branches the processing in accordance with the determination result obtained in step S430. That is, the unit 332 transmits a signal instructing the image processing method determination unit 1910 to execute a predetermined process with respect to the layer candidate point determined that the artifact occurs. On the other hand, when it determines with the true image area | region where an artifact does not generate | occur | produce, the unit 332 transmits the signal which instruct | indicates execution of a predetermined process to the display unit 340. As shown in FIG.

In step S2020, the image processing unit 1901 executes an analysis process when artifacts occur in the vicinity of candidate points of the predetermined layer. The processing of this step will be described later in detail using the flowchart shown in FIG. In step S2030, the display unit 340 superimposes the determination result of the layer on the single layer. When the boundary of the layer is represented by a line, a line of a predetermined color may be used for each boundary, or the region of the layer may be presented in a translucent color without explicitly indicating the boundary. When performing such display, it is noted that it is preferable to adopt a configuration in which the cross section of interest is selectable using, for example, a GUI or the like. In addition, these results may be displayed three-dimensionally using a known volume rendering technique.

The retinal layer thickness can be measured by measuring the distance between the calculated layer candidate point sequence corresponding to the retinal pigment epithelial layer and the internal boundary film 1 calculated in step S420 in each coordinate (x, y). have. In this case, the display unit 340 presents the information related to the measured layer shape as a distribution map of the layer thickness with respect to the whole 3D tomogram (x-y plane). However, the present invention is not limited to this specific display method. For example, the display unit 340 may display the area of each layer in the cross section of interest in synchronization with the display processing of the detection result. Alternatively, the display unit 340 may display the entire volume, or may calculate and display the volume in the area designated by the operator on the x-y plane.

In the following, the image processing for the artifact area in step S2020 of the present embodiment will be described. In this embodiment, a deformable model is applied to the floor position so that the floor position can be calculated even for a tomographic layer containing noise. Hereinafter, an example of using Snakes as the variable shape model will be described. In this case, the floor position is determined by minimizing the linear sum of the evaluation function values associated with the shape of the model and the evaluation function values associated with the luminance values near the control points constituting the model.

As the evaluation function associated with the shape, a linear sum of the difference value and the second derivative value of the control point positions constituting the model is used. As the linear sum decreases, the model shape becomes smoother. As an evaluation function associated with the luminance value, a value obtained by assigning a negative sign to the luminance value gradient near the control point constituting the model is used. This is so that as the distance to the edge decreases, the evaluation function value becomes smaller. Regarding the weight of the evaluation function used to deform the variable shape model, it is common to set the fixed value regardless of whether each control point constituting the model is included in the artifact area. In this case, the luminance value is attenuated in the artifact region, and since the change in the luminance value is small in the region, the layer position is determined based on the magnitude of the evaluation function value substantially associated with the model shape. In the artifact area, for example, if information such as an edge remains, the layer position is determined by focusing on the information related to the luminance (compared to calculating the layer position based on the smoothness of the model shape). The shape can be detected. For this reason, the control point in the artifact region is set so that the weight of the evaluation function associated with the luminance becomes larger than that in the true image region in accordance with the degree of attenuation of the luminance value.

However, when the luminance value in the artifact area is low and, for example, most of the information such as the edge does not remain, when determining the layer position, for example, the luminance information such as the edge cannot be used. Thus, the floor position is determined based on the magnitude of the evaluation function value associated with the model shape without increasing the weight of the evaluation function associated with the luminance value.

In the following, a method of specifically setting the weight of each evaluation function of the variable shape model will be described with reference to the flowchart shown in FIG. 21.

In step S2110, the statistical value associated with the luminance value in the artifact area calculated in step S640 of the flowchart of FIG. 6 showing the details of the artifact area determination processing of step S430, from the storage unit 320, the image processing method determination unit 1910 ) Reads. Based on the statistics, the unit 1910 uses the luminance information at the time of layer detection because the (damped) edges remain in the area, or the luminance information such as the edge, for example, because the luminance value is missing. Determine whether or not to use.

More specifically, the index E (i) is calculated, and if E (i) = 1, it is determined that the luminance information is used when determining the floor position, and if E (i) = 0, the luminance information is not used. To judge. The index E (i) is given by

Here, i is a control point number of the variable shape model, B is a statistic associated with a luminance value in the background region (for example, the region on the negative direction side of the z-axis of the inner boundary film 1), and F _i Is a statistic associated with a luminance value in the artifact region to which the control point i belongs. As the statistic associated with the luminance value, the maximum value is used. In addition, T _s is a threshold value.

Note that the statistic associated with the luminance value is not limited to this. For example, you may use an average value, a variance, or a standard deviation. In addition, the artifact region may be divided into arbitrary local regions, and a statistical value of luminance values in the local regions to which each control point belongs may be used as F _i .

In step S2120, the image processing method determination unit 1910 acquires information associated with the unevenness of the layer candidate point sequence around the artifact area. This is because when there are irregularities in the layer shape around the artifact region, irregularities may occur even within the region, and it is necessary to calculate the layer position by lowering the weight associated with the evaluation of the smoothness of the shape. As a method of calculating the concrete layer-shaped unevenness | corrugation, the statistics related to the angle between layer candidate points around an artifact area are computed. In this case, the maximum value is used as a statistic.

The angle between the layer candidate points in the layer candidate point i is between the line segment obtained by extending the line segment Q _{i -1} -Q _i to the Q _{i +1} side as shown in FIG. 22, and the line segment Q _i -Q _{i +1} . It calculates as angle (theta) _i between them. The larger the _i , the greater the unevenness. This angle calculation is performed for each layer candidate point around the artifact region, and the maximum value of the calculated angle is used as an index indicating the degree of irregularities of the layer candidate point sequence. Indicator associated with a degree of unevenness is not limited to the angle between the floor candidate points, the candidate point where the layers _{(Q i -1 - 2Q i +} Q i +1) 2 of the differentials of the order statistics in the (mean, variance , The maximum value, etc.). Alternatively, the number of extremes or inflection points may be calculated when the layer candidate around the artifact region is regarded as a curve. The statistics related to the degree of irregularities of the layer candidate points are not limited to the maximum value. For example, you may use an average value, a variance, or a standard deviation.

In step S2130, the image processing method determination unit 1910 sets the weight of the evaluation function of the variable shape model by using an indicator associated with the determination result in step S2110 and the degree of unevenness of the layer shape calculated in step S2120. As the weight of the evaluation function associated with the shape, a value inversely proportional to an index representing the degree of irregularities of the layered model calculated in step S2120 is set. The weight of the evaluation function associated with the luminance value is set as follows according to the determination result associated with the use of the luminance information in the artifact area calculated in step S2110.

(i) When using the luminance information in the artifact region when determining the layer position

The weight of the evaluation function associated with luminance is increased according to the degree of attenuation of the luminance value in the artifact region. Then, a value proportional to the ratio T _s / F _s between the luminance statistics F _{s in} the area calculated in step S640 and the luminance statistics T _{s in} the true image area is set as the weight of the evaluation function associated with luminance. However, the method of setting the weight of the evaluation function associated with the luminance is not limited to this. Arbitrary weighting functions may be set as long as the relationship of the reduction function is established between F _s and the weight of the evaluation function associated with the luminance.

(ii) When determining the position of the layer, when not using the luminance information in the artifact region

In the artifact region, the weight of the evaluation function associated with the luminance value of the shape model is set to the same value as that in the true image region. Note that in (ii), the method of setting the weight of the evaluation function associated with the luminance value is not limited to this. For example, the weight of the evaluation function associated with the luminance value may be reduced or may be set to zero.

In step S2140, the floor determination unit 334 calculates an evaluation value according to the weight of the evaluation function set in step S2130, performs iteration calculation using an optimization method such as a Greedy algorithm, and minimizes the evaluation function value. When the amount of change in the evaluation value is less than the predetermined value or when the number of iteration calculations exceeds the predetermined number of times, the unit 334 terminates the deformation of the layered model and determines the position of the layered model at the end as the layer position. .

Note that these shape models may be calculated as 2D or 3D curved models. In the present embodiment, an example of using Snakes as the variable shape model has been described, but a level set may be used. Further, in the model-based division method for referring to the luminance value at the time of deformation of the model, any method may be used as long as the weight of the evaluation function associated with the luminance value is set.

According to the above-described configuration, the image processing apparatus 10 specifies an artifact region and performs image processing in consideration of the layer-shaped unevenness around the region and the edge information in the region, thereby providing higher accuracy than the conventional method. The floor location can be calculated.

  [Example 8]

In the present embodiment, when calculating the position of the layer in the artifact region, it is determined whether or not luminance information such as an edge in the region is used. In the case of using the luminance information, the layer position is calculated after correcting the luminance value of the region. When the luminance information is not used, the layer position is calculated by interpolation. In this embodiment, the following matters are covered.

(i) In an area where luminance values, such as blood vessels and small alum, are attenuated but not missing, for example, the position of the layer is detected after converting the luminance value so that information such as remaining edges is easier to use, thereby making the layer position more accurate. Calculate

(ii) In an area where edge information is not available due to lack of luminance such as a large alum or a severely bleeding area, by interpolating surrounding layer positions in consideration of the occurrence position of the artifact region and the surrounding layer shape, a more accurate layer position is obtained. To calculate.

Since the configuration of the device connected to the image processing apparatus 10 according to the present embodiment is the same as that of the seventh embodiment, the detailed description thereof is omitted. 23 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. In the present embodiment, the image processing unit 2301 includes an image correction unit 333, and the image processing method determination unit 1910 includes an image correction method setting unit 1913 and an interpolation function setting unit 1914. The point is different from the case of the seventh embodiment.

The contents of the image processing in this embodiment will be described with reference to the image processing procedure shown in FIG. Note that the image processing procedure in this embodiment is the same as in the seventh embodiment except for the processing in step S2020. Therefore, below, the change of the process in step S2020 is demonstrated with reference to FIG. 24, and description of another step is abbreviate | omitted.

In step S2410, since weak edges remain in the artifact region, it is determined whether luminance information such as edges or the like is not used when calculating the position of the layer, or because luminance is lost. Since the concrete judgment procedure is the same as that in the case of step S2110 of the seventh embodiment, the description thereof is omitted.

In step S2420, the brightness utilization determining unit 1911 branches the processing according to the determination result in step S2410. That is, when determining that the luminance information in the artifact area is to be used when determining the floor position, the unit 1911 transmits a signal instructing the image correction method setting unit 1913 to execute a predetermined process. On the other hand, when it is determined that the luminance information in the area is not used, the unit 1911 transmits a signal instructing the interpolation function setting unit 1914 to execute the predetermined process.

In step S2430, parameters necessary for performing conversion (image correction) of luminance values in the artifact region are set. As an image correction method, various things are possible. In the present embodiment, the procedure for setting parameters in the following image correction method will be described.

(i) method based on linear transformation

(ii) method based on histogram conversion

(iii) how to emphasize the layer structure

When performing image correction based on the linear transformation of (i), the parameters of the linear transformation are as follows using the maximum luminance I _{maxF in the} artifact region, the minimum luminance I _{minF in the} region, and the maximum luminance I _maxT in the true image region. Set it up together.

Here, y is the luminance value of the artifact region after image correction, and x is the luminance value of the artifact region before image correction. In this case, image correction is performed to match the maximum luminance in the artifact region with the maximum luminance of the true image region.

In the method of (ii), various histogram transformations are performed in order to approximate the characteristics of the histogram in the artifact region to the characteristics of the true image region. In this method, the luminance average and dispersion in the artifact area are adjusted to be the same as the luminance average and dispersion in the true image area, respectively. In this case, using the standard deviation S _f and the average value A _f of the luminance value in the artifact region, and the standard deviation S _t and the average value A _t of the luminance value in the entire image excluding the artifact region, the image correction function is as follows. You can set the parameters.

Here, x is a signal before correction and y is a signal after correction.

In addition, in the method of (iii), when the layer structure enhancement process is performed using the eigenvalues of the Hesse matrix, the conditional expressions associated with the eigenvalues λ ₁ , λ ₂ , and λ ₃ (λ ₁ ≧ λ ₂ ≧ λ ₃ ) Is as expressed in Equation 5 described above. The Hesse matrix is a square matrix made by the entire quadratic partial derivative of the multivariate function, as given in equation (3), where I is the density value of the image. From these three eigenvalues, Equation 6 above can be used to emphasize the layer structure. Note that ω (λ _s ; λ _t ) is a weight used for the layer structure enhancement process, and is set as shown in Equation 7 above. In Equation 7, however, the specific combination of values of s and t is (s, t) = (1, 3) or (2, 3), and gamma and alpha are set to fixed values, respectively.

In order to perform the layer structure enhancement process according to the thickness of the layer to be detected, the parameter s of smoothing based on a Gaussian function executed as a preprocess of the layer structure enhancement process is set as follows.

(i) The layer thickness at the layer position is calculated from the luminance profile on the A-scan line at the layer position around the artifact region. In this case, the range over the lines whose difference with the luminance value in the layer position is within a predetermined value is calculated, and the length is used as the layer thickness.

(ii) The value of the resolution s of the Gaussian filter used at the time of smoothing is set in proportion to the value of the layer thickness around the artifact region.

Note that the method of image correction in the present embodiment is not limited to these methods. For example, note that any image correction method may be used as long as the relationship of the increment function is established between the luminance value before correction and the luminance value after correction in the artifact region and includes an adjustable parameter. do.

In step S2440, based on the image correction method set in step S2430, the image correction unit 333 performs conversion (image correction) of the luminance value in the artifact region to make it easier to detect the layer position. Further, in step S2450, the layer determination unit 334 acquires the image feature of the layer to be extracted based on the luminance information of the region corrected by the image correction unit 333, and connects these feature points to set the layer position. define. For example, the retinal pigment epithelial layer is originally the region with the highest brightness on each of the A-scan lines, and the luminance is likely to increase even within the artifact region. Thus, in each A-scan line of the image corrected region, the layer position is determined by connecting pixels of maximum luminance on the positive direction side of the z axis of the layer candidate point in the x axis direction.

By the above-described procedure, the floor position when the luminance use determining unit 1911 determines that the luminance information in the artifact area is used is determined. The processing contents of the image processing method determination unit 1910 when the luminance use determination unit 1911 determines that the luminance information in the artifact area is not used will be described below.

In step S2460, the image processing method determination unit 1910 acquires information associated with the range of the artifact area calculated by the artifact area determination unit 332. More specifically, if the label of the artifact region is i in FIG. 25, the unit 1910 displays the occurrence position (x _i , y _i ) of the artifact, the width W _{i of the} artifact, and a true image in the vicinity of the region. Information associated with the number d _i of layer candidate points belonging to the area n _i is obtained. In step S2470, the unit 1910 calculates an index representing the degree of irregularities of the layer candidate point sequence existing around the artifact area. Since such an index is the same as that computed in step S2120, the detailed description is abbreviate | omitted.

In addition, in step S2480, the image processing method determination unit 1910 selects the type or order of the interpolation function used when interpolating the layer candidate point sequences between the artifact regions from the information obtained in steps S2460 and S2470. And selection of layer candidate points to be used for interpolation.

(i) First, in each artifact region, the width W _i of the artifact region, and the value of the index associated with the layer shape in the vicinity of the region calculated in step S2470 (for example, the statistic of the angle θ _i between the layer candidate points, etc.) From, select the type or order of interpolation functions. For example, if W _i is less than a predetermined value, linear interpolation is selected. If W _i is greater than or equal to a predetermined value, B spline interpolation is selected. Alternatively, the following selection method can be used. That is, if the layer-shaped irregularities are large (average or maximum of θ _i is greater than or equal to a predetermined value), the interpolation curve passes through the control points and uses natural spline interpolation, which can yield a more accurate layer position than the B spline interpolation. do. Moreover, the following selection method can also be used. That is, for the same kind of interpolation function, the order of the interpolation function is set in proportion to the size of the statistics (average value, maximum value, etc.) of the angle θ _i between the layer candidate points around the artifact region.

(ii) Then, select the layer candidate points to be used for interpolation of the layer position in the artifact area. More specifically, the required minimum needed to perform the interpolation by the true image area n _{i -1} and n the number of candidate points belonging to the layer _i d _i d _{i -1,} and the selected interpolation function in the vicinity of the artifact region i Check if the number is met. If these numbers satisfy the minimum required number, the number of layer candidate points close to the artifact area among the layer candidate points belonging to these true picture areas is selected as necessary for interpolation. On the other hand, if these numbers do not meet the minimum required number, the layer candidate points belonging to other true picture areas are selected. For example, if the number of layer candidate points present in the true picture area (eg, the true picture area n _{i +1 in} FIG. 25) is not sufficient for interpolation, the layer candidates in the neighbor area n _{i + 2} . Select more points.

If there is no true image area with a sufficient number of layer candidate points in the current interpolation direction (for example, if there is an artifact area at the edge of the image area), the layer candidate points are selected as follows. That is, the interpolation direction is changed to a direction in which there are enough layer candidate points available for interpolation, and the layer candidate points belonging to the near true image regions in that direction are selected. For example, as shown in FIG. 25, when an artifact region exists at an edge of an image and the number of layer candidate points used for interpolation in the x direction is insufficient, near the artifact region on the yz plane passing through the artifact region. It is possible to select layer candidate points of the true picture area present in the. The type of the interpolation direction is not necessarily limited to the direction parallel to the x-axis or the y-axis, and may be changed to any direction in which there are sufficient layer candidate points available for interpolation. For example, in view of the fact that the layer shape tends to have image characteristics similar to concentric shapes in the macula, optic nerve head, and the like, a true image existing near the artifact area on the plane generated by circular scanning as shown in FIG. 25. The layer candidate points of the area may be selected. Information related to the image processing method determined in this step is transmitted to the floor determination unit 334.

In step S2450, the layer determination unit 334 determines the floor position in the artifact region by interpolating between the layer candidate points selected in step S2480 using an interpolation function of the kind selected in that step. The calculated floor position information is then output to the storage unit 320.

By the above-described procedure, image processing for the artifact area corresponding to the present embodiment is executed. Note that in this embodiment, image correction is executed after the determination of the image processing method. However, the timing for performing image correction is not limited to this. For example, after the processing by the artifact area determining unit 332, the image correction unit 333 may perform image correction according to the degree of attenuation of the luminance in the area. In this case, the image processing method determination unit receives the image correction result and sets the setting related to the interpolation processing in response to the determination result by the luminance use determination unit. In addition, the image correction method setting unit 1913 is included in the image correction unit 333.

According to the above-described configuration, the image processing apparatus 10 specifies the artifact region and determines whether to use luminance information such as an edge in the artifact region, for example. For example, when luminance information such as an edge is used, the layer determination process is performed after correcting the luminance value of the area. When the information is not used, the layer position can be calculated with high accuracy by performing interpolation processing in accordance with the range of the artifact region and the layer shape around the region.

[Example 9]

In the present embodiment, instead of obtaining the position of the layer in the artifact region using only the tomogram as in the seventh embodiment, the projection image is created from the tomogram of the eye to be examined, and the positional information of the tissue or lesion site extracted from the projection image is obtained. Backprojection on the tomogram narrows the artifact candidate region.

In this embodiment, the following matters are covered.

(i) Mapping the position information of the blood vessel (bleeding) region calculated from the projection image onto the tomogram, and searching and specifying the easy portion of the artifact region in the peripheral region of the mapped region, thereby increasing the range of the artifact region with higher precision. Calculate.

(ii) The layer model is applied by weighting the evaluation function in consideration of the edge information remaining in the artifact region and the unevenness of the layer shape around the artifact region, so that a more accurate floor position can be calculated.

Since the configuration of the apparatus connected to the image processing apparatus 10 according to the present embodiment is the same as that of the seventh embodiment, the description thereof is omitted. 26 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. The image processing unit 2601 of this embodiment is different from the image processing unit 1901 shown in FIG. 19 of the seventh embodiment in that it includes a projection image generating unit 335 and a feature extraction unit 336. Since the rest of the units are the same as in Fig. 19, the description thereof is omitted.

With reference to the image processing procedure shown in FIG. 27, the contents of the image processing of the present embodiment will be described below. Note that the processing of this embodiment is the same as the processing shown in Fig. 20 of the seventh embodiment except for the processing in steps S2710, S2720, and S2730. Therefore, below, the process in step S2710, step S2720, and step S2730 is demonstrated.

In step S2710, the projection image generating unit 335 generates an image by projecting a tomographic image. Since the specific generating method is the same as that described in step S1110 of FIG. 11 of the third embodiment, the description thereof is omitted. In step S2720, features such as tissues or lesions such as retinal blood vessels are extracted from the projected image generated by the projection image generating unit 335. Since the specific generating method is the same as that described in step S1120 of FIG. 11 of the third embodiment, the description thereof is omitted.

In step S2730, if the blood vessel area (x, y) on the projection image calculated in step S2720 is back projected on a tomogram, an area as shown by the dotted area 2801 in Fig. 28 (hereinafter referred to as a reverse projection area). ) Is obtained. In general, attenuation of the luminance value tends to occur on the positive side of the z axis of the retinal blood vessel. Therefore, when the projected position (in the x-y direction) of the extracted feature is back projected onto the tomogram, the back projected dotted area 2801 is more likely to include the artifact area 5. However, when the wrongly extracted region is back projected, the attenuation of the luminance in the back projected region does not occur. Even in the case of retroprojection of the correctly extracted retinal blood vessel region, there is a case that the luminance attenuation under the reverse projection region is slight and thus hardly affects layer extraction. Thus, it is determined whether artifacts occur in the reverse projection area and near the boundary of the area. When an artifact region occurs, a statistic associated with a luminance value within that region is calculated.

The determination method of the artifact region is basically the same as the method in steps S610 to S640 of the first embodiment except for the range of the layer candidate points to be calculated for the connection degree. More specifically, the calculation of the degree of connection is not performed for all the layer candidate points, but in the vicinity of the reverse projection area and the x-y direction of the area.

According to the above-described configuration, by specifying an artifact region using a tomographic image and a projected image, and applying a layer model by weighting the evaluation function in consideration of not only the layer shape around the region but also the edge information in the region, The floor position can be calculated with high precision.

[Example 10]

In the ninth embodiment, after determining by the luminance use determining unit, in the case of using luminance information such as an edge, for example, the layer position is calculated after image correction of the artifact region is performed, and the information. If not used, the floor position is calculated by interpolation.

This embodiment encompasses the following.

(i) Mapping the position information of the blood vessel (bleeding) region calculated from the projection image onto the tomogram, and searching and specifying the edge portion of the artifact region in the peripheral region of the mapped region, thereby increasing the range of the artifact region with higher precision. Calculate.

(ii) For example, in an area where information such as attenuated edges such as blood vessels remains, the floor position is more accurately calculated by detecting the floor position after converting the luminance value.

(iii) In an area where luminance is missing and no luminance information such as an edge is available, for example, the position of the layer is more accurately determined by interpolating the surrounding layer positions in consideration of the occurrence position of the artifact region and the shape of the surrounding layer. Calculate

Since the configuration of the apparatus connected to the image processing apparatus 10 according to the present embodiment is the same as that of the ninth embodiment, the description thereof is omitted. 29 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. The image processing unit 2901 of the present embodiment further includes a registration unit 337, and the image processing method determination unit 1910 sets the image correction method setting unit 1913 and the interpolation function instead of the evaluation function setting unit 1912. It differs from the ninth embodiment in that it includes a unit 1914. In the following, the contents of the image processing in the present embodiment will be described with reference to the image processing procedures shown in FIGS. 24 and 27. Note that processing other than step S2020 is the same as that in the ninth embodiment. Therefore, only the process of step S2020 is described below, and description of other steps is omitted.

In step S2020, the image processing of the artifact area is performed. Since the processing in this step is the same as in the case of steps S2410 to S2480 in Fig. 24 of the eighth embodiment, detailed description thereof is omitted.

According to the above-described configuration, the image processing apparatus 10 specifies an artifact region by using tomographic images and a projected image, and determines whether to use luminance information such as an edge in the region. When the information is used, the layer determination process is performed after correcting the luminance value of the area. When the information is not used, the layer position can be calculated with high accuracy by performing interpolation processing in accordance with the range of the artifact region and the layer shape around the region.

[Example 11]

This embodiment adds a process of narrowing the artifact candidate region in advance by backprojecting on a tomogram the position information of a tissue or a lesion site extracted from at least one of the surface image and the projected image of the eye to be examined in relation to the ninth embodiment. .

This embodiment encompasses the following matters.

(i) When a lesion site occurs that is easy to extract from the surface image, such as alum, the alum region is extracted from the surface image, the positional information is mapped onto the tomogram, and the edge portion of the artifact region in the peripheral region of the region is extracted. By searching and specifying, a range of artifact regions can be calculated with higher precision.

(ii) A more accurate floor position can be calculated by applying a layer model by weighting an evaluation function in consideration of the edge information remaining in the artifact region and the irregularities of the layer shape around the artifact region.

 The configuration of the device connected to the image processing apparatus 10 according to the present embodiment is different from that of the ninth embodiment in that the apparatus further includes the surface image pickup device 50 shown in FIG. 14. The image processing apparatus 10 according to the present embodiment is different from the ninth embodiment in that the image processing unit 3001 includes the surface image acquisition unit 315 as shown in FIG. 30. .

In the following, the contents of the image processing in the present embodiment will be described with reference to the image processing procedure shown in FIG. Note that processing other than steps S3110 to S3150 is the same as that in the ninth embodiment, and therefore description thereof is omitted. Incidentally, the steps S3110 to S3140 are the same as those of the steps S1610 to S1640 of the fifth embodiment described above, and thus description thereof is omitted.

In step S3150, image processing is performed in the artifact region. The processing in this step is basically the same as in the case of steps S2110 to S2140 of the seventh embodiment. However, in this embodiment, in step S2130, the image processing parameters may be set using information obtained from the fundus image. For example, when the brightness of the alum is very high on the fundus image, the possibility of attenuation is further increased in the positive direction of the z-axis of the alum area even on a tomogram, so that the shape is proportional to the luminance signal value of the alum area. Increase the weight of the evaluation function. As the luminance signal value of the alum area, the pixel value of the area on the fundus image may be directly referred to, or values of the area (multi-value data) obtained as a result of processing such as morphology calculation may be referred.

According to the above-described configuration, in the artifact region specified by using the surface image and the projection image, by applying the layer model by weighting the evaluation function in consideration of not only the layer shape around the region but also the luminance information in the region, The floor position can be calculated with high precision.

[Twelfth Example]

In the present embodiment, in the eleventh embodiment, after determining by the luminance use determining unit, in the case of using luminance information such as an edge, for example, the layer position is determined after image correction of the artifact region is performed, and the information is determined. If not used, the floor position is calculated by interpolation.

In the present embodiment, the following items are included, especially when artifacts are caused by alum.

(i) By mapping the positional information of the alum area calculated from the surface image onto the tomogram and searching and specifying the edge portion of the artifact area in the periphery of the mapped area, the range of the artifact area is calculated with higher precision.

(ii) For example, in a region where information of the attenuated edge, such as a blood vessel, remains, the floor position is more accurately calculated by detecting the floor position after converting the luminance value.

(iii) In an area where luminance is missing and no luminance information such as an edge is available, for example, the position of the layer is more accurately determined by interpolating the surrounding layer positions in consideration of the occurrence position of the artifact region and the shape of the surrounding layer. Calculate

Since the configuration of the apparatus connected to the image processing apparatus 10 according to the present embodiment is the same as that of the eleventh embodiment, the description thereof is omitted. 32 is a functional block diagram of the image processing apparatus 10 according to the present embodiment. In the present embodiment, the image processing method determination unit 1910 includes the image correction method setting unit 1913 and the interpolation function setting unit 1914 instead of the evaluation function setting unit 1912. It is different. In addition, the image processing procedure in this embodiment is basically the same as that of the eleventh embodiment. However, the process of step S3150 is executed as follows.

However, the processing executed in step S3150 also has the same procedure as in the tenth embodiment. That is, as shown in Fig. 24, for example, it is determined whether or not luminance information such as an edge in the artifact area is used. In the case of using this information, the layer determination process is performed after correcting the luminance value of the area. When the information is not used, the type and parameters of the interpolation function are set according to the range of the area and the layer shape around the area, and the interpolation process is performed.

According to the above-described configuration, it is determined whether or not luminance information in the area, for example, an edge, is used in the artifact area specified using the surface image and the projection image. When the information is used, the layer determination process is performed after correcting the luminance value of the area. When the information is not used, interpolation is performed in accordance with the range of the artifact area and the layer shape around the area. Thereby, a layer position can be calculated with high precision.

[Other Embodiments]

The above embodiment realizes the present invention as an image processing apparatus. However, the embodiment of the present invention is not limited to the image processing apparatus but may be implemented as software for realizing the function when executed by the CPU of the computer.

33 is a block diagram showing the basic configuration of a computer used for realizing the functions of the units of the image processing apparatus 10 as software. The CPU 3301 uses the programs and data stored in the RAM 3302 and the ROM 3303 to control the entire computer. In addition, the CPU 3301 controls the execution of a software program corresponding to each unit of the image processing apparatus 10 to realize the functions of the respective units. The RAM 3302 includes an area for temporarily storing computer programs and data loaded from the external storage device 3304, and also includes a work area necessary for the CPU 3301 to perform various processes. The function of the storage unit 320 is realized by the RAM 3302.

The ROM 3303 generally stores the computer's BIOS, setting data, and the like. The external storage device 3304 functions as a mass information storage device such as a hard disk drive, and stores an operating system, a program executed by the CPU 3301, and the like. The external storage device 3304 stores information given in the description of this embodiment, and such information is loaded into the RAM 3302 as needed. The monitor 3305 is comprised by a liquid crystal display etc., for example. For example, the monitor 3305 may display the content output from the display unit 340.

The keyboard 3306 and mouse 3307 are input devices. The operator can input various instructions to the image processing apparatus 10 using these input devices. The interface 3308 is used to exchange various data between the image processing apparatus 10 and an external device, and is composed of, for example, an IEEE1394, USB, or Ethernet? Port. Data obtained via the interface 3308 is fetched into the RAM 3302. The functions of the tomographic acquisition unit 310 and the result output unit 350 are realized via the interface 3308. Each of the above-described components are interconnected by a bus 3309.

Features of the present invention may be implemented by a computer (or devices such as a CPU or MPU, etc.) of a system or apparatus that reads and executes a program recorded in a memory device to perform the functions of the above-described embodiment (s). And to be implemented by a method comprising, for example, steps performed by a computer of a system or apparatus, for reading and executing a program recorded in a memory device to perform the functions of the above-described embodiment (s). It may be. To this end, the program is provided to the computer from various kinds of print media (eg, computer readable media), for example, functioning via a network or as a memory device.

While the invention has been described with reference to exemplary embodiments, it will be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

This application claims the priority of Japanese Patent Application No. 2009-133455, filed June 2, 2009, the entire contents of which are incorporated herein by reference.

Claims (13)

An image processing apparatus for processing a tomogram image obtained by imaging an eye to be examined with a tomography imaging device,
Layer candidate detection means for detecting layer candidates of the retina of the eye to be examined from the tomogram,
Artifact area determination means for determining an artifact area on the tomographic layer based on an image characteristic obtained by using the layer candidates, and
And image correction means for correcting the luminance value in the artifact region based on the determination result of the artifact region determination means and the image characteristic in the region. The method of claim 1,
And layer determination means for determining the position of the layer of the retina in the artifact region based on the luminance value in the artifact region corrected by the image correction means. The method of claim 2,
Further comprising luminance use determining means for determining whether to use the luminance value in the artifact region based on the magnitude of the luminance value in determining the position of the layer,
If it is determined that the luminance use determining means uses the luminance value, the layer determination means performs the determination processing of the position of the layer using the luminance value,
And the layer determining means determines the position of the layer by performing interpolation using the layer candidates in the vicinity of the artifact area when it is determined that the luminance use determining means does not use the luminance value. As an image processing apparatus which processes a tomographic image obtained by imaging an eye to be examined by a tomographic imaging device,
Layer candidate detection means for detecting layer candidates of the retina of the eye to be examined from the tomogram,
Artifact region determination means for determining an artifact region on the tomographic layer based on an image characteristic obtained by using the layer candidates, and
Layer determination means for determining the position of the layer of the retina in the artifact region based on the determination result of the artifact region determination means,
The layer determining means uses a shape model that is used to specify a layer shape included in the artifact region and is defined by a plurality of control points, and an evaluation function associated with the shape of the shape model, and the vicinity of the control point. And determine the position of the layer based on an evaluation function associated with a luminance value in. The method of claim 4, wherein
The layer determining means determines the weight of the evaluation function associated with the brightness value so that the weight of the evaluation function associated with the brightness value increases as the brightness value of the pixel in the artifact area becomes lower, or the brightness value of the pixel in the artifact area. And a weight of the evaluation function according to a ratio between luminance values of pixels other than the artifact region on the tomography layer,
The layer determining means determines the weight of the evaluation function associated with the shape so that the weight decreases as the degree of the unevenness increases as the degree of the unevenness of the shape of the layer specified by the layer candidates in the vicinity of the artifact region increases. The image processing apparatus to determine. The method according to any one of claims 1 to 5,
A picture characteristic obtained using the layer candidates includes a continuity between the layer candidates,
And the artifact region determining means determines an artifact region on the tomographic layer so as to have, as an edge point of the artifact region, a layer candidate determined to be low in connection and unconnected. The method of claim 6,
And the artifact region determining means uses, as an edge point of the artifact region, a layer candidate having a lower luminance value in an area deeper than each layer candidate among two layer candidates determined as unconnected. The method according to any one of claims 1 to 7,
An image characteristic obtained using the layer candidates includes a difference in the direction orthogonal to the A-scan direction on the tomography between the luminance values of the pixels in the regions specified by the two layer candidates,
And the artifact region determining means determines the artifact region on the tomographic layer based on the degree of difference between the plurality of luminance profiles in the A-scan direction. The method according to any one of claims 1 to 8,
Projection image generating means for adding the respective pixels on the tomography in the A-scan direction on the tomography corresponding to the depth direction of the retina to generate a projection image, and
Further comprising feature extracting means for extracting from said projected image a feature region comprising at least one of biological tissue and a lesion site in said eye,
And the artifact area determining means determines the artifact area in the vicinity of the feature area and the feature area. 10. The method of claim 9,
The feature extraction means further extracts a feature region from the surface image obtained by imaging the eye to be examined,
And the artifact region determining means determines the artifact region further using the feature region extracted from the surface image. As a control method of the image processing apparatus which processes the tomographic image obtained by image | photographing the to-be-tested eye with a tomographic imaging device,
A layer candidate detecting step of controlling layer candidate detecting means to detect layer candidates of the retina of the eye to be examined from the tomography,
An artifact area determining step of controlling an artifact area determining means to determine an artifact area on the tomographic layer based on an image feature obtained by using the layer candidates, and
And an image correction step of controlling the image correction means to correct the luminance value in the artifact area based on the determination result of the artifact area determination step and the image characteristic in the area. As a control method of the image processing apparatus which processes the tomographic image obtained by image | photographing the to-be-tested eye with a tomographic imaging device,
A layer candidate detecting step of controlling layer candidate detecting means to detect layer candidates of the retina of the eye to be examined from the tomography,
An artifact area determining step of controlling an artifact area determining means to determine an artifact area on the tomographic layer based on an image feature obtained by using the layer candidates, and
A layer determination step of controlling layer determination means to determine the position of the layer of the retina in the artifact region based on the determination result of the artifact region determination step,
In the layer determination step, a shape model, which is used to specify the shape of the layer included in the artifact region and is defined by a plurality of control points, is used, and an evaluation function associated with the shape of the shape model and the vicinity of the control point. And determine the position of the layer based on an evaluation function associated with a luminance value in. A computer program for causing a computer to function as each means of the image processing apparatus according to any one of claims 1 to 10.

Images