JPWO2019156139A1 - Image processing device, image processing method and image processing program - Google Patents

Abstract

The image processing device 20 of the present invention is a smoothing means for executing a smoothing process on three-dimensional image data, and generates a plurality of tomographic images continuous in a predetermined direction from the three-dimensional image data after the smoothing process. A front image generating means 36 for generating one front image by superimposing a plurality of tomographic images is provided. A first filter in which a smoothing means performs a Gaussian blurring process on a pixel of interest and a plurality of peripheral pixels located around the pixel of interest on a plane orthogonal to a predetermined direction to replace the brightness value of the pixel of interest. The means 34 includes a second filter means 35 that performs a moving average process on a plurality of target pixels in a desired range centered on a pixel of interest continuous in a predetermined direction to replace the brightness value of the pixel of interest. ..

Description

The present invention relates to an image processing apparatus, an image processing method, and an image processing program that generate one front image by layering a plurality of tomographic images generated from three-dimensional image data.

One of the ophthalmic diagnostic machines, there is a tomographic imaging device called OCT (Optical Coherence Tomography) that captures a tomographic image of the retina. If general OCT imaging is performed, the obtained tomographic image is captured at a speed of, for example, 40 images / second, and 100 or more images can be acquired in one examination (imaging of a certain part of the retina). In recent years, in order to grasp the morphology of retinal blood vessels, a method called OCT angiography (OCTA: Optical Coherence Tomography Angiography), which is a pseudo angiography method using this OCT, is known.

OCT angiography allows non-invasively creating images such as angiography without the use of fluorescent agents. Specifically, several consecutive images of the same part of the fundus of the eye to be inspected are taken to acquire B-scan images with a time difference of several ms, and a short-term change between the B-scan images with a time difference is measured by a blood flow change. By assuming that there is, a vascular tomographic image (motion contrast image) that depicts microvessels in the fundus is generated. One vascular tomographic image is generated from a plurality of B-scan image groups with a time difference, and this is repeated while changing the scan position in the C-scan direction (y-direction) with respect to the observation target area, so that the images are spatially continuous. Multiple vascular tomographic images can be obtained. From these continuous multiple blood vessel images, one three-dimensional data set capable of three-dimensionally capturing the blood vessel morphology in the retina is constructed. That is, the three-dimensional data set is data that three-dimensionally models the observation target region of the fundus of the eye to be examined.

From this three-dimensional data set, a vascular tomographic image is generated as a tomographic image of a plurality of xy planes continuous in the depth direction (z direction). Then, by superimposing a plurality of continuous xy-plane vascular tomographic images in a desired range, a single front image (Enface image) in which the retinal blood vessels in the range are superimposed and displayed is generated and used for various diagnoses. To do. Several types of methods have been proposed for extracting changes in blood flow, such as the OMAG (Optical Microangiography) method and the SSADA (Split-spectrum Amplitude-decorrelation Angiography) method, as shown in Patent Document 1 and Non-Patent Document 1. There is.

Special Table 2013-518695

Anqi Zhang et al., “Methods and algorithms for optical coherence tomography-based angiography: a review and comparison”, Journal of Biomedical Optics, Vol. 20 (10), 100901 (October 2015)

Here, when generating a front image for diagnosis, if there is salt grain noise (spike noise) in the region, the noise is mixed in the gap between the blood vessels, and the extraction result of the blood vessels is deteriorated. There is. Also, even if you try to generate a front image by simply averaging to reduce the influence of noise, the averaging process raises the black level of the background, which reduces the contrast of the front image and makes a diagnosis. There is a problem that a suitable front image cannot be easily obtained.

The present invention has been made in view of these points, and an object of the present invention is to provide an image processing apparatus, an image processing method, and an image processing program capable of generating a frontal image suitable for diagnosis. In particular, it is an object of the present invention to provide an image processing apparatus, an image processing method, and an image processing program capable of generating a frontal image having improved contrast and improved visibility of blood vessels.

In order to achieve the above object, first, the present invention comprises a smoothing means for executing a smoothing process on three-dimensional image data, and after smoothing a plurality of tomographic images continuous in a predetermined direction. A frontal image generation means that generates from three-dimensional image data and superimposes the plurality of tomographic images to generate one frontal image, and the smoothing means is provided with a pixel of interest and the predetermined direction. A first filter means that performs Gaussian blurring processing on a plurality of peripheral pixels located around the attention pixel on an orthogonal plane to replace the brightness value of the attention pixel, and the said one continuous in the predetermined direction. Provided is an image processing apparatus comprising: a second filter means for executing a moving averaging process on a plurality of target pixels in a desired range centered on the pixel of interest to replace the brightness value of the pixel of interest. (Invention 1).

According to the above invention (Invention 1), two different filtering processes, Gaussian blurring process for the plane direction of the tomographic image used for generating the front image and moving average processing for the depth direction, are executed to obtain three-dimensional image data. By processing, it is possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels, and with a realistic amount of calculation and calculation time, a 3D image suitable for diagnosis and A frontal image can be generated.

In a tomographic image, the resolution in the depth direction is higher than that in the plane direction, so even if moving average processing, which is easily affected by the values of adjacent pixels, is performed in the depth direction, data on minute structures will be lost. There is less risk, and as a result, it is possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels.

On the other hand, if the moving average process is performed in the plane direction, the value of the adjacent pixel greatly affects the value of one's own data, so the contrast decreases and the data of the structure disappears, or the connectivity of blood vessels There is a concern that it will get worse. Therefore, in the plane direction, it is possible to prevent data having a minute structure from being buried by adopting a process called Gauss blurring process in which the weight of its own pixel value is larger than that of surrounding pixel values.

It should be noted that the Gauss blurring process requires a much larger amount of calculation than the moving average process and takes a long time to process. Therefore, it is not preferable to perform the Gaussian blurring process in all directions from the viewpoint of the calculation amount and the calculation time. From these facts, the present invention improves smoothness without impairing the connectivity of blood vessels by executing two different filtering processes, a Gaussian blurring process in the plane direction and a moving average process in the depth direction. It is possible to reduce the black level of the background, and it is configured so that a three-dimensional image or a front image suitable for diagnosis can be generated with a realistic calculation amount and calculation time.

In the above invention (Invention 1), the front image generating means selects and selects a predetermined number of pixels in descending order of luminance value from a plurality of pixels at the same position of each of the plurality of tomographic images. It is preferable to generate the front image by setting the average value of the brightness values of a predetermined number of pixels as the brightness value at the position of the front image (Invention 2).

Further, in the above inventions (1 and 2), it is preferable to further provide a noise removing means for executing a noise removing process on the three-dimensional image data before the smoothing process (Invention 3).

Secondly, the present invention comprises a smoothing step of executing a smoothing process on three-dimensional image data, and generating a plurality of tomographic images continuous in a predetermined direction from the three-dimensional image data after the smoothing process. A front image generation step of layering a plurality of tomographic images to generate one front image is provided, and in the smoothing step, the attention pixel and the attention pixel on a plane orthogonal to the predetermined direction. A Gaussian blurring process is performed on a plurality of peripheral pixels located around the image to replace the brightness value of the pixel of interest, and a plurality of objects in a desired range centered on the pixel of interest continuous in a predetermined direction. Provided is an image processing method characterized by executing a moving averaging process on a pixel to replace the brightness value of the pixel of interest (Invention 4).

According to the above invention (Invention 4), two different filtering processes, Gaussian blurring process for the plane direction of the tomographic image used for generating the front image and moving average process for the depth direction, are executed to obtain three-dimensional image data. By processing, it is possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels, and with a realistic amount of calculation and calculation time, a 3D image suitable for diagnosis and A frontal image can be generated.

In a tomographic image, the resolution in the depth direction is higher than that in the plane direction, so even if moving average processing, which is easily affected by the values of adjacent pixels, is performed in the depth direction, data on minute structures will be lost. There is less risk, and as a result, it is possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels.

On the other hand, if the moving average process is performed in the plane direction, the value of the adjacent pixel greatly affects the value of one's own data, so the contrast decreases and the data of the structure disappears, or the connectivity of blood vessels There is a concern that it will get worse. Therefore, in the plane direction, it is possible to prevent data having a minute structure from being buried by adopting a process called Gauss blurring process in which the weight of its own pixel value is larger than that of surrounding pixel values.

It should be noted that the Gauss blurring process requires a much larger amount of calculation than the moving average process and takes a long time to process. Therefore, it is not preferable to perform the Gaussian blurring process in all directions from the viewpoint of the calculation amount and the calculation time. Based on these facts, the present invention improves smoothness without impairing the connectivity of blood vessels by executing two different filtering processes, Gaussian blurring process in the plane direction and moving average process in the depth direction. It is possible to reduce the black level of the background, and it is configured so that a three-dimensional image or a front image suitable for diagnosis can be generated with a realistic calculation amount and calculation time.

In the above invention (Invention 4), in the front image generation step, a predetermined number of pixels are selected and selected from a plurality of pixels at the same position of each of the plurality of tomographic images in descending order of brightness value. The image processing method according to claim 4, wherein the front image is generated by using the average value of the brightness values of a predetermined number of pixels as the brightness value at the position of the front image (the image processing method according to claim 4 is preferable). Invention 5).

Further, in the above inventions (Inventions 4 and 5), it is preferable to further include a noise removing step of executing a noise removing process on the three-dimensional image data before the smoothing process (Invention 6).

Thirdly, the present invention is for causing the computer to function as an image processing device according to any one of the inventions 1 to 3, or for causing the computer to execute the image processing method according to any one of the inventions 4 to 6. (Invention 7).

According to the image processing apparatus, the image processing method, and the image processing program of the present invention, a front image suitable for diagnosis can be generated.

It is a block diagram which shows the whole structure of the image processing apparatus which concerns on one Embodiment of this invention. It is explanatory drawing which shows the state which acquires the fundus tomography image by scanning the fundus. It is explanatory drawing which shows the state which a plurality of fundus tomographic images of the same place of the fundus are acquired with a time lag, and a vascular tomographic image is generated. It is a flow chart which shows the flow of the whole image processing in this embodiment. It is a flow figure which shows the flow of the noise removal processing in this embodiment. It is a flow figure which shows the flow of the smoothing process in this embodiment. It is explanatory drawing (the 1) which shows typically the relationship between the pixel of interest and the adjacent pixel in the noise removal processing by the image processing apparatus which concerns on this embodiment. It is explanatory drawing (the 2) which shows typically the relationship between the pixel of interest and the adjacent pixel in the noise removal processing by the image processing apparatus which concerns on this embodiment. (A) is an explanatory diagram schematically showing the relationship between the pixel of interest and the peripheral pixel in the smoothing process by the image processing apparatus according to the present embodiment, and (b) is a schematic diagram showing the relationship between the pixel of interest and the target pixel. It is explanatory drawing shown in. It is explanatory drawing (the 1) which shows typically the mode that the front image is generated by the image processing apparatus which concerns on this embodiment. It is explanatory drawing (the 2) which shows typically the mode that the front image is generated by the image processing apparatus which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, a tomographic image of the fundus E of the eye to be inspected is acquired by a tomographic image capturing apparatus (OCT), and one three-dimensional image capable of three-dimensionally capturing the vascular morphology in the fundus by OCT angiography from the obtained tomographic image. The data is constructed, and image processing is performed on the three-dimensional image data to generate a single front image in which blood vessels are superimposed and displayed in the depth direction. The image processing produces a diagnostically suitable frontal image with improved contrast and improved visibility of blood vessels. The image to be image-processed in the present invention is not limited to three-dimensional image data capable of three-dimensionally capturing the morphology of blood vessels in the fundus, and another object is photographed by another type of device. It can also be applied to the constructed 3D image data.

FIG. 1 is a block diagram showing the entire system that acquires and processes a tomographic image of the fundus of the eye to be inspected. The tomographic image photographing apparatus 10 is an apparatus (OCT: Optical Coherence Tomography) for photographing a tomographic image of the fundus of the eye to be inspected, and operates by, for example, a Fourier domain method. Since the tomographic image capturing apparatus 10 is known, detailed description thereof will be omitted. However, the tomographic imaging apparatus 10 is provided with a low coherence light source, and the light from the low coherence light source is divided into reference light and signal light. To. As shown in FIG. 2, the signal light is raster-scanned on the fundus E, for example, in the x and y directions. The signal light scanned and reflected by the fundus E is superimposed on the reference light reflected by the reference mirror to generate interference light, and an OCT signal indicating information in the depth direction (z direction) of the fundus is generated based on the interference light. To do.

The image processing device 20 has a control unit 21 realized by a computer composed of a CPU, RAM, ROM, and the like, and the control unit 21 controls the entire image processing by executing an image processing program. Further, the image processing device 20 is provided with a tomographic image forming unit 22, a storage unit 23, a vascular tomographic image forming unit 24, a three-dimensional data construction unit 25, a display unit 26, and an operation unit 27.

The tomographic image forming unit 22 is realized by a dedicated electronic circuit that executes a known analysis method such as the Fourier domain method, or an image processing program executed by the CPU described above, and is an OCT signal generated by the tomographic image capturing apparatus 10. A tomographic image of the fundus of the eye to be examined is formed based on.

For example, as shown in FIG. 2, when the fundus E is scanned in the x direction at the position of y _N (N = 1, 2, ..., T) in the y direction, the scan is performed a plurality of times (at the time of the scan). M times) Sampling is performed. At each sampling point in the x direction, tomographic images (A scan images) A _h (h = 1, 2, ..., M) in the z direction are acquired, and from these m A scan images A _h. A tomographic image _BN (N = 1, 2, ..., T) is formed. Since the A-scan image is stored, for example, with a width of 1 pixel in the x direction and a length of n pixels in the z direction, the tomographic image _BN is an image having a size of m × n pixels (m pixels in the x direction). (Length, length of n pixels in the z direction), which is also called a B-scan image.

When the B-scan image is formed several times (for example, i times) continuously at each position of y _N (N = 1, 2, ..., T) with a time difference of several ms, i is performed at each same location. B-scan images B _NI (N = 1, 2, ..., T) (I = 1, 2, ..., I) with a time difference of several ms are formed. The t × i B-scan images _BNI formed by the tomographic image forming unit 22 are stored in the storage unit 23 configured by, for example, a semiconductor memory, a hard disk device, or the like. The storage unit 23 also stores the above-mentioned image processing program and the like.

The vascular tomographic image forming unit 24 is realized by a dedicated electronic circuit that executes a known method for extracting blood flow changes such as the OMAG method and the SSADA method, or an image processing program executed by the CPU described above, and is a tomographic tomography. A vascular tomographic image depicting microvessels of the fundus at that position based on a plurality of tomographic images (B scan image _BNI ) with a time difference of several ms with respect to the same location on the fundus of the eye to be inspected formed by the image forming unit 22. Form F _N (N = 1, 2, ..., T).

For example, as shown in FIG. 3, i B-scan images B _{(N-1) I} (I = 1, 2, ..., I) having a time difference of several ms at the position of y _N-1. vascular tomographic image _{F T-1} is, _y time difference of several ms at the position of the _N a i sheets of B-scan image _{B NI (I = 1,2, ·····} , i) vascular tomographic image F from the _From the i B scan images B _{(N + 1) I} (I = 1, 2, ..., I) with a time difference of several ms at the position where _N is y _{N + 1,} the vascular tomographic image F _{N + 1} is the blood vessel. Each is formed by the tomographic image forming unit 24. In this way, t vascular tomographic images F _N (N = 1, 2, ..., T) are formed at each position of y _N (N = 1, 2, ..., T). And stored in the storage unit 23.

Three-dimensional data constructing unit 25, as shown in FIG. 3, the t pieces of blood vessel tomographic image F _N consecutive vascular tomographic image forming unit spatially formed by 24, sterically capture that vessels form in the retina One three-dimensional image data G that can be created is constructed and stored in the storage unit 23. The three-dimensional image data G is data in which the observation target region of the fundus of the eye to be examined is three-dimensionally modeled so that the blood vessel morphology in the retina can be captured three-dimensionally.

The image processing device 20 is provided with an image processing unit 30. The image processing unit 30 includes a calculation means 31 as a noise removing means, a counting means 32 and a replacing means 33, a first filter means 34 and a second filter means 35 as a smoothing means, and a front image generating means 36. , Performs various image processing on the three-dimensional image data G.

As will be described later, the calculation means 31, the counting means 32, and the replacement means 33 are constructed by the three-dimensional data construction unit 25, and perform noise removal processing on the three-dimensional image data G stored in the storage unit 23. The calculation means 31 calculates the difference value between the brightness value of the pixel of interest in the image to be processed and the brightness value of each of the plurality of adjacent pixels located around the pixel of interest, and the counting means 32. Counts the number of adjacent pixels whose calculated difference value is equal to or less than a predetermined threshold (brightness difference threshold) as the number of non-applicable adjacent pixels, and the replacement means 33 counts the number of non-applicable adjacent pixels as a predetermined number (non-applicable allowable). If the number is less than or equal to (number), the brightness value of the pixel of interest is replaced with the average value of the brightness values of a plurality of adjacent pixels.

First filter means 34 and second filter means 35 is adapted to perform smoothing processing on the three-dimensional image data G ₁ after the noise removal processing, the first filter means 34, to be processed three-dimensional and target pixel in the image data G _1, the luminance of the plurality of the pixel of interest by performing a Gaussian blurring process with respect to the peripheral pixels located around the target pixel on a plane perpendicular to the predetermined direction (depth direction) Replacing the values, the second filter means 35 executes a moving averaging process on a plurality of target pixels in a desired range centered on the pixels of interest that are continuous in a predetermined direction (depth direction), and the brightness of the pixels of interest. Replace the value.

Further, the front image generation unit 36 generates a plurality of tomographic images continuous to the smoothing process after the three-dimensional image data G ₂ from a predetermined direction (depth direction), and overlaid the plurality of tomographic images To generate one front image, a predetermined number of pixels are selected from a plurality of pixels at the same position in each of the plurality of tomographic images in descending order of brightness value, and a predetermined number of selected pixels are selected. A front image is generated by setting the average value of the brightness values of the pixels as the brightness value at the position of the front image. Each means or each image processing in the image processing unit 30 is realized by using a dedicated electronic circuit or by executing an image processing program.

The display unit 26 is composed of, for example, a display device such as an LCD, and displays various images stored in the storage unit 23, various images generated or processed by the image processing device 20, and accompanying information such as information about the subject. To do.

The operation unit 27 has, for example, a mouse, a keyboard, an operation pen, a pointer, an operation panel, etc., and is used for selecting an image displayed on the display unit 26 or for the operator to give an instruction to the image processing device 20 or the like. Be done.

Subsequently, a flow of executing various image processes by the image processing unit 30 on the three-dimensional image data will be described with reference to the flow chart of FIG. Note that S100 to S400 in FIG. 4A correspond to steps 100 to 400 in the description of the flow described below, and S101 to S107 in FIG. 4B correspond to steps 101 to 107 in FIG. 4C. S201 to S205 in the inside correspond to steps 201 to 205.

In the present embodiment, first, image processing for removing noise from the constructed three-dimensional image data G is performed (step 100). The three-dimensional image data G is a set of t spatially continuous vascular tomographic images F _N (N = 1, 2, ..., T), and is one of the vascular tomographic images F _N. was processed image F _T, the noise removal process is used in conjunction also vascular tomographic image F _T-1 and vascular tomographic image F _{T + 1} before and after. As shown in FIG. 5B, the processing target image _FT is an image having a size of m × n pixels, and the vascular tomographic image _FT-1 and the vascular tomographic image _{FT + 1} are the processing target images F. a tomographic image that is continuous with _T, an image similar to the target image F _T having a size of m × n pixels. The vascular tomographic image _FT-1 , the image to be processed _FT, and the vascular tomographic image _{FT + 1} are all stored in the storage unit 23.

First calculating means 31, as shown in FIG. 5, the processing pixel of interest in the target image F _T and one set as a target pixel Q, the processing target image F _T and the tomographic image F _T-1 and before and after the A total of 26 pixels spatially located around the pixel of interest in the tomographic image _{FT + 1} are set as adjacent pixels q _i (i = 1-26) (step 101). That is, adjacent pixels q _i, in addition to a total of eight pixels of the pixel positioned in contact with the four corners of the pixel and the target pixel located at the upper, lower, left and right of the target pixel Q in the processing target image F _T, vascular tomographic image F _{T -1} and a total of 26 pixels, including a total of 18 pixels to be positioned around the spatially target pixel Q when the blood vessel tomographic image F _{T + 1} arranged in the processing target image F _T. 5, the processing object fourth column from the left of the image F _T, and the target pixel Q, which is set in the third row from the top, the processing target image F _T and longitudinal vascular tomographic image F _T-1 and vascular tomographic thereof The relationship between 26 adjacent pixels q _i spatially located around the pixel of interest Q in the image _{FT + 1} is shown. Also shows the processing target image F _T and before and after vascular tomographic image F _T-1 and vascular tomographic image F _{T + 1} the pixel of interest Q from the state arranged state cut out only adjacent pixels q _i in FIG.

Incidentally, not necessarily adjacent pixels is 26 set in the case of setting the end portion of the pixel of interest Q in the processing target image F _T, when the target pixel Q is set to an end in the processing target image F _T , The number of adjacent pixels may be appropriately set to 26 or less according to the position of the pixel of interest. For example the first column the pixel of interest from the left of the processing target image F _T, when it is set in the third row from the top, above the target pixel Q, down, right, upper right, located lower right 5 A total of 17 pixels, which is a combination of the pixel, the pixel Q of interest in the continuous front and rear vascular tomographic images, and 12 pixels located adjacent to the five pixels, may be set as adjacent pixels. Further, even when there is no continuous vascular tomographic image before and after, the number of adjacent pixels may be appropriately set to 26 or less according to the position of the pixel of interest. Further, the method of setting adjacent pixels may be appropriately changed according to the type of three-dimensional image data to be processed, the purpose of noise removal, and the like.

Further, depending on the type of image to be processed, the purpose of noise removal, etc., up to the vascular tomographic image _FT-1 and the vascular tomographic image _{FT + 1} , which are continuous before and after the vascular tomographic image _FT-2 and the vascular tomographic image _{FT + 2.} May be used as the setting target of the adjacent pixel, and the setting of the adjacent pixel may be extended to the range of 5 × 5 instead of the range of 3 × 3, or may be set up, down, left and right of the pixel of interest. _Two pixels located adjacent to the pixel of interest in the anterior and posterior vascular tomographic images that are continuous with the four pixels (for example, the four pixels q ₂ , q ₄ , q ₅ , and q ₇ in FIG. 5). The method of setting the adjacent pixels may be appropriately changed, for example, six pixels including the pixels (for example, the two pixels of q ₁₃ and q ₂₂ in FIG. 5) are set as adjacent pixels.

Then calculating unit 31, a luminance value D of the pixel of interest Q, 26 pieces of luminance values _d i (i = _1~26) the processing target image _{F T} adjacent pixels _{q i,} vascular tomographic image _{F T-1} and It is acquired from the original image data of the vascular tomographic image _{FT + 1} (step 102).

After obtaining the luminance value d _i of the luminance value D and the adjacent pixels q _i of the target pixel Q, calculating means 31, the first of the brightness value D of the pixel of interest Q obtained by subtracting the luminance value d _i of each adjacent pixel q _i the difference value is calculated for each of the 26 contiguous pixels _{q i} (step 103a). This first difference value is calculated for the number of adjacent pixels and is used to estimate whether or not the pixel of interest Q is white point noise.

Subsequently, the white point of the pixel of interest Q is determined. Specifically, the counting means 32 compares whether or not the calculated first difference value is equal to or less than the first luminance difference threshold PKT stored in the storage unit 23 in advance, and the first luminance difference threshold PKT. The following number of adjacent pixels is counted as the number of first non-applicable adjacent pixels (step 104a). In other words, conditional expression: number of neighboring pixels satisfying the luminance values d _i ≦ PKT luminance value D- adjacent pixels q _i of the target pixel Q is the first unmatched number of adjacent pixels.

After counting the number of first non-applicable adjacent pixels, the replacement means 33 determines whether or not the number of first non-applicable adjacent pixels is equal to or less than the first non-applicable allowable number PFT stored in the storage unit 23 in advance. (Step 105a). If the number of first non-applicable adjacent pixels is equal to or less than the first non-applicable allowable number PFT, it is determined that the pixel of interest Q is white point noise.

On the other hand, if the first non-relevant number of adjacent pixels is not less than the first non-relevant allowable number PFT, calculating means 31, second by subtracting the luminance value D of the pixel of interest Q from the luminance value d _i of each adjacent pixel q _i the difference value is calculated for each of the 26 contiguous pixels _{q i} (step 103b). Like the first difference value, this second difference value is also calculated by the number of adjacent pixels, and is used to estimate whether or not the pixel of interest Q is black point noise.

Subsequently, the black point of the pixel of interest Q is determined. Specifically, the counting means 32 compares whether or not the calculated second difference value is equal to or less than the second luminance difference threshold HLT stored in the storage unit 23 in advance, and compares the second luminance difference threshold HLT. The following number of adjacent pixels is counted as the number of second non-applicable adjacent pixels (step 104b). That is, the conditional expression: the luminance value d _i adjacent pixels q _i - the number of neighboring pixels satisfying the luminance value D ≦ HLT of the pixel of interest Q is the second non-relevant number of adjacent pixels.

After counting the number of second non-applicable adjacent pixels, the replacement means 33 determines whether or not the number of second non-applicable adjacent pixels is equal to or less than the second non-applicable allowable number HFT stored in the storage unit 23 in advance. (Step 105b). If the number of second non-applicable adjacent pixels is equal to or less than the second non-applicable allowable number HFT, it is determined that the pixel of interest Q is black dot noise.

As a result of the determination in steps 105a and 105b, when it is determined that the pixel of interest Q is white point noise (when the number of first non-applicable adjacent pixels is equal to or less than the first non-applicable allowable number PFT), the pixel of interest Q is black. If it is determined that the point noise (when the second non-relevant number of adjacent pixels is less than or equal to a second non-relevant allowable number HFT), substitution unit 33, an average value of luminance values d _i of each adjacent pixel q _i Calculate (step 106). Then, an image in which the brightness value of the pixel of interest Q is replaced with the calculated average value is generated (step 107) and stored in the storage unit 23. As a result of the determination in step 105a, the number of first non-applicable adjacent pixels is not less than or equal to the first non-applicable allowable number PFT, and as a result of the determination in step 105b, the number of second non-applicable adjacent pixels is not less than or equal to the second non-applicable allowable number HFT. If not, the brightness value of the pixel of interest Q is left as it is without being replaced with the average value.

The processing described above, in the processing target image F _T, for example, repeated while setting the order all the pixels from the pixels of the upper left end as the target pixel, the image subjected to noise removal processing in the overall final processed target image F _T Will be generated. When the above-described noise removal processing to the t pieces of blood vessel tomographic image F _N constituting the three-dimensional image data is performed, the three-dimensional image data G ₁ after the noise removal processing is stored in the storage unit 23. It is not always necessary to process all vascular tomographic images, and an arbitrary number of vascular tomographic images can be selected according to the type of three-dimensional image data to be processed, the purpose of noise removal, and the like.

As described above, in the noise removal processing in the present embodiment, the pixel of interest in the image to be processed is compared with the adjacent pixel located around the pixel of interest, and the brightness value of the pixel of interest is a predetermined value rather than the adjacent pixel. When it is brighter or darker than this, it is replaced with the average value of adjacent pixels. If the difference in brightness between the adjacent pixel used for replacement and the pixel of interest is increased, that is, if the above-mentioned first luminance difference threshold and second luminance difference threshold are set large, the number of pixels of interest to be replaced decreases, and conversely, if the difference is set small. The number of pixels of interest to be replaced increases.

On the other hand, in the determination of replacement, the strictest determination formula is used to determine whether the pixel of interest is brighter than a predetermined value or more than all adjacent pixels, or whether the pixel of interest is darker than a predetermined value or more than all adjacent pixels. It can also be used. In this case, by setting the first non-applicable number and the second non-applicable allowable number to 0, the pixel of interest is brighter than all the adjacent pixels by a predetermined value or more, or the pixel of interest is brighter than all the adjacent pixels. However, only when it is darker than a predetermined value, the process of replacing the brightness value of the pixel of interest with the average value of the brightness values of the adjacent pixels is performed. The smaller the first non-applicable allowable number and the second non-applicable allowable number, the smaller the number of pixels of interest to be replaced with the average value, and the more the first non-applicable allowable number and the second non-applicable allowable number are, the more they are replaced. The number of pixels of interest increases.

The same value may be used for the first luminance difference threshold value and the second luminance difference threshold value, or different numerical values may be set individually. Further, the same value may be used for the first non-applicable number and the second non-applicable number, or different numerical values may be set individually. The optimum values to be stored in the storage unit 23 in advance may be examined and applied in advance depending on the quality of the image and the properties (shape, brightness, size, etc.) of the structure to be extracted. it can.

In particular, if white point noise is mixed into the gap between blood vessels, the extraction result of blood vessels is deteriorated. However, by performing the above noise removal processing, the pixel of interest does not have bright pixels around it and has independent high brightness. In the case of a pixel, it can be determined that the pixel of interest is a white point noise pixel and can be replaced with the average value of adjacent pixels. If the first non-applicable number is increased, even if there are several high-luminance pixels in the vicinity, the pixel of interest can be determined to be a white point noise pixel and the adjacent pixel can be replaced with the average value. , The noise removal effect can be enhanced and the black level of the background can be improved, but on the other hand, microvessels are less likely to be extracted.

Although both the white point noise and the black point noise are removed in the above-mentioned noise removal processing, it is not always necessary to remove both of them, and it is white depending on the type of three-dimensional image data to be processed and the purpose of noise removal. Either one of the point noise and the black point noise may be removed.

Then performing image processing for smoothing the three-dimensional image data G ₁ after the noise removal processing (step 200). First filter means 34 First, as shown in FIG. 7 (a), one sets the pixel of interest in a three-dimensional image data G ₁ as the target pixel P (step 201), XY plane that exist in the target pixel P a total of eight pixels located around the pixel of interest P in the above set as a peripheral pixel _p i (i = _1~8) (step 202). Incidentally, FIGS. 7 (a) and (b), by cutting a portion from the three-dimensional image data G _1, 3 pixels in the x direction, three pixels in the y direction, only total 45 pixels of five pixels in the z-direction schematically It is displayed as a conversion.

Then the first filter means 34 performs the Gaussian blurring process to target a total of nine pixel of interest P and eight surrounding pixels p _i, replaces the luminance value of the target pixel P (step 203). The Gauss blurring process is a filtering process in which the pixel values in the vicinity are weighted according to the distance from the pixel of interest and the pixel values are averaged. Specifically, acquires luminance value e _i of the pixel of interest P and eight surrounding pixels p _i a _(i = 1 to 8) from the original image data, and increase the weight closer to the target pixel P, the target pixel P from consisting as to be smaller weights far, averaging by weighting the luminance value of the pixel of interest P and eight surrounding pixels p _i using a Gaussian distribution function as shown in equation 1 below, the target pixel P Let it be the brightness value.

For example, to target a total of nine pixel of interest P and eight surrounding pixels p _i, it is possible to use the following formula 2 as a kernel K to be used for weighting.

When weighting at a rate corresponding to the respective positions of the peripheral pixel _{p i} with kernel K described above, for example peripheral pixels _p 1 shown in FIG. _7, p _3, p 6, the luminance values of _{p 8 e} 1, _{e 3} , the weight of _e 6, _{e 8} 1/50, the weight of the peripheral pixel _{p 2, p 4, p 5} , the luminance value of _{p 7 e 2, e 4,} e 5, e 7 is 6/50, attention The brightness value of the pixel P is weighted by 22/50 to calculate an average value, and this average value is used as the brightness value of the pixel P of interest.

Followed by a second filter means 35 is set, five pixels around the target pixel P is continuous in the Z-direction, i.e. the pixel of interest pixel P and pixel C _{i (i} = 1~4) shown in FIG. 7 (step 204), executes moving average processing on the target pixel of interest P and the pixel _C i (i = 1 to 4), replaces the luminance value of the target pixel P (step 205). Specifically, to get the luminance value of the target pixel P and the pixel C _{i (i} = 1~4) from the original image data, averaged simply the obtained luminance values on the luminance value of the pixel of interest P.

The above process, the three-dimensional image in the data G _1, repeated while setting the order all the pixels as a target pixel, finally the three-dimensional image data G ₁ of performed smoothing process in the entire three-dimensional image data G ₂ Is generated and stored in the storage unit 23. Incidentally, not necessarily set for all the pixels of the three-dimensional image data G ₁ as the target pixel, it is not necessary to the smoothing process, the three-dimensional image data to be processed type and the smoothing process in accordance with the purpose or the like of the smoothing The target range can be set arbitrarily.

Thus, the three-dimensional image Gaussian blur processing for the planar direction (xy direction) of the data G _1, perform two different filtering process of the moving average processing for the depth direction (z direction) the three-dimensional image data G By processing ₁ , a three-dimensional image or a frontal image suitable for diagnosis can be generated.

In a tomographic image, the resolution in the depth direction is higher than that in the plane direction, so even if moving average processing is performed in the depth direction, the smoothness is improved and the black level of the background is lowered without impairing the connectivity of blood vessels. It is possible. On the other hand, if the moving average processing is performed in the plane direction, the contrast is lowered and the blood vessel connectivity is deteriorated. Therefore, by adopting the Gauss blur processing in the plane direction, data of a minute structure is buried. It is possible to prevent it from being stored.

Since the Gauss blurring process requires a much larger amount of calculation and takes more time than the moving average process, it is not preferable to perform the Gaussian blurring process in all directions from the viewpoint of the calculation amount and the calculation time. Therefore, as will be described later, three-dimensional image data after smoothing j xy plane tomographic images _HL (L = 1, 2, ..., J) continuous in the depth direction (z direction). generated from G _2, two in generating one of the front image E by overlaid the j pieces of xy plane tomographic image H _L, Gaussian blurring process for the planar direction, that the moving average processing for the depth direction By performing two different filtering processes, it is possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels, and it is suitable for diagnosis with a realistic amount of calculation and calculation time. It is possible to generate a three-dimensional image or a front image. In particular, as in the present embodiment, when a front image is generated after smoothing the three-dimensional image data after the noise removal processing, the front image with improved contrast and extremely good blood vessel visibility is obtained. Can be obtained.

When the pixel P of interest is set at the end in the three-dimensional image data G ₁ , eight peripheral pixels are not necessarily set, and the pixel P of interest is set at the end in the three-dimensional image data G ₁ . In this case, the number of peripheral pixels may be appropriately set to 8 or less according to the position of the pixel of interest.

Further, the kernel used in the Gauss blurring process by the first filter means 34 does not necessarily have to be a combination of the above-mentioned rates in the range of 3 × 3, depending on the type of three-dimensional image data to be processed, the purpose of smoothing, and the like. For example, kernels with different rate combinations in the range of 5 × 5 may be used. If Gaussian blurring processing is performed using a kernel in the range of 5 × 5, a total of 24 pixels located around the pixel of interest on the xy plane in which the pixel of interest exists are set as peripheral pixels.

Further, the number of target pixels to be subjected to the moving average processing by the second filter means 35 does not necessarily have to be five, and as long as the pixels in the range centered on the pixel of interest are the target pixels, the third order to be processed is used. Depending on the type of the original image data, the purpose of smoothing, etc., the number of target pixels may be a desired number such as 3, 5, or 7.

Next, a process of generating a front image E from the three-dimensional image data G ₂ after the smoothing process will be described. The front image generating means 36 generates j xy plane tomographic images _HL (L = 1, 2, ..., J) continuous in the z direction from the three-dimensional image data G ₂ after the smoothing process. (Step 300). Incidentally, xy plane tomographic image H _L need not necessarily be generated through the 3D image data G ₂ as the target, set to an arbitrary portion to be observed as a diagnosis target in a three-dimensional image data G _2, the portion It is possible to generate a plurality of xy plane tomographic images _HL only for the target.

Subsequently, as shown in FIG. 8, j xy plane tomographic images _HL are layered to generate one front image E (step 400). At this time, the maximum brightness value is simply selected from the brightness values of a plurality of pixels (pixels marked in FIG. 8) at the same position in each of the j xy plane tomographic images _HL , and the front image is displayed. The brightness value of the position of E may be used, but in the present embodiment, a predetermined number of pixels are selected from a plurality of pixels at the same position of each of the j xy plane tomographic images _{HL in} descending order of the brightness value. Then, the front image E is generated by setting the average value of the brightness values of the selected predetermined number of pixels as the brightness value at the position of the front image E. A schematic representation of this process is shown in FIG.

FIG. 9 schematically shows a process of setting the average value of the brightness values of the selected predetermined number of pixels as the brightness value at the position of the front image E, and (a) shows a thick blood vessel at the position. In some cases, (b) indicates that there is a thin blood vessel at that position, and (c) indicates that there is a noise emission spot at that position. For example, when there are 12 xy plane tomographic images _HL , the state in which 12 pixels at the same position are arranged in a row as they are is shown on the left side of each of (a), (b), and (c) of FIG. Has been done. These are rearranged in descending order of luminance value, and are shown on the right side of each of (a), (b), and (c) in FIG. Of the 12 pixels rearranged in this way, five pixels are selected from the top in descending order of brightness value, and the average value of the brightness values of these five selected pixels is the brightness value at the position of the front image E. By doing so, the front image E is generated.

As described above, when j xy plane tomographic images _HL are layered to generate one front image E, the maximum brightness value among the pixel values of a plurality of pixels at the same position is simply set as the front image E. If a predetermined number of pixels are selected in descending order of the brightness value and the average value of the brightness values of the selected predetermined number of pixels is used as the brightness value of the position of the front image E instead of the brightness value of the position. For example, if there is a thick blood vessel at the position as shown in FIG. 9A, the brightness value at the position in the front image E becomes large, and the blood vessel can be clearly seen in the front image E. .. Further, as shown in FIG. 9B, if there is a thin blood vessel at the position, the brightness value of the position of the front image E is calculated by reflecting the thin blood vessel, and the brightness level of the microblood vessel is maintained also in the front image E. As shown in FIG. 9C, even if there is a noise bright spot at the position, if the brightness value of the other selected pixels is small, the position of the front image E can be averaged. The brightness value becomes small, and the sesame salt noise in the front image E can be reduced. As a result, the black level of the background can be lowered, and a front image suitable for diagnosis can be generated.

Although the image processing apparatus, the image processing method, and the image processing program according to the present invention have been described above based on the drawings, the present invention is not limited to the above embodiment, and various modifications can be made.

The present invention can be used as a method for generating a single front image by layering a plurality of tomographic images generated from three-dimensional image data.

10 Tomographic image imaging device 20 Image processing device 21 Control unit 22 Tomographic image forming unit 23 Storage unit 24 Vascular tomographic image forming unit 25 Three-dimensional data construction unit 26 Display unit 27 Operation unit 30 Image processing unit 31 Calculation means 32 Counting means 33 Replacement Means 34 First filter means 35 Second filter means 36 Front image generation means

Claims (7)

A smoothing means that executes smoothing processing on 3D image data,
A frontal image generation means that generates a plurality of tomographic images continuous in a predetermined direction from three-dimensional image data after smoothing, and superimposes the plurality of tomographic images to generate one frontal image. Prepare,
The smoothing means
A first filter means for replacing the brightness value of the pixel of interest by executing Gaussian blurring processing on the pixel of interest and a plurality of peripheral pixels located around the pixel of interest on a plane orthogonal to the predetermined direction. ,
Having a second filter means for performing a moving average process on a plurality of target pixels in a desired range centered on the attention pixel continuous in a predetermined direction to replace the brightness value of the attention pixel. An image processing device as a feature. The front image generating means selects a predetermined number of pixels from a plurality of pixels at the same position of each of the plurality of tomographic images in descending order of the brightness value, and the brightness value of the selected predetermined number of pixels The image processing apparatus according to claim 1, wherein the front image is generated by using the average value as the brightness value of the position of the front image. The image processing apparatus according to claim 1 or 2, further comprising a noise removing means for executing a noise removing process on the three-dimensional image data before the smoothing process. A smoothing process that executes smoothing processing on 3D image data,
A front image generation step of generating a plurality of tomographic images continuous in a predetermined direction from three-dimensional image data after smoothing processing and layering the plurality of tomographic images to generate one front image. Prepare,
In the smoothing step
Gaussian blurring is performed on the pixel of interest and a plurality of peripheral pixels located around the pixel of interest on a plane orthogonal to the predetermined direction to replace the brightness value of the pixel of interest.
An image processing method characterized in that a moving average process is performed on a plurality of target pixels in a desired range centered on the attention pixel continuous in a predetermined direction to replace the brightness value of the attention pixel. In the front image generation step, a predetermined number of pixels are selected from a plurality of pixels at the same position in each of the plurality of tomographic images in descending order of brightness value, and the brightness value of the selected predetermined number of pixels is determined. The image processing method according to claim 4, wherein the front image is generated by using the average value as the brightness value of the position of the front image. The image processing method according to claim 4 or 5, further comprising a noise removing step of executing a noise removing process on the three-dimensional image data before the smoothing process. Image processing for causing the computer to function as the image processing apparatus according to any one of claims 1 to 3 or for causing the computer to execute the image processing method according to any one of claims 4 to 6. program.

Images