WO2019156139A1

WO2019156139A1 - Image processing device, image processing method, and image processing program

Info

Publication number: WO2019156139A1
Application number: PCT/JP2019/004336
Authority: WO
Inventors: 克己薮崎
Original assignee: 興和株式会社
Priority date: 2018-02-08
Filing date: 2019-02-07
Publication date: 2019-08-15
Also published as: JPWO2019156139A1

Abstract

This image processing device 20 comprises: a smoothing means that performs smoothing processing on three-dimensional image data; and an en-face image generation means 36 that generates a plurality of tomographic images that are sequential in a prescribed direction from the smoothed three-dimensional image data and layers the plurality of tomographic images to generate a single en-face image. The smoothing means has: a first filter means 34 that commutes the luminance value of a pixel of interest by performing Gaussian blur processing on the pixel of interest and a plurality of surrounding pixels that are positioned around the pixel of interest in a plane that is orthogonal to the prescribed direction; and a second filter means 35 that commutes the luminance value of the pixel of interest by performing moving average processing on a plurality of target pixels that are sequential in the prescribed direction and are within a desired range that is centered on the pixel of interest.

Description

Image processing apparatus, image processing method, and image processing program

The present invention relates to an image processing apparatus, an image processing method, and an image processing program for generating a single front image by overlaying a plurality of tomographic images generated from three-dimensional image data.

There is a tomographic imaging apparatus called OCT (Optical Coherence Tomography), which is one of the ophthalmologic diagnosis machines and takes a tomographic image of the retina. If general OCT imaging is performed, the obtained tomographic image is captured at a speed of, for example, 40 images / second, and 100 or more images are acquired by one inspection (imaging of a part of the retina). In recent years, a technique called OCT angiography (OCTA: Optical Coherence Tomography Angiography), which is a pseudo angiographic method using OCT, is known in order to grasp the retinal blood vessel morphology.

In OCT angiography, an angiographic image can be created non-invasively without using a fluorescent agent. Specifically, several identical images of the fundus of the eye to be examined are continuously photographed to obtain B-scan images with a time difference of several ms, and a short-time change between the B-scan images with a time difference is represented by a change in blood flow. Assuming that there is a blood vessel tomographic image (motion contrast image) depicting a microvessel of the fundus. One vascular tomographic image is generated from a plurality of B-scan image groups having a time difference, and this is repeated spatially by changing the scan position in the C-scan direction (y-direction) with respect to the observation target region. A plurality of vascular tomographic images can be obtained. From these continuous plural blood vessel images, one three-dimensional data set capable of capturing the blood vessel morphology in the retina three-dimensionally is constructed. That is, the three-dimensional data set is data in which the observation target region of the fundus of the eye to be examined is three-dimensionally modeled.

From this three-dimensional data set, a blood vessel tomographic image is generated as a tomographic image of a plurality of xy planes that are continuous in the depth direction (z direction). A plurality of continuous xy plane vascular tomographic images in a desired range are layered to generate a single front image (Enface image) on which the retinal blood vessels in the range are superimposed and used for various diagnoses. To do. For extracting blood flow changes, for example, as shown in Patent Document 1 and Non-Patent Document 1, several methods such as OMAG (Optical Microangiography) method and SSADA (Split-spectrum Amplitude-decorrelation Angiography) method have been proposed. Yes.

Special table 2013-518695 gazette

Here, when generating a front image for diagnosis, if there is salt particle noise (spike noise) in the region, the noise is mixed into the gap between the blood vessels and the extraction result of the blood vessels is deteriorated. There is. In addition, even if an attempt is made to generate a front image simply by averaging in order to reduce the influence of noise, the averaging process increases the black level of the background, which reduces the contrast of the front image, which is useful for diagnosis. There was a problem that a suitable front image could not be obtained easily.

The present invention has been made in view of such points, and an object thereof is to provide an image processing apparatus, an image processing method, and an image processing program capable of generating a front image suitable for diagnosis. In particular, an object is to provide an image processing apparatus, an image processing method, and an image processing program capable of generating a front image with improved contrast and improved blood vessel visibility.

In order to achieve the above object, firstly, the present invention includes a smoothing unit that performs a smoothing process on three-dimensional image data, and a plurality of tomographic images continuous in a predetermined direction after the smoothing process. Front image generating means for generating one front image by overlaying the plurality of tomographic images, and generating the one or more tomographic images, wherein the smoothing means includes the pixel of interest and the predetermined direction. First filter means for performing Gaussian blurring processing on a plurality of peripheral pixels located around the pixel of interest on an orthogonal plane and replacing the luminance value of the pixel of interest; and the continuous in the predetermined direction There is provided an image processing apparatus comprising: a second filter unit that performs a moving average process on a plurality of target pixels in a desired range centered on a target pixel to replace a luminance value of the target pixel. Do ( Akira 1).

According to the above invention (Invention 1), the three-dimensional image data is obtained by executing two different filtering processes, namely, a Gaussian blur process for the plane direction of the tomographic image used for generating the front image and a moving average process for the depth direction. By processing, it becomes possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels, and with a realistic calculation amount and calculation time, a three-dimensional image suitable for diagnosis and A front image can be generated.

In the tomographic image, the resolution in the depth direction is higher than that in the plane direction. Therefore, in the depth direction, even if moving average processing that is easily affected by the value of the adjacent pixel is performed, the data of the minute structure is lost. The risk is low, and as a result, it is possible to improve the smoothness and lower the black level of the background without impairing the blood vessel connectivity.

On the other hand, if the moving average processing is performed in the plane direction, the data value of the adjacent pixel greatly affects the data value of the adjacent pixel, resulting in a decrease in contrast and loss of structure data, or blood vessel connectivity. There is concern that it will get worse. Therefore, by adopting a process for increasing the weight of its own pixel value over the surrounding pixel value, such as Gaussian blurring, in the plane direction, it is possible to prevent the data having a minute structure from being buried.

It should be noted that the Gaussian blurring process is much more computationally intensive than the moving average process and takes a long time to process. Therefore, it is not preferable to perform the Gaussian blurring process in all directions from the viewpoint of the calculation amount and the calculation time. For these reasons, the present invention performs two different filtering processes, a Gaussian blur process in the plane direction and a moving average process in the depth direction, to improve smoothness without impairing blood vessel connectivity. The background black level can be reduced, and a three-dimensional image and a front image suitable for diagnosis can be generated with a realistic calculation amount and calculation time.

In the above invention (Invention 1), the front image generation means selects and selects a predetermined number of pixels in descending order of luminance value from a plurality of pixels at the same position in each of the plurality of tomographic images. It is preferable to generate the front image by using an average value of luminance values of a predetermined number of pixels as a luminance value at the position of the front image (Invention 2).

In the above inventions (1, 2), it is preferable to further comprise noise removal means for executing noise removal processing on the three-dimensional image data before the smoothing processing (invention 3).

Secondly, the present invention generates a smoothing step for performing a smoothing process on three-dimensional image data, and generates a plurality of tomographic images continuous in a predetermined direction from the three-dimensional image data after the smoothing process, A front image generating step of generating a single front image by overlaying the plurality of tomographic images, and in the smoothing step, the target pixel and the target pixel on a plane orthogonal to the predetermined direction A Gaussian blur process is performed on a plurality of peripheral pixels located around the pixel to replace the luminance value of the pixel of interest, and a plurality of objects in a desired range centered on the pixel of interest continuous in the predetermined direction Provided is an image processing method characterized in that a moving average process is performed on a pixel to replace a luminance value of the target pixel (invention 4).

According to the above invention (Invention 4), the two-dimensional filtering process is executed by performing two different filtering processes, namely, a Gaussian blur process for the plane direction of the tomographic image used for generating the front image and a moving average process for the depth direction. By processing, it becomes possible to improve smoothness and reduce the black level of the background without impairing the connectivity of blood vessels, and with a realistic calculation amount and calculation time, a three-dimensional image suitable for diagnosis and A front image can be generated.

In the above invention (invention 4), in the front image generation step, a predetermined number of pixels are selected from a plurality of pixels at the same position in each of the plurality of tomographic images in descending order of luminance value and selected. The image processing method according to claim 4, wherein the front image is generated by using an average value of luminance values of a predetermined number of pixels as a luminance value at the position of the front image. Invention 5).

Further, in the above inventions (Inventions 4 and 5), it is preferable to further include a noise removal step of performing noise removal processing on the three-dimensional image data before the smoothing processing (Invention 6).

Thirdly, the present invention is for causing a computer to function as an image processing apparatus according to any one of inventions 1 to 3, or for causing a computer to execute the image processing method according to any one of inventions 4 to 6. An image processing program is provided (Invention 7).

According to the image processing apparatus, the image processing method, and the image processing program of the present invention, a front image suitable for diagnosis can be generated.

1 is a block diagram illustrating an overall configuration of an image processing apparatus according to an embodiment of the present invention. It is explanatory drawing which shows the state which acquires a fundus tomographic image by scanning a fundus. It is explanatory drawing which shows the state which acquires multiple sheets of the fundus tomographic image of the same location of the fundus by time difference, and produces | generates a blood vessel tomographic image. It is a flowchart which shows the flow of the whole image processing in this embodiment. It is a flowchart which shows the flow of the noise removal process in this embodiment. It is a flowchart which shows the flow of the smoothing process in this embodiment. It is explanatory drawing (the 1) which shows typically the relationship between the attention pixel and adjacent pixel in the noise removal process by the image processing apparatus which concerns on this embodiment. It is explanatory drawing (the 2) which shows typically the relationship of the attention pixel and adjacent pixel in the noise removal process by the image processing apparatus which concerns on this embodiment. (A) is explanatory drawing which shows typically the relationship between the attention pixel and peripheral pixel in the smoothing process by the image processing apparatus which concerns on this embodiment, (b) is typical about the relationship between a attention pixel and an object pixel. It is explanatory drawing shown in. It is explanatory drawing (the 1) which shows typically a mode that a front image is produced | generated by the image processing apparatus which concerns on this embodiment. It is explanatory drawing (the 2) which shows typically a mode that a front image is produced | generated by the image processing apparatus which concerns on this embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Here, a tomographic image (OCT) is used to acquire a tomographic image of the fundus E of the eye to be examined, and one three-dimensional image that can grasp the blood vessel morphology on the fundus three-dimensionally from the obtained tomographic image by OCT angiography. Data is constructed and image processing is performed on the three-dimensional image data to generate a single front image in which blood vessels are superimposed and displayed in the depth direction. As a result of the image processing, a front image suitable for diagnosis is generated with improved contrast and improved blood vessel visibility. Note that the image to be subjected to image processing in the present invention is not limited to the three-dimensional image data capable of capturing the blood vessel morphology on the fundus three-dimensionally, and other objects are photographed with other types of devices. The present invention can also be applied to the constructed three-dimensional image data.

FIG. 1 is a block diagram showing the entire system for acquiring and processing a tomographic image of the fundus of the eye to be examined. The tomographic imaging apparatus 10 is an apparatus (OCT: Optical Coherence Tomography) that captures a tomographic image of the fundus of the subject's eye and operates, for example, in the Fourier domain method. Since the tomographic imaging apparatus 10 is known, detailed description thereof is omitted, but the tomographic imaging apparatus 10 is provided with a low coherence light source, and light from the low coherence light source is divided into reference light and signal light. The As shown in FIG. 2, the signal light is raster scanned on the fundus oculi E, for example, in the x and y directions. The signal light scanned and reflected by the fundus E is superimposed on the reference light reflected by the reference mirror to generate interference light, and an OCT signal indicating information on the depth direction (z direction) of the fundus is generated based on the interference light. To do.

The image processing apparatus 20 includes a control unit 21 realized by a computer including a CPU, a RAM, a ROM, and the like, and the control unit 21 controls the entire image processing by executing an image processing program. The image processing apparatus 20 includes a tomographic image forming unit 22, a storage unit 23, a vascular tomographic image forming unit 24, a three-dimensional data construction unit 25, a display unit 26, and an operation unit 27.

The tomographic image forming unit 22 is realized by a dedicated electronic circuit that executes a known analysis method such as a Fourier domain method, or an image processing program executed by the CPU, and an OCT signal generated by the tomographic imaging apparatus 10. Based on the above, a tomographic image of the fundus of the subject eye is formed.

For example, as shown in FIG. 2, when the fundus oculi E is scanned in the x direction at a position of y _N (N = 1, 2,..., T) in the y direction, m) sampling is performed. At each sampling time in the x direction, a tomographic image (A scan image) A _h (h = 1, 2,..., M) in the z direction is acquired, and from these m A scan images A _h. A tomographic image B _N (N = 1, 2,..., T) is formed. Since the A scan image is stored with a width of one pixel in the x direction and a length of n pixels in the z direction, the tomographic image B _N is an image having a size of m × n pixels (m pixels in the x direction). (Length, length of n pixels in the z direction) and is also called a B-scan image.

If a B-scan image is formed several times (for example, i times) continuously at a time difference of several ms at each position of y _N (N = 1, 2,..., T), i for each same location. B scan images B _NI (N = 1, 2,..., T) (I = 1, 2,..., I) having a time difference of several ms are formed. The t × i B-scan images _BNI formed by the tomographic image forming unit 22 are stored in a storage unit 23 configured by, for example, a semiconductor memory or a hard disk device. The storage unit 23 further stores the above-described image processing program and the like.

The vascular tomographic image forming unit 24 is realized by a dedicated electronic circuit that executes a known method for extracting a blood flow change, such as the OMAG method or the SSADA method, or an image processing program executed by the above-described CPU. Based on a plurality of tomographic images (B scan image B _NI ) having a time difference of several ms with respect to the same portion of the fundus of the eye to be examined formed by the image forming unit 22, a blood vessel tomographic image depicting a microvessel of the fundus at that position F _N (N = 1, 2,..., T) is formed.

For example, as shown in FIG. 3, i B-scan images B _{(N−1) I} (I = 1, 2,..., I) with a time difference of several ms at the position of y _N−1. To the vascular tomographic image F _T-1 from the i-th B scan images B _NI (I = 1, 2,..., I) having a time difference of several ms at the position of y _N. _From the i B scan images B _{(N + 1) I} (I = 1, 2,..., I) with a time difference of several ms at the position where _N is y _{N + 1,} the vascular tomographic image F _{N + 1} Each of them is formed by the tomographic image forming unit 24. In this way, t vascular tomographic images F _N (N = 1, 2,..., T) at the positions of y _N (N = 1, 2,..., T) are formed. And stored in the storage unit 23.

Three-dimensional data constructing unit 25, as shown in FIG. 3, the t pieces of blood vessel tomographic image F _N consecutive vascular tomographic image forming unit spatially formed by 24, sterically capture that vessels form in the retina One 3D image data G that can be stored is constructed and stored in the storage unit 23. The three-dimensional image data G is data in which the observation target region of the fundus of the eye to be examined is three-dimensionally modeled so that the blood vessel morphology in the retina can be captured three-dimensionally.

The image processing device 20 is provided with an image processing unit 30. The image processing unit 30 includes a calculation unit 31 as a noise removal unit, a counting unit 32 and a replacement unit 33, a first filter unit 34 and a second filter unit 35 as a smoothing unit, and a front image generation unit 36. Various image processing is executed on the three-dimensional image data G.

As will be described later, the calculation unit 31, the counting unit 32, and the replacement unit 33 are constructed by the three-dimensional data construction unit 25 and execute noise removal processing on the three-dimensional image data G stored in the storage unit 23. The calculating unit 31 calculates a difference value between the luminance value of the target pixel in the image to be processed and the luminance values of a plurality of adjacent pixels located around the target pixel, and the counting unit 32 Counts the number of adjacent pixels whose calculated difference value is less than or equal to a predetermined threshold (luminance difference threshold) as the number of non-corresponding adjacent pixels, and the replacing means 33 determines that the number of non-corresponding adjacent pixels is a predetermined number (not applicable Number) or less, the luminance value of the target pixel is replaced with the average value of the luminance values of a plurality of adjacent pixels.

First filter means 34 and second filter means 35 is adapted to perform smoothing processing on the three-dimensional image data G ₁ after the noise removal processing, the first filter means 34, to be processed three-dimensional and target pixel in the image data G _1, the luminance of the plurality of the pixel of interest by performing a Gaussian blurring process with respect to the peripheral pixels located around the target pixel on a plane perpendicular to the predetermined direction (depth direction) The value is replaced, and the second filter means 35 performs a moving average process on a plurality of target pixels in a desired range centered on the target pixel continuous in a predetermined direction (depth direction), and the luminance of the target pixel. Replace the value.

Further, the front image generation unit 36 generates a plurality of tomographic images continuous to the smoothing process after the three-dimensional image data G ₂ from a predetermined direction (depth direction), and overlaid the plurality of tomographic images A predetermined number of pixels are selected in descending order of luminance value from a plurality of pixels at the same position in each of the plurality of tomographic images, and the selected predetermined number of pixels are generated. A front image is generated by using an average value of luminance values of pixels as a luminance value at the position of the front image. Each means or each image processing in the image processing unit 30 is realized by using a dedicated electronic circuit or by executing an image processing program.

The display unit 26 is configured by a display device such as an LCD, for example, and displays various images stored in the storage unit 23, various images generated or processed by the image processing device 20, accompanying information such as information on the subject, and the like. To do.

The operation unit 27 includes, for example, a mouse, a keyboard, an operation pen, a pointer, an operation panel, and the like. The operation unit 27 is used for selecting an image displayed on the display unit 26 or for giving an instruction to the image processing apparatus 20 by the operator. It is done.

Subsequently, the flow of executing various image processing by the image processing unit 30 on the three-dimensional image data will be described with reference to the flowchart of FIG. Note that S100 to S400 in FIG. 4A correspond to Steps 100 to 400 in the description of the flow described below, and S101 to S107 in FIG. 4B correspond to Steps 101 to 107, and FIG. S201 to S205 in the middle correspond to steps 201 to 205.

In the present embodiment, first, image processing for removing noise from the constructed three-dimensional image data G is performed (step 100). The three-dimensional image data G is a set of t vascular tomographic images F _N (N = 1, 2,..., T) that are spatially continuous, and one of the vascular tomographic images F _N. was processed image F _T, the noise removal process is used in conjunction also vascular tomographic image F _T-1 and vascular tomographic image F _{T + 1} before and after. As shown in FIG. 5B, the processing target image _FT is an image having a size of m × n pixels, and the vascular tomographic image _FT-1 and the vascular tomographic image _{FT + 1} are the processing target image FT. a tomographic image that is continuous with _T, an image similar to the target image F _T having a size of m × n pixels. The vascular tomographic image F _T−1 , the processing target image _FT, and the vascular tomographic image F _{T + 1} are all stored in the storage unit 23.

First calculating means 31, as shown in FIG. 5, the processing pixel of interest in the target image F _T and one set as a target pixel Q, the processing target image F _T and the tomographic image F _T-1 and before and after the A total of 26 pixels spatially located around the pixel of interest in the tomographic image _{FT + 1} are set as adjacent pixels q _i (i = 1 to 26) (step 101). That is, adjacent pixels q _i, in addition to a total of eight pixels of the pixel positioned in contact with the four corners of the pixel and the target pixel located at the upper, lower, left and right of the target pixel Q in the processing target image F _T, vascular tomographic image F _{T -1} and a total of 26 pixels, including a total of 18 pixels to be positioned around the spatially target pixel Q when the blood vessel tomographic image F _{T + 1} arranged in the processing target image F _T. 5, the processing object fourth column from the left of the image F _T, and the target pixel Q, which is set in the third row from the top, the processing target image F _T and longitudinal vascular tomographic image F _T-1 and vascular tomographic thereof The relationship between 26 adjacent pixels q _i spatially located around the pixel of interest Q in the image _{FT + 1} is shown. Also shows the processing target image F _T and before and after vascular tomographic image F _T-1 and vascular tomographic image F _{T + 1} the pixel of interest Q from the state arranged state cut out only adjacent pixels q _i in FIG.

Incidentally, not necessarily adjacent pixels is 26 set in the case of setting the end portion of the pixel of interest Q in the processing target image F _T, when the target pixel Q is set to an end in the processing target image F _T The adjacent pixels may be appropriately set within 26 in accordance with the position of the target pixel. For example the first column the pixel of interest from the left of the processing target image F _T, when it is set in the third row from the top, above the target pixel Q, down, right, upper right, located lower right 5 17 pixels including the pixel and 12 pixels located adjacent to the target pixel Q and the five pixels in the successive vascular tomographic images may be set as adjacent pixels. Further, even when there are no vascular tomographic images continuous before and after, adjacent pixels may be appropriately set within 26 according to the position of the target pixel. Furthermore, the method for setting adjacent pixels may be appropriately changed according to the type of 3D image data to be processed, the purpose of noise removal, and the like.

Further, depending on the type of image to be processed, the purpose of noise removal, etc., the vascular tomographic image _FT-2 and the vascular tomographic image _{FT + 2 that} are continuous before and after the vascular tomographic image _FT-1 and vascular tomographic image _{FT + 1.} May also be used as a setting target for adjacent pixels, and the setting of adjacent pixels may be extended to a 5 × 5 range instead of a 3 × 3 range, _Two pixels positioned adjacent to the target pixel in the vascular tomographic images before and after the four consecutive pixels (for example, four pixels q ₂ , q ₄ , q ₅ , and q ₇ in FIG. 5). The method of setting adjacent pixels may be changed as appropriate, for example, six pixels including pixels (for example, two pixels q ₁₃ and q ₂₂ in FIG. 5) are set as adjacent pixels.

Subsequently, the calculation unit 31 calculates the luminance value D of the target pixel Q and the luminance values d _i (i = 1 to 26) of the 26 adjacent pixels q _i as the processing target image F _T , the vascular tomographic image F _T−1, and the like. Obtained from the original image data of the vascular tomographic image _{FT + 1} (step 102).

After obtaining the luminance value D of the pixel of interest Q and the luminance value d _i of the adjacent pixel q _i , the calculation unit 31 first subtracts the luminance value d _i of each adjacent pixel q _i from the luminance value D of the pixel of interest Q. A difference value is calculated for each of the 26 adjacent pixels q _i (step 103a). This first difference value is calculated by the number of adjacent pixels, and is used to estimate whether the target pixel Q is white point noise.

Subsequently, the white point of the target pixel Q is determined. Specifically, the counting means 32 compares whether or not the calculated first difference value is equal to or less than the first luminance difference threshold value PKT stored in the storage unit 23 in advance, and the first luminance difference threshold value PKT. The number of adjacent pixels that was below is counted as the first non-applicable number of adjacent pixels (step 104a). That is, the number of adjacent pixels satisfying the conditional expression: luminance value D of target pixel Q−luminance value d _i ≦ PKT of adjacent pixel q _i is the first non-applicable number of adjacent pixels.

After counting the number of first non-applicable adjacent pixels, the replacement unit 33 determines whether the first non-applicable adjacent pixel number is equal to or less than the first non-applicable allowable number PFT stored in the storage unit 23 in advance. (Step 105a). If the number of first non-applicable adjacent pixels is equal to or less than the first non-applicable permissible number PFT, the target pixel Q is determined to be white point noise.

On the other hand, if the number of first non-applicable neighboring pixels is not less than or equal to the first non-applicable allowable number PFT, the calculating unit 31 obtains a second value obtained by subtracting the luminance value D of the target pixel Q from the luminance value d _i of each adjacent pixel q _i . A difference value is calculated for each of the 26 adjacent pixels q _i (step 103b). Similar to the first difference value, the second difference value is also calculated by the number of adjacent pixels, and is used to estimate whether the target pixel Q is black point noise.

Subsequently, the black point of the target pixel Q is determined. Specifically, the counting means 32 compares whether or not the calculated second difference value is equal to or less than the second luminance difference threshold value HLT stored in the storage unit 23 in advance, and the second luminance difference threshold value HLT is compared. The number of adjacent pixels that was below is counted as the second non-applicable number of adjacent pixels (step 104b). That is, the conditional expression: the luminance value d _i adjacent pixels q _i - the number of neighboring pixels satisfying the luminance value D ≦ HLT of the pixel of interest Q is the second non-relevant number of adjacent pixels.

After counting the number of second non-applicable adjacent pixels, the replacement unit 33 determines whether the second non-applicable adjacent pixel number is equal to or less than the second non-applicable allowable number HFT stored in the storage unit 23 in advance. (Step 105b). If the number of second non-applicable adjacent pixels is equal to or less than the second non-applicable allowable number HFT, the target pixel Q is determined to be black point noise.

As a result of the determinations in steps 105a and 105b, when the target pixel Q is determined to be white point noise (when the number of first non-applicable neighboring pixels is equal to or less than the first non-applicable permissible number PFT) and the target pixel Q is black When it is determined that it is point noise (when the second non-applicable adjacent pixel number is equal to or less than the second non-applicable allowable number HFT), the replacement unit 33 calculates the average value of the luminance values d _i of the adjacent pixels q _i. Calculate (step 106). Then, an image in which the luminance value of the target pixel Q is replaced with the calculated average value is generated (step 107) and stored in the storage unit 23. As a result of the determination in step 105a, the first non-applicable adjacent pixel number is not less than or equal to the first non-applicable allowable number PFT, and as a result of the determination in step 105b, the second non-applicable adjacent pixel number is equal to or less than the second non-applicable allowable number HFT. Otherwise, the luminance value of the target pixel Q is left as it is without being replaced with the average value.

The processing described above, in the processing target image F _T, for example, repeated while setting the order all the pixels from the pixels of the upper left end as the target pixel, the image subjected to noise removal processing in the overall final processed target image F _T Generated. When the above-described noise removal processing to the t pieces of blood vessel tomographic image F _N constituting the three-dimensional image data is performed, the three-dimensional image data G ₁ after the noise removal processing is stored in the storage unit 23. Note that not all vascular tomographic images need to be processed, and any number of vascular tomographic images can be selected according to the type of 3D image data to be processed, the purpose of noise removal, and the like.

As described above, the noise removal processing according to the present embodiment compares the pixel of interest in the image to be processed with the adjacent pixels located around the target pixel, and the luminance value of the pixel of interest is a predetermined value higher than that of the adjacent pixel. If it is brighter or darker than this, it is replaced with the average value of adjacent pixels. Increasing the luminance difference between the adjacent pixel used for replacement and the target pixel, that is, setting the first luminance difference threshold and the second luminance difference threshold described above to be large reduces the number of target pixels to be replaced, and conversely setting it to be small. The number of target pixels to be replaced increases.

On the other hand, in the replacement determination, the most strict determination formula is used to determine whether the target pixel is brighter than a predetermined value than all adjacent pixels, or whether the target pixel is darker than a predetermined value than all adjacent pixels. It can also be used. In this case, by setting the first non-applicable permissible number and the second non-applicable permissible number to 0, the target pixel is brighter than a predetermined value than all the adjacent pixels, or the target pixel is more than all the adjacent pixels. Only when the brightness is darker than the predetermined value, the process of replacing the luminance value of the target pixel with the average value of the luminance values of the adjacent pixels is performed. As the first non-applicable permissible number or the second non-applicable permissible number is decreased, the number of target pixels replaced with the average value decreases, and the replacement is performed as the first non-applicable permissible number or the second non-applicable permissible number is increased. The number of pixels of interest increases.

The same value may be used for the first luminance difference threshold and the second luminance difference threshold, or different numerical values may be set individually. Moreover, the same value may be used for the first non-applicable allowable number and the second non-applicable allowable number, or different numerical values may be set individually. The condition values stored in advance in the storage unit 23 may be applied after considering the optimum values according to the quality of the image and the properties (shape, brightness, size, etc.) of the structure to be extracted. it can.

In particular, if white point noise is mixed in between the blood vessels, the extraction result of the blood vessels is reduced. However, by performing the noise removal process described above, the pixel of interest does not have bright pixels around it. In the case of a pixel, it can be determined that the pixel of interest is a white point noise pixel and can be replaced with the average value of adjacent pixels. Note that if the first non-applicable permissible number is increased, even if there are several high-luminance pixels in the vicinity, it is possible to determine that the pixel of interest is a white point noise pixel and replace the adjacent value with the average value. The noise removal effect can be enhanced and the black level of the background can be improved. On the other hand, microvessels are difficult to be extracted.

In the above noise removal processing, both white point noise and black point noise are removed, but it is not always necessary to remove both, and depending on the type of 3D image data to be processed and the purpose of noise removal, etc. Either one of the point noise and the black point noise may be removed.

Then performing image processing for smoothing the three-dimensional image data G ₁ after the noise removal processing (step 200). First filter means 34 First, as shown in FIG. 7 (a), one sets the pixel of interest in a three-dimensional image data G ₁ as the target pixel P (step 201), XY plane that exist in the target pixel P In the above, a total of eight pixels located around the pixel of interest P are set as the peripheral pixels p _i (i = 1 to 8) (step 202). Incidentally, FIGS. 7 (a) and (b), by cutting a portion from the three-dimensional image data G _1, 3 pixels in the x direction, three pixels in the y direction, only total 45 pixels of five pixels in the z-direction schematically Is displayed.

Then the first filter means 34 performs the Gaussian blurring process to target a total of nine pixel of interest P and eight surrounding pixels p _i, replaces the luminance value of the target pixel P (step 203). The Gaussian blurring process is a filtering process that averages pixel values by applying weights to neighboring pixel values according to the distance from the target pixel. Specifically, the luminance values e _i (i = 1 to 8) of the target pixel P and the eight peripheral pixels p _i are acquired from the original image data, and the weight is increased as the target pixel P is closer to the target pixel P. The luminance value of the target pixel P and the eight peripheral pixels p _i is weighted and averaged using a Gaussian distribution function as shown in Equation 1 below so that the weight decreases as the distance from the center increases. The luminance value.

For example, to target a total of nine pixel of interest P and eight surrounding pixels p _i, it is possible to use the following formula 2 as a kernel K to be used for weighting.

When weighting is performed at a rate corresponding to each position of the peripheral pixel p _i using the above-described kernel K, for example, the luminance values e ₁ and e _{3 of the} peripheral pixels p ₁ , p ₃ , p ₆ and p ₈ shown in FIG. , E ₆ , e ₈ have a weight of 1/50, and luminance values e ₂ , e ₄ , e ₅ , e ₇ of peripheral pixels p ₂ , p ₄ , p ₅ , p ₇ have a weight of 6/50. An average value is calculated by weighting the luminance value of the pixel P by 22/50, and this average value is set as the luminance value of the pixel of interest P.

Subsequently, the second filter unit 35 sets five pixels centered on the pixel of interest P continuous in the Z direction, that is, the pixel of interest P and the pixel C _i (i = 1 to 4) shown in FIG. 7 as target pixels. Then, the moving average process is executed on the target pixel P and the pixel C _i (i = 1 to 4) as targets, and the luminance value of the target pixel P is replaced (step 205). Specifically, the luminance values of the pixel of interest P and the pixels C _i (i = 1 to 4) are acquired from the original image data, and the acquired luminance values are simply averaged to obtain the luminance value of the pixel of interest P.

The above process, the three-dimensional image in the data G _1, repeated while setting the order all the pixels as a target pixel, finally the three-dimensional image data G ₁ of performed smoothing process in the entire three-dimensional image data G ₂ Is generated and stored in the storage unit 23. Incidentally, not necessarily set for all the pixels of the three-dimensional image data G ₁ as the target pixel, it is not necessary to the smoothing process, the three-dimensional image data to be processed type and the smoothing process in accordance with the purpose or the like of the smoothing The target range can also be set arbitrarily.

As described above, the three-dimensional image data G ₁ is obtained by executing two different filtering processes, namely, a Gaussian blur process for the plane direction (xy direction) of the three-dimensional image data G _{1 and} a moving average process for the depth direction (z direction). By processing ₁ , a three-dimensional image and a front image suitable for diagnosis can be generated.

In a tomographic image, the depth direction has a higher resolution than the plane direction, so even if moving average processing is performed in the depth direction, the smoothness is improved and the black level of the background is reduced without impairing blood vessel connectivity. It is possible. On the other hand, if the moving average process is performed in the plane direction, the contrast is lowered and the blood vessel connectivity is deteriorated. For the plane direction, the data of the minute structure is buried by adopting the Gaussian blurring process. It becomes possible to prevent it.

Note that the Gaussian blurring process is much more computationally intensive and time-consuming than the moving average process, so it is not preferable to perform the Gaussian blurring process in all directions from the viewpoint of the calculation amount and calculation time. Therefore, as described later, three-dimensional image data after smoothing j xy planar tomographic images H _L (L = 1, 2,..., J) continuous in the depth direction (z direction). When a single front image E is generated by overlaying the j xy planar tomographic images _HL generated from G _2, Gaussian blurring processing is performed in the plane direction, and moving average processing is performed in the depth direction. By executing three different filtering processes, it is possible to improve smoothness and reduce the black level of the background without damaging the connectivity of blood vessels, and it is suitable for diagnosis with realistic calculation amount and calculation time. 3D images and front images can be generated. In particular, when the front image is generated after the smoothing process is performed on the three-dimensional image data after the noise removal process is performed as in the present embodiment, the front image with improved contrast and extremely good blood vessel visibility. Can be obtained.

In the case of setting the end portion of the target pixel P in the three-dimensional image data G ₁ are not necessarily the peripheral pixels is eight sets, the target pixel P is set to end in the three-dimensional image data G ₁ In this case, the number of peripheral pixels may be appropriately set within 8 according to the position of the target pixel.

Further, the kernel used in the Gaussian blur processing by the first filter means 34 is not necessarily a combination of the above-mentioned rates in a 3 × 3 range, and depends on the type of 3D image data to be processed, the purpose of smoothing, and the like. Thus, for example, a kernel having a combination of different rates in the range of 5 × 5 may be used. If Gaussian blurring processing is performed using a kernel in a 5 × 5 range, a total of 24 pixels positioned around the pixel of interest on the xy plane where the pixel of interest exists is set as the peripheral pixel.

Further, the number of target pixels to be subjected to the moving average process by the second filter unit 35 is not necessarily five, and as long as the target pixel is a pixel in a range centered on the target pixel, the third order to be processed Depending on the type of the original image data, the purpose of smoothing, etc., the desired number of target pixels such as 3, 5, 7, etc. may be used.

Next, processing for generating a front image E from the three-dimensional image data G ₂ after the smoothing process. The front image generation unit 36 generates a continuous in the z-direction from the three-dimensional image data _{G 2} after the smoothing process j pieces of xy plane tomographic image _{H L (L = 1,2, ·····} , j) (Step 300). Incidentally, xy plane tomographic image H _L need not necessarily be generated through the 3D image data G ₂ as the target, set to an arbitrary portion to be observed as a diagnosis target in a three-dimensional image data G _2, the portion A plurality of xy plane tomographic images H _L can be generated only for the target.

Subsequently, as shown in FIG. 8, a single front image E is generated by overlaying j xy planar tomographic images _HL (step 400). At this time, the front image is obtained by simply selecting the maximum luminance value from the luminance values of a plurality of pixels (pixels marked in FIG. 8) at the same position in each of the j xy planar tomographic images _HL . In this embodiment, a predetermined number of pixels are selected from a plurality of pixels at the same position in each of the j xy planar tomographic images _{HL in} descending order of the luminance value. Then, the front image E is generated by setting the average value of the luminance values of the selected predetermined number of pixels as the luminance value at the position of the front image E. FIG. 9 shows a schematic view of this process.

FIG. 9 schematically shows a process in which the average value of the luminance values of a predetermined number of selected pixels is used as the luminance value at the position of the front image E. FIG. 9A shows a thick blood vessel at the position. In some cases, (b) shows a case where there is a thin blood vessel at the position, and (c) shows a case where there is a noise bright spot at the position. For example, when there are twelve xy planar tomographic images _HL , the state where twelve pixels at the same position are arranged in a line as they are is shown on the left side of each of FIGS. 9 (a), (b), and (c). Has been. These are rearranged in the descending order of luminance values, as shown on the right side of each of FIGS. 9A, 9B, and 9C. Among the 12 pixels rearranged in this way, the five pixels from the top are selected in descending order of the luminance value, and the average value of the luminance values of the selected five pixels is determined as the luminance value at the corresponding position of the front image E. By doing so, the front image E is generated.

As described above, when a single front image E is generated by overlaying j xy planar tomographic images _HL , the maximum luminance value among the pixel values of a plurality of pixels at the same position is simply set as the front image E. If the predetermined number of pixels are selected in descending order of the luminance value, and the average value of the luminance values of the selected predetermined number of pixels is set as the luminance value at the position of the front image E, For example, as shown in FIG. 9A, if there is a thick blood vessel at the position, the luminance value at the position of the front image E is reflected and the blood vessel can be clearly seen in the front image E. . Further, as shown in FIG. 9B, if there is a thin blood vessel at the position, the luminance value at the position of the front image E is calculated and the brightness level of the micro blood vessel is maintained in the front image E. As shown in FIG. 9C, even if there is a noise bright spot at the position, if the luminance value of the other selected pixels is small, the average of the position of the front image E is obtained by averaging. The luminance value becomes small, and sesame salt noise in the front image E can be reduced. As a result, the black level of the background can be reduced, and a front image suitable for diagnosis can be generated.

As described above, the image processing apparatus, the image processing method, and the image processing program according to the present invention have been described with reference to the drawings. However, the present invention is not limited to the above-described embodiments, and various modifications can be made.

The present invention can be used as a method for generating a single front image by overlaying a plurality of tomographic images generated from three-dimensional image data.

DESCRIPTION OF SYMBOLS 10 Tomography apparatus 20 Image processing apparatus 21 Control part 22 Tomographic image formation part 23 Storage part 24 Vascular tomographic image formation part 25 Three-dimensional data construction part 26 Display part 27 Operation part 30 Image processing part 31 Calculation means 32 Counting means 33 Replacement Means 34 First filter means 35 Second filter means 36 Front image generation means

Claims

Smoothing means for executing a smoothing process on the three-dimensional image data;
Front image generation means for generating a plurality of tomographic images continuous in a predetermined direction from the three-dimensional image data after the smoothing process, and generating a single front image by overlaying the plurality of tomographic images; Prepared,
The smoothing means comprises:
First filter means for performing Gaussian blurring processing on the target pixel and a plurality of peripheral pixels positioned around the target pixel on a plane orthogonal to the predetermined direction to replace the luminance value of the target pixel; ,
Second filtering means for executing a moving average process on a plurality of target pixels in a desired range centered on the target pixel continuous in the predetermined direction to replace the luminance value of the target pixel. A featured image processing apparatus.
The front image generation means selects a predetermined number of pixels in descending order of luminance values from a plurality of pixels at the same position in each of the plurality of tomographic images, and sets the luminance values of the selected predetermined number of pixels. The image processing apparatus according to claim 1, wherein the front image is generated by using an average value as a luminance value at the position of the front image.
The image processing apparatus according to claim 1, further comprising a noise removing unit that performs a noise removing process on the three-dimensional image data before the smoothing process.
A smoothing step for performing a smoothing process on the three-dimensional image data;
A front image generation step of generating a plurality of tomographic images continuous in a predetermined direction from the three-dimensional image data after the smoothing process, and generating a single front image by overlaying the plurality of tomographic images. Prepared,
In the smoothing step,
Performing a Gaussian blurring process on the pixel of interest and a plurality of peripheral pixels located around the pixel of interest on a plane orthogonal to the predetermined direction to replace the luminance value of the pixel of interest;
An image processing method comprising: performing a moving average process on a plurality of target pixels in a desired range centering on the target pixel continuous in the predetermined direction to replace a luminance value of the target pixel.
In the front image generation step, a predetermined number of pixels are selected in descending order of luminance values from a plurality of pixels at the same position in each of the plurality of tomographic images, and the luminance values of the selected predetermined number of pixels are selected. The image processing method according to claim 4, wherein the front image is generated by using an average value as a luminance value at the position of the front image.
The image processing method according to claim 4, further comprising a noise removal step of performing a noise removal process on the three-dimensional image data before the smoothing process.
An image processing for causing a computer to function as the image processing apparatus according to any one of claims 1 to 3, or for causing a computer to execute the image processing method according to any one of claims 4 to 6. program.