JP4915836B2 - Method, program, and apparatus for operating image processing apparatus - Google Patents

Description

The present invention relates to an operation method, a program, and an apparatus for an image processing apparatus , and in particular, uses a plurality of preoperative images obtained by a medical imaging apparatus such as a magnetic resonance imaging apparatus, an X-ray CT apparatus, and an ultrasonic apparatus. Thus, noise information and organ deformation on the intraoperative image obtained by the medical imaging apparatus are discriminated, and registration is performed when organ deformation occurs on the image, and removal is performed when noise information is present. The present invention relates to a technique for providing the latest intraoperative images with high image quality.

  An operation that assists the surgeon by recognizing the three-dimensional spatial position and posture of the surgical site in real time with a pre- or intraoperative medical image is called an image-guided operation. In image-guided surgery, a surgical operation can be performed while supporting the recognition of the position of the organ and internal morphological information and the understanding of the positional relationship between the surgical instrument and the organ.

  However, when preoperative medical images are used for image-guided surgery, there is a discrepancy between the 3D spatial position and posture planned before the operation and the actual 3D spatial position and posture during the operation due to the deformation of the organ during the operation. There are problems that arise. In recent years, therefore, a surgical plan has been made with intraoperative images taken intraoperatively with a medical imaging apparatus, and the influence of organ deformation has been avoided as much as possible. However, there are various problems depending on the characteristics of various intraoperative images, and the selection must be made carefully depending on the purpose.

As an example of such a system for displaying an intraoperative image, Patent Document 1 describes an intraoperative MR image obtained by a magnetic resonance imaging apparatus (hereinafter referred to as an MRI apparatus) and an ultrasonic diagnostic apparatus (hereinafter referred to as a US apparatus). An image display system that performs alignment with an elastic image obtained and contributes to determination of a boundary between edema and a malignant tumor in an image captured during surgery is disclosed.
JP 2005-102713 A

  When a CT image is used as an intraoperative image for image-guided surgery, there are problems that are unavoidable due to the principle of the device, such as low image distortion, poor soft tissue rendering ability, and patient exposure during CT imaging. On the other hand, when MRI images with excellent soft tissue visualization and no problem of exposure are used for image-guided surgery, they lead to geometric distortion of the MRI images, image quality deterioration due to high-speed imaging, and electrical noise such as anesthesia machines In addition, there is a problem that the image quality of the MRI image is deteriorated due to obstruction of the antenna characteristics of the reception coil of the resonance signal. In addition, the ultrasonic apparatus is excellent in real time in terms of image capturing, but the image quality cannot be denied compared with the CT apparatus or the MRI apparatus.

In view of the above problems, the present invention provides an operation method, a program, and an apparatus for an image processing apparatus capable of generating and displaying an intraoperative image with high image quality while using a preoperative image as a reference in image guided surgery. With the goal.

In order to achieve the above object, an operation method of an image processing apparatus according to the present invention is a first method of reading a preoperative image obtained by imaging a surgical target region of a subject by a first medical image photographing apparatus before surgery. A plurality of reading steps; a second reading step for reading an intraoperative image obtained by imaging a surgical target site of the subject by a second medical imaging device during surgery; and a plurality of preoperative images having an arbitrary image size. And dividing the intraoperative image into a plurality of second divided areas composed of areas corresponding to the first divided areas, the first divided areas and the first divided areas. For each corresponding second divided region, a calculation step for calculating a mutual information amount indicating a degree of coincidence between the first divided region and the second divided region, the mutual information amount, and a predetermined noise in the intraoperative image This is included A determination step for determining whether or not the predetermined noise is included in the second divided area, and when it is determined that the predetermined noise is included in the second divided area A noise removing step for removing the predetermined noise from the second divided region and generating an intraoperative image from which the noise has been removed; and a display step for displaying the intraoperative image from which the predetermined noise has been removed. It is characterized by.

  The “first medical image capturing device” and the “second medical image capturing device” may be the same type or different types. For example, a combination of an MRI apparatus and an X-ray CT apparatus may be used.

  Further, the preoperative image is a three-dimensional image composed of a plurality of cross-sectional images obtained by photographing the surgical target site of the subject, and a position at which position information indicating the surgical target site of the subject is acquired during the surgery. An information acquisition step is further included, and in the first reading step, a cross-sectional image obtained by photographing the surgical target site of the subject is extracted or reconstructed from the three-dimensional image and read as the preoperative image. The method further includes a step of replacing the cross-sectional image corresponding to the intraoperative image from which the predetermined noise has been removed in the three-dimensional image with the intraoperative image from which the predetermined noise has been removed generated in the noise removing step. Good.

  Further, in the determination step, the mutual information amount is further compared with a predetermined organ deformation determination value indicating a region where the organ deformation site is imaged, and the organ deformation site is imaged in the second divided region. It is determined whether or not a region is included, and when it is determined that the region obtained by imaging the organ deformation site is included in the second divided region, the first divided region matches the second divided region. An alignment step for aligning the first divided region, a recalculation step for recalculating the mutual information amount between the aligned first divided region and the second divided region, and the recalculated A re-discrimination step of comparing the mutual information amount with the noise discrimination threshold again to discriminate whether or not the predetermined noise is included in the second divided region, wherein the noise removal step It may be performed based on another step of the determination result.

  The predetermined noise includes at least one of spike noise in which a pixel value is missing or white noise having a pixel value not corresponding to the density value of the subject, and the noise determination threshold includes the spike noise. At least one of a spike noise discrimination threshold indicating that the white noise is included, and in the determination step, the mutual information and the spike noise determination threshold or the white noise When it is determined that the spike noise is included by comparing with at least one of the determination threshold values, the first divided region corresponding to the position where the spike noise is generated in the second divided region in the noise removal step. Is obtained, and the pixel value is obtained from the spike node in the second divided region. When it is determined that the white noise is included in the pixel value of the pixel in which a shift occurs, in the noise removal step, a smoothed image obtained by smoothing the second divided region, and the second divided An edge-enhanced image obtained by subjecting the region to the structure enhancement process may be generated, and the smoothed image and the edge-enhanced image may be combined to remove the white noise.

  In the display step, the intraoperative image, a preoperative image taken at the subject position corresponding to the intraoperative image, a virtual endoscopic image reconstructed based on the three-dimensional image, and the noise removal At least one of the previous intraoperative images may be displayed side by side.

The image processing program according to the present invention is characterized in that to execute the method of operating any of the image processing apparatus described above in a computer.

  Further, the image processing apparatus according to the present invention stores a first storage means for storing a three-dimensional image composed of a plurality of cross-sectional images obtained by imaging a surgical target site of a subject with a first medical image imaging apparatus before surgery. And a position information acquisition means for acquiring position information indicating a surgical target part of the subject during the surgery, and a cross-sectional image obtained by photographing the surgical target part of the subject from the three-dimensional image based on the positional information. A first reading means for extracting or reconstructing and reading as a preoperative image; and a second reading means for reading an intraoperative image obtained by imaging a surgical target region of the subject by a second medical image capturing apparatus during surgery. Dividing means for dividing the preoperative image into a plurality of first divided areas having an arbitrary image size and dividing the intraoperative image into a plurality of second divided areas having areas corresponding to the first divided areas; The first A calculation means for calculating a mutual information amount indicating a degree of coincidence between the first divided region and the second divided region for each divided region and the second divided region corresponding to the first divided region; and the mutual information amount And a determination means for comparing whether or not the predetermined noise is included in the second divided region, and a determination unit for comparing whether or not the predetermined noise is included in the second divided region; When it is determined that the noise is included, noise removal means for removing the noise from the second divided region and generating an intraoperative image from which the noise has been removed, and an intraoperative image from which the noise has been removed are displayed. A display means and a cross-sectional image obtained by imaging the subject position corresponding to the intraoperative image from which the noise has been removed in the three-dimensional image are placed on the intraoperative image from which the noise has been removed generated in the noise removal step. A replacement means for storing in the first storage means, and when the intraoperative image from which the noise is removed is generated, the replacement means removes the noise from the tomographic image of the three-dimensional image. It replaces with an intraoperative image, The said 1st reading means repeats the operation | movement which extracts the cross-sectional image which image | photographed the surgery object site | part of the said subject from the three-dimensional image containing the said replaced intraoperative image, It is characterized by the above-mentioned.

  According to the present invention, in image-guided surgery, the image quality of an intraoperative image can be improved by performing processing for noise removal and organ deformation of the intraoperative image using a preoperative image.

  Embodiments of the present invention will be described below with reference to the accompanying drawings. Note that components having the same function are denoted by the same reference symbols throughout the drawings for describing the embodiment of the invention, and the repetitive description thereof is omitted.

<Hardware configuration>
An intraoperative image noise removal system 1 in FIG. 1 includes a medical image photographing device 10 that photographs an internal diagnostic image (medical image) of a subject, an image processing device 20 that performs noise removal and various image processing techniques using a medical image, The surgical instrument 30 handled by the surgeon and the position detector 40 that detects position information and posture information of the surgical instrument 30 and transmits the information to the medical image capturing apparatus 10 and the image processing apparatus 20 are provided. The medical imaging apparatus may be any apparatus that can generate a medical image obtained by imaging the inside of the subject, such as an MRI apparatus, an X-ray CT apparatus, or a US apparatus.

  The image processing apparatus 20 includes a processing unit 21 that performs noise removal processing on a medical image, and a display unit 22 that displays the medical image from which noise has been removed. Although not shown, the processing unit 21 is a central processing unit (CPU) that mainly controls the operation of each component, a main memory that reads a control program and image data of the image display system, and serves as a work area, and an image display system Control program, various application software, data recording device for storing image data of subject, display memory for temporarily storing display data, pointing device such as mouse, trackball and touch panel, and controller for pointing device And a keyboard having keys and switches for setting various parameters, a data bus connecting each of the above components, the medical imaging device 10 and the position detection device 40, and an intraoperative image and surgical instrument position information. / Interface part for accepting input of posture information Configured to include a. The display unit 22 includes a display such as a CRT or a liquid crystal display device.

  The surgical instrument 30 is a surgical instrument inserted into the body of a subject, such as an endoscope, forceps, or catheter.

  The position detection device 40 stores, for example, the insertion amount of the endoscope in a memory, and combines the insertion amount with a travel pattern of a blood vessel acquired in advance, so that the endoscope distal end is based on the insertion site of the subject. You may comprise as an apparatus which detects in which position.

  FIG. 2 is a program block diagram showing a program for realizing the image display system 1 according to the present embodiment.

  The program includes a position information acquisition unit 21a that acquires position information and posture information indicating the surgical target site of the subject from the position detection device 40 during surgery, and a three-dimensional stored in the server 25 based on the position information and posture information. A first reading unit 21b that extracts or reconstructs a cross-sectional image obtained by imaging a surgical target site from an image and reads it as a preoperative image, and an intraoperative result obtained by imaging the surgical target site of a subject using the MRI apparatus 11 during surgery. The second reading unit 21c that reads the image 50 and the preoperative image are divided into a plurality of first divided areas having an arbitrary image size, and the intraoperative image is divided into a plurality of first areas including areas corresponding to the first divided areas. A division unit 21d that divides the image into two divided regions, a phase indicating the degree of coincidence between the first divided region and the second divided region for each of the first divided region and the second divided region corresponding to the first divided region. The calculation unit 21e that calculates the information amount compares the mutual information amount with a noise determination threshold value indicating that the intraoperative image 50 includes predetermined noise, and whether or not the predetermined noise is included in the second divided region. And a noise removing unit 21g that generates the intraoperative image 50 from which noise is removed from the second divided region when it is determined that the noise is included in the second divided region. And a display control unit 21 h that performs control to display the intraoperative image 50 from which noise has been removed on the display unit 22, and a cross-sectional image obtained by imaging a subject position corresponding to the intraoperative image 50 from which noise has been removed from the three-dimensional image. Is replaced with an intraoperative image from which noise has been removed, and a replacement unit 21 i for storing in the server 25. These programs are stored in the hard disk of the processing unit 21, loaded into the main memory as appropriate, and executed by the CPU.

<Outline of processing>
The outline of the processing of the image display system 1 will be described based on FIG.

  The position information and posture information of the surgical instrument, for example, the endoscope 31 and the tip of the cantilever, are acquired by the position detection device and transmitted to the medical image photographing device, for example, the MRI apparatus 11. The MRI apparatus 11 that has received the position information and posture information captures a cross section based on the position information / posture information at high speed with Fluoroscopy, and transmits the cross section data (intraoperative image before noise removal) 50 to the image processing apparatus 20. The image processing device 20 performs noise removal processing on the intraoperative image 50 to generate an intraoperative image 51 from which noise has been removed, and replaces the intraoperative image 51 with a preoperative image corresponding thereto to obtain a three-dimensional volume. Update part of the data. The processing unit 21 performs organ extraction and three-dimensional image processing on the three-dimensional volume data updated as needed.

  On the display unit 22, an intraoperative image 50, an intraoperative image 51 from which noise has been removed, and a virtual endoscopic image 52 reconstructed based on three-dimensional volume data updated as needed are displayed in parallel. The mode of parallel display may be any combination of the above three images, or may be configured to display one image at a time and display the image to be viewed in the foreground as necessary. The surgeon continues the operation while viewing the displayed image processing result. The changing position information / attitude information at that time is transmitted to the MRI apparatus 11 as needed. In this manner, the cross-sectional image of the organ at the tip of the surgical instrument that changes from moment to moment is updated, and image guided surgery is performed with the processed image using the latest image.

  However, images taken during the operation include white noise 70 and the like as shown in FIG. 4 depending on what induces electrical noise such as an anesthesia machine, or obstructs the antenna characteristics of the reception coil of a resonance signal such as a magnetic substance. It includes spike noise 71 or a noise in which they are mixed. Furthermore, although the three-dimensional volume data is updated with the latest image, the image may be shifted without being able to follow the speed of organ deformation. Unless noise and organ deformation are recognized from these images and noise removal and organ registration cannot be dealt with, subsequent organ extraction and three-dimensional image processing may not be performed appropriately.

<Noise removal processing>
The noise removal process will be described with reference to FIG.

  Intraoperative images 50a and 50b taken during the operation have all frequency components equal to the image depending on what induces electrical noise such as an anesthesia machine or obstructs the antenna characteristics of the receiving coil of a resonance signal such as a magnetic substance. Included, white noise 70 in which pixel values corresponding to density values different from the actual density values are displayed, spike noise 71 in which pixel values are missing, or noise in which they are mixed. Furthermore, although the three-dimensional volume data is updated with the latest image, the image may be shifted without being able to follow the speed of organ deformation. Unless noise and organ deformation are recognized from these images and noise removal and organ registration cannot be dealt with, subsequent organ extraction and three-dimensional image processing may not be performed appropriately.

  Therefore, the intraoperative images 50a and 50b are imaged based on the position information and posture information of the surgical instrument 30, and at the same time, a cross-sectional image 80 having the same position and posture is reconstructed and created from pre-operative three-dimensional volume data. Using the preoperative image 80 as a reference, noise removal processing is performed on the intraoperative images 50a and 50b to create the intraoperative image 51 from which noise has been removed. A part of the preoperative three-dimensional volume data is updated with the intraoperative image 51, and preoperative three-dimensional volume data including the latest image is created. By doing so, the latest image corresponding to the organ deformation accompanying the surgical operation can be used as a reference.

  The maximum Mutual Information method is used for noise removal processing. This is a method used in the registration process, in which mutual information is regarded as the similarity between two images (probability distribution), and registration is performed by tuning the image position so that this is maximized. .

  The registration process will be described with reference to FIG.

  The original image (pre-operative image 80) is rotated / moved (step S50), and the evaluation standard between the original image (pre-operative image 80) and the reference image (in-operative image 50) is calculated (step S51). According to the evaluation criteria (corresponding to “mutual information amount” described later), the parameter (dx, dy, dz, θx, θy, θz) smoothly reaches the maximum value (or minimum value) as it approaches the true value. What you reach is good. The evaluation reference value is compared with a threshold value indicating an optimal state, and if the evaluation reference value is equal to or greater than the threshold value, the original image and the reference image are determined as matching images (step S52). If the evaluation reference value is less than the threshold value, the parameter is corrected (optimized) (step S53), and the process returns to step S52 again. By repeating these calculations, it is converged to a true value) and a matching image is created.

The evaluation criterion in the maximum mutual information method quantifies the similarity between images from the signal intensity (pixel count) of two images. The average mutual information I (A, B) is used to evaluate the similarity between images. In calculating the average mutual information amount, the information amount (joint entropy) H (A, B) of the two events A and B as a whole is the sum of the information amounts H (A) and H (B) of each event. In addition, considering that the information amount I (A, B) relating to one is included in the other, H (A) + H (B) −I (A, B). This I (A, B) is the mutual information amount. Considering application to an image, if the amount of information of event A is a _i (i = 1 to m), the appearance frequency of the signal intensity of the image (that is, a probability distribution p (a _i ) consisting of a normalized histogram) ) Expressed by a weighted average (expected value) (Expression 1). The amount of information of event B is obtained in the same way.

The combined average information amount (joint entropy) H (A, B) of two events A and B is that event A is a _i (i = 1 to _m ) and event B is b _j (j = 1 to n). If the simultaneous probability of taking p is p (a _i ∩b _j ), Equation 2 is obtained.

It becomes.

  The average mutual information amount is used for registration of different images because the region of uniform signal intensity in the different images shows a roughly uniform count distribution in the same region of other different images. Therefore, the property that the mutual information represents the commonality of the two images is expressed by a measure of the similarity between the images.

  FIG. 6 shows a noise information discrimination method using the mutual information amount. As shown in FIG. 4, an image in which the position and posture of the intraoperative image and the preoperative image coincide with each other is obtained based on the position information and posture information of the surgical instrument. It is assumed that the intraoperative image contains white noise and spike noise, and the preoperative image contains as little noise as possible. Since mutual information represents the similarity of two images (probability distribution), these two images are divided into arbitrary image sizes, and the joint probability density (Joint probability) and marginal probability distribution (Marginal probability) for each divided region. ) And the mutual information (Mutual information) is calculated based on these. The mutual information amount is a value indicating the commonality of the two images. When the noise is large, the mutual information amount is small, and when the noise is small, the mutual information amount is large. Further, the joint event probability of the mutual information amount is represented by a two-dimensional histogram (see FIG. 6A). In addition, the amount of mutual information is reduced by organ deformation in the image, and is recognized as a divided region to be registered. The simultaneous probability density, peripheral probability distribution, and mutual information can be calculated as shown in FIG.

  Next, a processing procedure of the image display system 1 will be described with reference to FIG.

In step S1, a three-dimensional image of the surgical site of the subject is photographed and stored using a medical image photographing device before surgery (S1).

  In step S2, surgery is started. The position information acquisition unit 21a of the image processing device 20 acquires position information and posture information of the surgical instrument 30 inserted into the subject from the position detection device 40 (S2). Similarly, the position detection apparatus 40 transmits position information and posture information to a medical image photographing apparatus, for example, the MRI apparatus 11.

In step S3, the first reading unit 21b reads the cross-sectional image corresponding to the position information from the three-dimensional image into the cross-sectional image corresponding to the position information acquired in S2 (S3).
In step S4, the MRI apparatus 11 that has received the position information and the posture information captures a cross section based on the position information / posture information at a high speed with Fluoroscopy, and generates an intraoperative image 50 (S4).

  In step S5, the second reading unit 21c reads the intraoperative image 50 photographed in S4. Then, the dividing unit 21d divides the preoperative and intraoperative images into regions of arbitrary image sizes (S5). The divided areas set in the preoperative image 60 are referred to as first divided areas, and the divided areas obtained by dividing the intraoperative image corresponding to the positions of the first divided areas are referred to as second divided areas.

  In step S6, the mutual information MI of the first divided area and the second divided area is calculated (S6).

  In step S7, it is determined from the mutual information MI <α whether there is a position shift due to organ deformation in the region (S7). α is a threshold that mutual information can take when there is organ deformation. If there is organ deformation and registration processing is necessary, the process proceeds to S8. When the mutual information MI is smaller than α, the process proceeds to step S8, and when it is equal to or larger than α, the process proceeds to step S9.

  In step S8, a registration process is performed. As the original volume image, a limited preoperative volume image in which the data amount is expanded in the depth direction within the same range as the divided area is used. An intraoperative divided region is used as a reference image. The process of this step will be described later with reference to FIG.

  In step S9, it is determined whether there is local noise. β is a determination value for determining noise (mainly white noise). When the mutual information MI is smaller than β, it is determined that there is noise and the process proceeds to S10. If the mutual information MI is equal to or greater than β, it is determined that there is no noise and the process proceeds to S11.

  In step S10, an adaptive filter is used as a local noise removal method. An overview of Adaptive Filter processing is shown. The Adaptive Filter creates a structure emphasized image (outline enhanced image) and a noise-removed image (smoothed image) from the input image, and obtains an output image by mixing both. The smoothing process is basically a process for removing noise by averaging pixel values in a matrix. As the matrix size is increased, the noise removal effect is increased, but the reduction in resolution is also increased. Therefore, in the Adaptive Filter, in order to suppress a decrease in resolution, a standard deviation from neighboring pixels is calculated, and the matrix size is changed according to the standard deviation. Since the granularity of the image becomes worse as the noise increases, the pixels having a larger standard deviation are smoothed with a larger matrix size.

  In the Adaptive Filter, the output pixel Po is determined by Equation (4).

なお、Ｐｉは注目画素、Ｗｉは重みである。注目画素に対する重みを注目画素からの距離だけでなく、画素値の差や標準偏差といった統計量に基づいて重みを決定する。つまり、マトリクス内で類似した値を持つ画素ほど重みを大きくして平均化する。

Pi is a pixel of interest and Wi is a weight. The weight for the target pixel is determined based on not only the distance from the target pixel but also a statistic such as a pixel value difference or standard deviation. That is, the pixels having similar values in the matrix are averaged by increasing the weight.

  In the Adaptive Filter, the mixing ratio of the smoothing component and the sharpening component is changed according to the standard deviation. Pixels with a larger standard deviation are mixed by increasing the ratio of the smoothing component and decreasing the ratio of the contour enhancement component. Since the standard deviation is remarkably high in the vicinity of the contour, the pixels having the standard deviation equal to or larger than the threshold are regarded as the contour, and the smoothing component ratio is minimized and the contour enhancement component is maximized.

  In this way, most noise information can be removed by first removing relatively large noise components in areas such as positioning due to organ deformation and spike noise, and then removing white noise. it can. In this method, a preoperative MRI image is taken as an example of a reference image, but a CT image or an ultrasound image may be used. The target intraoperative image may also be a CT image or an ultrasound image. The feature of this method is to determine the presence / absence of noise and the presence / absence of organ deformation by utilizing the fact that even in the case of different images, a substantially uniform count distribution is shown in the same region.

  In step S11, the divided regions from which noise has been removed are combined to generate an intraoperative image after noise removal.

  In step S12, the intraoperative image generated in S11 is replaced with a tomographic image corresponding to the intraoperative image of the volume data taken before the operation.

  In step S13, it is determined whether or not to end this process (S13). When the process ends, the process proceeds to “Y” and ends. If not completed, proceed to “N” and return to step S2 to newly acquire the position information and posture information of the surgical instrument, and newly acquired from the volume image replaced with the intraoperative image after noise removal in S3 A tomographic image corresponding to the position information is read.

  Next, the registration process in step S8 will be described with reference to FIG.

  The registration process in step S81 employs the method shown in FIG. As the original volume image, a limited preoperative volume image in which the data amount is expanded in the depth direction within the same range as the divided area is used. An intraoperative divided region is used as a reference image. Rotation and enlargement processing is performed on preoperative volume images, and alignment processing is performed in consideration of organ deformation. Thereafter, the mutual information amount is obtained again.

  In step S82, the mutual information amount is compared with a determination value γ for determining the presence or absence of spike noise (S82). If the mutual information amount is smaller than the determination value γ, it is determined that the divided area includes noise such that spike data is partially lost.

  In step S83, in the case of spike noise, it is determined from the probability distribution of the intraoperative image, assuming that the signal intensity of the noise is concentrated on a certain value and the histogram is biased. When represented by a two-dimensional histogram, the histogram of the intraoperative image is frequently counted in parallel to the histogram axis of the preoperative image. Thereby, noise is detected (S83).

  In step S84, the pixel having the spike noise signal intensity is replaced with the pixel value of the preoperative image (S84).

  In the above embodiment, α (determination of organ deformation), β (determination of white noise), and γ (determination of spike noise) are used as the determination values to be compared with the mutual information amount, but as appropriate according to the type of image noise. A determination value may be provided. In addition, an example of α> γ> β may be used as the magnitude relationship among α, β, and γ, but the present invention is not limited to this.

  According to the present embodiment, since intraoperative MRI images include image degradation due to high-speed imaging and white noise and spike noise due to external factors, even if image processing techniques such as organ extraction are applied to the images, It may not be applied as accurately as the previous image. Further, if a method of removing white noise, such as a conventional noise removal method, is employed, spike noise is enhanced. According to the present invention, various noise information can be removed regardless of white noise or spike noise, and an image processing method for a preoperative image can be applied to an intraoperative image.

Intraoperative image denoising device block diagram Block diagram of intraoperative image denoising program Schematic diagram of the process of the intraoperative image denoising device Schematic diagram of image quality improvement method Diagram showing the general process flow of the registration process The figure explaining the calculation method of the noise judgment processing using mutual information amount Flow chart showing processing procedure of intraoperative image noise removal processing The flowchart which shows the process sequence of step S8 of FIG.

Explanation of symbols

1: intraoperative image noise removal system, 10: medical imaging device, 20: image processing device, 30: surgical instrument, 40: position detection device

Claims (7)

A first reading step for reading a preoperative image obtained by imaging a surgical target region of a subject by a first medical imaging device before surgery;
A second reading step of reading an intraoperative image obtained by imaging a surgical target region of the subject by a second medical imaging apparatus during surgery;
A division step of dividing the preoperative image into a plurality of first divided regions having an arbitrary image size and dividing the intraoperative image into a plurality of second divided regions consisting of regions corresponding to the first divided regions;
A calculation step of calculating a mutual information amount indicating a degree of coincidence between the first divided area and the second divided area for each of the first divided area and the second divided area corresponding to the first divided area;
A determination step of comparing the mutual information amount with a noise determination threshold value indicating that predetermined noise is included in the intraoperative image, and determining whether the predetermined noise is included in the second divided region;
When it is determined that the predetermined noise is included in the second divided area, a noise removing step of removing the predetermined noise from the second divided area and generating an intraoperative image from which the noise is removed;
A display step of displaying an intraoperative image from which the predetermined noise has been removed;
A method for operating an image processing apparatus, comprising : The preoperative image is a three-dimensional image composed of a plurality of cross-sectional images obtained by imaging the surgical target site of the subject,
A position information acquisition step of acquiring position information indicating a surgery target site of the subject during the surgery;
In the first reading step, a cross-sectional image obtained by imaging the surgical target site of the subject is extracted or reconstructed based on the position information from the three-dimensional image and read as the preoperative image,
The method further includes the step of replacing the cross-sectional image corresponding to the intraoperative image from which the predetermined noise has been removed from the three-dimensional image with the intraoperative image from which the predetermined noise has been removed generated in the noise removing step.
The method of operating an image processing apparatus according to claim 1. In the determination step, a comparison is further made between the mutual information amount and a predetermined organ deformation determination value indicating an area where the organ deformation site is imaged, and an area where the organ deformation site is imaged in the second divided area Determine whether it is included,
When it is determined that the region obtained by imaging the organ deformation site is included in the second divided region, the first divided region is aligned so that the first divided region matches the second divided region. An alignment step;
A recalculation step of recalculating the mutual information amount between the aligned first divided region and the second divided region;
A re-determination step of comparing the recalculated mutual information amount with the noise determination threshold again to determine whether or not the predetermined noise is included in the second divided region, and
The noise removal step is executed based on a determination result of the redetermination step.
The method for operating an image processing apparatus according to claim 1, wherein: The predetermined noise includes at least one of spike noise in which a pixel value is missing or white noise having a pixel value that does not correspond to the density value of the subject,
The noise determination threshold includes at least one of a spike noise determination threshold indicating that the spike noise is included or a white noise determination threshold indicating that the white noise is included,
In the determination step, the mutual information amount is compared with at least one of the spike noise determination threshold or the white noise determination threshold;
When it is determined that the spike noise is included, the pixel value of the first divided region corresponding to the position where the spike noise is generated in the second divided region is acquired in the noise removing step. Is replaced with the pixel value of the pixel in which the spike noise occurs in the second divided region,
When it is determined that the white noise is included, in the noise removal step, a smoothed image obtained by smoothing the second divided region, and a contour emphasized image obtained by performing structure enhancement processing on the second divided region, , And combining the smoothed image and the contour-enhanced image to remove the white noise,
The method for operating an image processing apparatus according to claim 1, wherein: In the display step, the intraoperative image, a preoperative image captured at the subject position corresponding to the intraoperative image, a virtual endoscopic image reconstructed based on the three-dimensional image, and before the noise removal Displaying at least one of the intraoperative images side by side,
The method for operating an image processing apparatus according to claim 1, wherein: An image display program for causing a computer to execute the operation method of the image processing apparatus according to claim 1. First storage means for storing a three-dimensional image consisting of a plurality of cross-sectional images obtained by imaging a surgical target site of a subject by a first medical imaging device before surgery;
Position information acquisition means for acquiring position information indicating a surgery target site of the subject during the surgery;
First reading means for extracting or reconstructing a cross-sectional image obtained by imaging the surgical target region of the subject from the three-dimensional image based on the position information, and reading as a preoperative image;
A second reading means for reading an intraoperative image obtained by imaging a surgical target region of the subject by a second medical image capturing apparatus during surgery;
A dividing unit that divides the preoperative image into a plurality of first divided regions having an arbitrary image size, and divides the intraoperative image into a plurality of second divided regions made up of regions corresponding to the first divided regions;
Calculating means for calculating a mutual information amount indicating a degree of coincidence between the first divided area and the second divided area for each of the first divided area and the second divided area corresponding to the first divided area;
A determination unit that compares the mutual information amount with a noise determination threshold value indicating that predetermined noise is included in the intraoperative image, and determines whether or not the predetermined noise is included in the second divided region;
When it is determined that the noise is included in the second divided region, the noise is removed from the second divided region, and a noise removing unit that generates an intraoperative image from which noise is removed;
Display means for displaying an intraoperative image from which the noise has been removed;
The cross-sectional image obtained by imaging the subject position corresponding to the intraoperative image from which the noise has been removed in the three-dimensional image is replaced with the intraoperative image from which the noise has been removed generated in the noise removal step. Replacement means for storing in the storage means,
When the intraoperative image from which the noise has been removed is generated, the replacement unit replaces the tomographic image of the three-dimensional image with the intraoperative image from which the noise has been removed, and the first reading unit performs the replacement of the intraoperative image. Repeating the operation of extracting a cross-sectional image obtained by photographing the surgical site of the subject from a three-dimensional image including an image;
An image display device characterized by that.

Images