JP7277131B2 - Image processing device, image processing method, and program - Google Patents

Description

The disclosure of the present specification relates to an image processing device, an image processing method, and a program.

In the medical field, as a technique used when comparing two original images (hereinafter referred to as a reference image and a comparison image) captured at different times, an image that expresses the temporal change between the images as a difference (hereinafter referred to as a difference image). Subtraction techniques are known to obtain .

In subtraction technology, in order to generate a difference image that appropriately captures the temporal change of the imaging target, it is necessary to accurately perform registration processing that appropriately matches each pixel of the reference image and the comparison image as preprocessing.

Non-Patent Document 1 discloses a method of improving processing accuracy by using both an original image and a result of performing extraction processing of a specific region in the image for alignment processing. In this technique, an image representing the distance from the contour of the extraction region (hereinafter referred to as a distance image) is calculated, and both the original image and the distance image are used simultaneously in the registration process, thereby increasing the registration accuracy. there is

Min Chen, et. al. "Distance Transforms in Multi Channel MR Image Registration" Proc SPIE Int Soc Opt Eng. 2011 Mar 11:2011(7962):79621D.

However, in the technique described in Non-Patent Document 1, two types of images, an original image and a distance image based on the result of extracting a predetermined part, are used simultaneously in the alignment process, so the calculation cost associated with the process is high. There was a problem of getting taller.

The present disclosure aims at registering two images taken at different times with higher accuracy and lower computational cost.

It should be noted that it is not limited to the above object, but it is a function and effect derived from each configuration shown in the mode for carrying out the invention described later, and it is also disclosed in the present specification that there is a function and effect that cannot be obtained by the conventional technology. It can be positioned as one of the other purposes.

An image processing apparatus disclosed in the present specification includes an acquisition unit that acquires a first image and a second image of a subject captured at different times, and a predetermined image of the subject that is extracted from the first image. generating a first distance image having pixel values based on the distance from the contour of the area indicating the part and having a resolution lower than that of the first image; and extracting the predetermined part from the second image. A generation unit that generates a second distance image having pixel values based on the distance from the contour of the area showing and having a resolution lower than that of the second image, and the first distance image and the second distance image a first calculation unit that calculates first deformation information by aligning distance images; and a first calculation unit that aligns the first image and the second image based on the first deformation information. and a second calculation unit that calculates the second deformation information.

According to the disclosure herein, two images taken at different times can be registered with higher accuracy and lower computational cost.

2 is a diagram showing the configuration of functions of the image processing apparatus according to the first embodiment; FIG. 4 is a flow chart showing an example of a processing procedure of an image processing apparatus; It is a figure which shows an example of the image produced|generated by an image processing apparatus.

Hereinafter, exemplary embodiments of the image processing apparatus disclosed in this specification will be described in detail with reference to the drawings. However, the components described in this embodiment are merely examples, and the technical scope of the image processing apparatus disclosed in this specification is determined by the claims. It is not limited by form.

<First embodiment>
An image processing apparatus according to the first embodiment displays a plurality of three-dimensional images acquired by various imaging apparatuses (modalities) such as a computed tomography apparatus (hereinafter referred to as a CT (Computed Tomography) apparatus). It is a device that performs The image processing apparatus according to the present embodiment displays a two-dimensional tomographic image corresponding to two three-dimensional images captured at different points in time, and a difference image generated from the two two-dimensional tomographic images. It has an alignment function that computes corresponding positions between Furthermore, in the process of obtaining corresponding positions, the image processing apparatus according to the present embodiment utilizes the distance image calculated from the result of extracting the predetermined anatomical region together with the original image in the multi-resolution registration process. characterized by

In the present embodiment, an example of alignment processing of bones included in an image captured by a CT apparatus will be described. It should be noted that the effect of the present embodiment can be obtained by performing similar processing even when aligning images obtained by other modalities or other anatomical structures.

The configuration and processing of the image processing apparatus of this embodiment will be described below with reference to FIG.

FIG. 1 is a block diagram showing a configuration example of an image processing system (also referred to as a medical image processing system) including the image processing apparatus of this embodiment. The image processing system includes an image processing device 10, a network 21, and a database 22 as its functional configuration. The image processing apparatus 10 is communicably connected to a database 22 via a network 21 . The network 21 includes, for example, a LAN (Local Area Network) and a WAN (Wide Area Network).

The database 22 holds and manages images of the subject and information associated with the images. The image processing apparatus 10 can acquire images held in the database 22 via the network 21 . The image processing apparatus 10 includes a communication IF (Interface) 31 (communication section), a ROM (Read Only Memory) 32, a RAM (Random Access Memory) 33, a storage section 34, an operation section 35, a display section 36, and a control section 37. Prepare.

The communication IF 31 (communication section) is configured by a LAN card or the like, and realizes communication between an external device (for example, the database 22 or the like) and the image processing apparatus 10 . The ROM 32 is composed of a non-volatile memory or the like, and stores various programs. The RAM 33 is composed of a volatile memory or the like, and temporarily stores various kinds of information as data. The storage unit 34 is configured by an HDD (Hard Disk Drive) or the like, and stores various kinds of information as data. The operation unit 35 includes a keyboard, a mouse, a touch panel, and the like, and inputs instructions from a user (for example, a doctor) to various devices.

The display unit 36 is configured by a display or the like, and displays various information to the user. The control unit 37 is configured by a CPU (Central Processing Unit) or the like, and controls processing in the image processing apparatus 10 . The control unit 37 includes an acquisition unit 51, an extraction unit 52, a generation unit 53, a change unit 54, a first calculation unit 55, a second calculation unit 56, and a display control unit 57 as its functional configuration.

The acquisition unit 51 acquires from the database 22 a reference image and a comparison image to be subjected to alignment processing. In other words, the acquisition unit 51 corresponds to an example of an acquisition unit that acquires a first image and a second image of the subject captured at different times. Note that, in the present embodiment, an image captured in the past compared to the reference image is used as the comparison image. These images are three-dimensional images of the subject acquired by various modalities. In this embodiment, an example in which this image is a CT image will be described, but it may be another type of image. That is, the two images, the reference image and the comparison image, can be applied regardless of the type of image as long as they are three-dimensional images or two-dimensional images, and any image can be used as long as it is an image to be compared. There may be. That is, the images may be images of the same subject, or may be images of different people, such as images of a healthy person and a patient. Also, images of different modalities may be used.

The extraction unit 52 extracts a region indicating a predetermined site of the subject from each of the reference image and the comparison image acquired by the acquisition unit 51 .

The generation unit 53 calculates a distance value based on the extraction region for each of the regions indicating the predetermined part of the subject extracted by the extraction unit 52 regarding the reference image and the comparison image, and generates a first distance image and a second distance image. Generate an image. More specifically, using the information of the region extracted by the extraction unit 52, for example, a pixel value based on the distance value from the contour of the extraction region is calculated for each pixel in the two images, and the first distance Generate an image and a second range image. That is, the generation unit 53 generates from the first image a first distance image having pixel values based on the distance from the contour of the region extracted in the first image. Also, a second distance image having pixel values based on the distance from the contour of the region extracted in the second image is generated from the second image.

The changing unit 54 changes the input image to a preset resolution. In this embodiment, a process of reducing the resolution is performed on the first range image and the second range image.

The first calculation unit 55 calculates corresponding positions in the reference image and the comparison image based on the first distance image and the second distance image generated by the generation unit 53 or the image generated by the change unit 54. First image deformation information is calculated as the information to be displayed. That is, the first deformation information is calculated by aligning the first distance image and the second distance image.

The second calculation unit 56 calculates corresponding positions between the reference image and the comparison image based on the reference image and the comparison image acquired by the acquisition unit 51 and the first image deformation information calculated by the first calculation unit 55. Second image deformation information is calculated as the information to be displayed. That is, the second calculator 56 calculates the second deformation information by aligning the first image and the second image based on the first deformation information.

That is, the alignment process in this embodiment is performed in multiple steps (multiple resolutions) based on the range image and the original image (reference image and comparison image) whose resolution has been reduced. The basic concept of alignment processing in this embodiment is based on the concept called Coarse-to-Fine. In this way of thinking, after rough alignment processing is performed first, the results are used in the next alignment processing to perform detailed alignment. In other words, high-precision alignment is performed by connecting a plurality of alignment processes step by step.

In step S105 shown in the flowchart of FIG. 2, which will be described later, first, in order to roughly align the entire image, low-resolution range images are used to calculate first image deformation information as first-stage image deformation information. . Then, in step S106, in order to perform more detailed alignment as the second-stage alignment process, the alignment process is performed using a reference image and a comparison image having resolutions higher than those of the distance image used in the first-stage process. to calculate the second image deformation information.

Based on the result calculated by the second calculation unit 56, the display control unit 57 displays at least two or more of the input images in a display form that allows easy comparison of corresponding positions. 36 is displayed in the image display area. For example, in addition to the reference image and the comparative image, at least one of a deformed image obtained by deforming the comparative image based on the first deformation information or the second deformation information, and a difference image indicating the difference between the reference image and the deformed image. is displayed on the display.

Each component of the image processing apparatus 10 described above functions according to a computer program. For example, the control unit 37 (CPU) reads and executes a computer program stored in the ROM 32 or the storage unit 34 using the RAM 33 as a work area, thereby realizing the functions of each component. Note that some or all of the functions of the components of the image processing apparatus 10 may be realized by using a dedicated circuit. Moreover, some functions of the components of the control unit 37 may be realized by using a cloud computer.

For example, an arithmetic device located at a different location from the image processing device 10 is communicably connected to the image processing device 10 via the network 21, and data is transmitted and received between the image processing device 10 and the arithmetic device. 10 or the functions of the components of the controller 37 may be realized.

Next, an example of processing of the image processing apparatus 10 of FIG. 1 will be described using FIG.

FIG. 2 is a flow chart showing an example of the processing procedure of the image processing apparatus 10. As shown in FIG. In this embodiment, an example of alignment based on the anatomical features of bones will be described, but this embodiment can also be applied when setting other parts.

(Step S101: Image Acquisition/Display)
In step S<b>101 , when the user instructs image acquisition via the operation unit 35 , the acquisition unit 51 acquires the reference image and the comparison image specified by the user from the database 22 and stores them in the RAM 33 . In other words, step S101 corresponds to an example of an image acquisition step of acquiring a first image and a second image of the subject captured at different times. At this time, the display control section 57 may display the image acquired from the database 22 within the image display area of the display section 36 .

(Step S102: Extraction of anatomical structure (bone) region of each image)
In step S<b>102 , the extraction unit 52 extracts regions (anatomical structure regions) indicating predetermined parts of the subject from each of the reference image and the comparison image acquired by the acquisition unit 51 . In the present embodiment, the region indicating the predetermined site of the subject represents the bone region that is the alignment target. It should be noted that the bone region may be segmented and extracted, such as ribs, sternum, or pelvis. In this case, the bone region can be further subdivided and extracted by using the method of acquiring a feature amount representing the continuity of pixel values, which is described in, for example, Japanese Patent Laid-Open No. 2017-192691. Also, the region indicating the predetermined site of the subject is not limited to the above, and may be an organ such as the liver, lung field, or brain. For this region extraction, for example, known methods such as various organ region extraction methods using threshold processing, graph cutting, and the like may be used.

For example, when extracting a bone region from a CT image, in the case of a non-contrast-enhanced inspection image, a certain It can be roughly extracted by extracting pixel values equal to or greater than a threshold value (for example, 150 [HU]). That is, by extracting pixel values equal to or greater than a predetermined threshold value, a region indicating a predetermined portion of the subject is extracted.

After that, in order to fill the gaps in the extraction region, shape shaping may be performed using morphological operations such as filling processing and closing processing. Also, in this extraction process, not only the pixel values of the original image but also known feature values that can be calculated from the original image may be used. If area data to be aligned is prepared in advance, the image may be read here. Also, the processing after the area extraction described above may not be performed.

In the present embodiment, pixels (foreground pixels/graphic pixels) corresponding to extracted regions are assumed to store predetermined pixel values such as pixel value = 255, and pixels corresponding to non-extracted regions ( A prescribed pixel value such as pixel value=0 is also stored in the background pixel).

In other words, in the image representing the region indicating the predetermined part of the subject extracted in step S102, constant values are stored inside and outside the extraction region, respectively. The distribution becomes uniform. That is, by storing different binary pixel values in pixels within the extracted region and pixels outside the extracted region, a region indicating a predetermined site of the subject is extracted. Note that the predetermined pixel values stored inside and outside the extracted region are not limited to those described above, and pixel values different from the predetermined pixel values are stored inside and part of the extracted region. may be

After the image processing related to region extraction is completed, the extracting unit 52 extracts a region indicating a predetermined site of the subject from each of the reference image and the comparative image, and extracts the region as shown in FIG. 3(a). An image reflecting the result of region extraction is stored in the RAM 33 .

(Step S103: Generation of distance image)
In step S103, the generation unit 53 calculates the distance value from the contour of the region for each pixel based on the information of the region extracted in step S102 in the reference image and the comparison image. Based on the calculated distance values, a first distance image and a second distance image are generated and stored in the RAM 33 . That is, a first distance image having pixel values based on the distance from the contour of the region extracted in the first image is generated from the first image. Also, a second distance image having pixel values based on the distance from the contour of the region extracted in the second image is generated from the second image. This calculation shall be performed on images in which the extraction results are reflected for each of the reference image and the comparison image, and a known method such as distance conversion processing for the extraction region may be used. Referring to FIG. 3, a range image such as that shown in FIG. 3B is generated by subjecting an image that has undergone binary area extraction as shown in FIG. 3A to distance conversion processing.

In this embodiment, the Euclidean distance from the contour of the extraction region is calculated for each pixel of the image, but other distance indices such as Manhattan distance may be used. In addition, known value transformation such as reciprocal of the distance value from the contour or sigmoid transformation may be performed so that the value near the contour of the figure pixel becomes high. In addition, a positive or negative sign is given to the distance value to calculate a signed distance that distinguishes between the inside (foreground/figure, pixel value = 255 area) and the outside (background, pixel value = 0 area) of the extraction area. or an unsigned distance may be used.

The distance image (FIG. 3(b)) generated by performing the above processing is compared with the image (FIG. 3(a)) reflecting the result of region extraction generated in step S102. There are features with different density values inside (foreground) and outside (background) of the extraction region.

Specifically, while FIG. 3A is a binary image, FIG. 3B is a multivalued image. For example, in the distance image shown in FIG. 3B, pixels in the extracted region have larger pixel values as the Euclidean distance from the contour of the extracted region increases. At this time, by performing a known value transformation such as sigmoid transformation on the pixels in the extracted region, the pixels whose Euclidean distance from the outline of the extracted region is equal to or greater than a predetermined value will almost uniformly have the same pixel value. can be converted to

Alternatively, the distance image stores the first pixel value in the pixels in the extracted region whose Euclidean distance from the contour of the extracted region is included in the first range, and stores the first pixel value in the first range. A second pixel value larger than the first pixel value may be stored in a pixel included in the second range having a longer Euclidean distance than the pixel included in the range.

Also, in the distance image, pixels outside the extracted region have smaller pixel values as the Euclidean distance from the contour of the extracted region increases. At this time, pixels outside the extracted region are subjected to a known value conversion such as sigmoid conversion so that pixels whose Euclidean distance from the contour of the extracted region is equal to or greater than a predetermined value almost uniformly have the same pixel value. may be converted.

Alternatively, among at least some pixels outside the extracted region, the first pixel value is stored in the pixels whose Euclidean distance from the contour of the extracted region is included in the first range, and the first A second pixel value smaller than the first pixel value may be stored in a pixel included in the second range having a longer Euclidean distance than the pixel included in the range of . Note that the above density value conversion method and the like are merely an example, and the present invention is not limited to this.

The gradation expression as described above has the effect of widening the capture range against positional deviation in alignment processing. That is, by using the distance image for alignment processing, it is possible to cope with a larger positional deviation, and therefore, an improvement in the alignment accuracy can be expected even when the initial positional deviation is large.

In this embodiment, the distance image based on the distance value from the contour of the area extracted in step S102 is used. A calculable distance may be used. Alternatively, a distance image may be generated based on one or more feature points identified by a known method of extracting feature points from an image using a known feature amount such as SIFT (Scale-invariant feature transform). good. At this time, the image used when calculating the feature points may be an image reflecting the result of region extraction by the extraction unit 52, or may be the original image (reference image and comparison image) acquired by the acquisition unit 51. may Note that when the feature points are directly calculated from the original image acquired by the acquisition unit 51, the region extraction processing in step S102 may be omitted.

(Step S104: Image resolution reduction)
In step S104, the changing unit 54 performs a process of lowering the resolution of the first distance image and the second distance image generated by the generating unit 53, and generates an image whose image size is smaller than that of the input image. It is generated and stored in the RAM 33 .

In this embodiment, the resolution reduction process is performed for each distance image of the reference image and the comparison image, but this process may be performed after step S101 or step S102. You can do it multiple times step by step. That is, the flow may be such that the resolutions of the reference image and the comparison image are reduced, and a predetermined portion of the subject is extracted from the images with the reduced resolution.

That is, the order of steps S102 to S104 is not limited to the above. A first distance image having a resolution lower than that of the image should be generated. Also, a second distance image having pixel values based on the distance from the contour of the region extracted in the second image and having a resolution lower than that of the second image may be generated.

It is desirable that the order of the step of extracting the region and the step of lowering the resolution of the image be adaptively determined in consideration of the accuracy of the region extraction processing of the anatomical structure to be registered.

For example, if you want to extract detailed structures such as ribs and sternum, if the resolution is lowered too much before the area extraction process, the extraction process may become difficult. is desirable.

On the other hand, if you want to extract a relatively large structure such as the lung field, liver, or pelvis, you can extract the region relatively easily even with a low-resolution image, so lower the image resolution before the extraction process. good too. By doing so, fine noise can be removed. In addition, since the calculation cost of the area extraction processing and the distance calculation processing in the latter stage can be reduced, speeding up of the processing can be expected.

Alternatively, in step S102, predetermined information (for example, part information such as the name of an organ to be aligned) may be received as input information, and processing may be performed in a predetermined order of steps for each piece of predetermined information. For example, when the pelvis is specified as an organ to be aligned, extraction of an anatomical structure region and generation of a range image are performed after resolution reduction. Further, when ribs are specified as an organ to be aligned, resolution reduction processing is performed after extraction of an anatomical structure region and generation of a range image. That is, step S102 corresponds to an example of a region information obtaining step of obtaining information on a predetermined region of the subject. By doing so, it is possible to automatically switch to an appropriate processing procedure based on predetermined information given as an input.

(Step S105: Alignment using distance image)
In step S105, the first calculation unit 55 performs registration processing of the reference image and the comparison image based on the low-resolution first range image and the second range image generated in step S104. That is, as the first image deformation information, for each pixel of the low-resolution first range image, the corresponding position in the low-resolution second range image is calculated. Here, the first image deformation information is, for example, information having the same resolution as that of the low-resolution first distance image, and (low-resolution second distance image ) can be generated as a displacement field image that holds displacement vectors. Note that if the image resolution reduction process in step S104 is performed before the distance image generation process in step S103, the output image in step S103 is a low-resolution distance image. should be used.

Then, the first calculation unit 55 generates an image (first deformation image) is stored in the RAM 33 . It should be noted that it is not always necessary to generate this first modified image, and it is also possible to simply output and store the first image modification information. Alternatively, it is also possible to simply execute processing using the first image transformation information as input and output the result to the second calculator 56 .

The reason why the low-resolution distance image is used for the first-stage alignment processing performed in step S105 among the multiple-stage alignments performed in this embodiment is as follows. As described above, this first-stage alignment process is intended for rough alignment, so high-resolution images with detailed information are not suitable for processing.

The reason for this is that in the initial stage before alignment, the positions of the two images are often largely misaligned, and in such cases, the matching/mismatching information for each pixel unit value of the high-resolution image is used. This is because it becomes noise in the alignment process. That is, the risk of falling into a local optimum in alignment processing increases. Using a low-resolution image has the effect of using rough image information intensively, so there is an advantage in using a low-resolution image in the initial stage of alignment processing.

Furthermore, since the number of pixels to be processed is reduced by using an image with reduced resolution, a reduction in calculation cost can be expected. These advantages are the same as general multi-resolution registration using image pyramids and the like.

Next, in this embodiment, in order to cope with the case where the alignment error is large in the initial state, when performing the alignment process at a low resolution, instead of the original image such as the reference image or the comparison image, the distance Images will be used. By using a distance image that can be calculated from the result of extracting a region indicating a predetermined part of the subject, it is possible to perform processing specialized for the region indicating the predetermined part of the subject when calculating a rough corresponding position. .

For example, when aligning bones, the depth image obtained by extracting the bone region is used instead of the original image showing organs other than the bones. can be aligned. The reason why the distance image is used instead of the extracted image directly is that the distance image can cope with a larger initial alignment error.

The alignment process between these two distance images (the distance image of the reference image and the distance image of the comparison image) can be performed by a linear alignment method such as affine transformation, FFD (Free Form Deformation), LDDMM (Large Deformation Diffeomorphic Metric Mapping) or a combination thereof.

Furthermore, these alignment processes and combinations of alignment processes may be repeated multiple times. Further, the positioning process itself using the distance image in step S105 may be processed within the framework of multi-resolution.

(Step S106: Alignment using original image)
In step S106, the second image deformation information is calculated by aligning the reference image and the comparison image based on the first image deformation information. Specifically, after calculating the first image deformation information, the second calculation unit 56 uses the first image deformation information as an initial value to generate an original image with a resolution higher than that of the image used in the first stage of processing. Using the image, alignment processing is performed to obtain second image deformation information, which is more detailed deformation information. That is, as the second image deformation information, for each pixel of the reference image, the corresponding position in the comparative image is calculated. Here, the second image deformation information is, for example, information having the same resolution as that of the reference image, and can be generated as a displacement field image holding a displacement vector (to the corresponding pixel of the comparison image) in each pixel of the reference image. Note that the above-described known alignment method may be used for this detailed alignment process as well, and a method different from the method for calculating the first image deformation information may be used. In addition to the method of comparing the reference image and the comparison image as they are in the original image, alignment may be performed using images after performing preprocessing such as noise removal and edge enhancement on the respective images. Also, in the case where the reference image and the comparison image are images obtained by different modalities, for example, various methods of registration between different modalities using mutual information may be used. Since the first image deformation information is calculated using a low-resolution image, the resolution of the deformation information may be changed in the second-stage processing according to the necessity of the registration method used in the second-stage registration processing. You may match with the resolution of the image to be used.

Here, the reason why a high-resolution range image is not used for alignment in step S106 is that range images have the following disadvantages. Alignment based on a range image or an extraction result of a region indicating a predetermined part of the subject has a point that the accuracy of the alignment process strongly depends on the accuracy of the region extraction process due to its nature. This is not a problem if the region extraction process is accurate enough and the process is robust, but it can be a problem in the detailed registration process if there are oversights or oversights in the results of the region extraction process. may become.

For example, if an area that represents a detailed structure of an anatomical structure is overlooked in the area extraction process, the area is not treated as a target for registration processing, so the overlooked area cannot be registered in the extraction process. There is a problem that the In general, considering the problem of extracting a certain anatomical structure, it can be imagined that automatic extraction of fine structures is difficult.

Therefore, if the accuracy of the area extraction process is insufficient for the accuracy required for the alignment process, it is considered preferable to use an original image with a larger amount of information at the stage of detailed alignment. From these facts, it can be seen that when combining the processing of rough alignment and the processing of detailed alignment, the overall processing performance can be improved by selecting the optimum image at each stage. In the present embodiment, a distance image from the contour of the bone is used for rough alignment, and an original image with a larger amount of information is used for detailed alignment. It realizes a high degree of alignment processing.

Note that the alignment process in this step may use the method of Non-Patent Document 1 instead of using only the original image. Even in this case, the processing in step S105 reduces the positional deviation of the alignment target, so the effect of reducing the calculation cost until obtaining the final alignment result is maintained.

Finally, the second calculation unit 56 generates an image (deformed image) by performing image deformation so as to match the comparison image with the reference image based on the second image deformation information and the second image deformation information. is stored in the RAM 33. It should be noted that it is not always necessary to generate this deformed image, and it is sufficient to simply output and store the second image deformation information. Alternatively, it is also possible to simply execute processing using the second image transformation information as input and output the result. In one example, the result of calculating the difference between the reference image and the deformed image may be output. Also, an ROI (Region of Interest) on the reference image corresponding to the ROI (Region of Interest) on the comparison image may be derived based on the second deformation information. Alternatively, the slices of the second image that substantially correspond to the slices of the first image may be identified based on the second deformation information, and the correspondence information thereof may be output.

(Step S107: Image display based on alignment result)
In step S107, the display control unit 57 causes the deformed image to be displayed in the image display area together with the reference image and the comparison image, which are the input images. At this time, an image representing the difference between the two images, that is, a difference image (for example, an image obtained by subtracting the deformed image from the reference image) may be calculated and displayed within the image display area.

According to the present embodiment, both the low-resolution range image and the high-resolution original image are used step-by-step for the multi-resolution registration process, so that two images taken at different times can be combined at a lower computational cost. , there is an effect that the user can be provided with a result of alignment with higher accuracy.

According to this embodiment, within the framework of multi-resolution registration processing, by using a range image that is more suitable for rough registration than the original image in combination with the original image, the calculation cost can be reduced. , it is possible to provide the user with highly accurate alignment results.

(Modification)
In the above-described embodiment, before the first calculator 55 performs the process of calculating the first deformation information in the process of step S105, a distance image having a resolution lower than that of the original image is generated and used for the alignment process. An example to use is shown. However, the procedure up to calculation of the deformation information for performing rough alignment at low calculation cost is not necessarily limited to this. For example, instead of changing the resolution before the alignment process, the deformation information may be calculated using a method that achieves the same effect during the alignment process in step S105. More specifically, in the alignment process in step S105, when comparing the first range image to be processed and the second range image, at least one image (for example, the first range image) may be used to reduce the number of pixels to be processed by sampling and referring to . For example, when calculating the first deformation information, the first distance image is sampled so as to have a resolution lower than that of the original image and referred to, and the first distance image and the first distance image are referenced at the sampled positions (pixels). Calculation of the corresponding positions of the two distance images may be performed. At this time, the resolution of the first deformation information is the same as that of the first distance image. That is, since the resolution is lower than that of the second deformation information, it is possible to maintain the effect of providing the user with highly accurate alignment results at a lower calculation cost, as in the first embodiment.

<Other embodiments>
In addition, the technology disclosed herein can be embodied as, for example, a system, device, method, program, recording medium (storage medium), or the like. Specifically, it may be applied to a system composed of a plurality of devices (for example, a host computer, an interface device, an imaging device, a web application, etc.), or may be applied to a device composed of a single device. good.

Moreover, it goes without saying that the object of the present invention is achieved by the following. That is, a recording medium (or storage medium) recording software program code (computer program) for realizing the functions of the above-described embodiments is supplied to the system or device. Such a storage medium is, of course, a computer-readable storage medium. Then, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the recording medium. In this case, the program code itself read from the recording medium implements the functions of the above-described embodiments, and the recording medium recording the program code constitutes the present invention.

10 image processing device 21 network 22 database 31 communication IF
32 ROMs
33 RAM
34 storage unit 35 operation unit 36 display unit 37 control unit 51 acquisition unit 52 extraction unit 53 generation unit 54 change unit 55 first calculation unit 56 second calculation unit 57 display control unit

Claims (19)

an acquisition unit that acquires a first image and a second image obtained by imaging a subject at different times;
generating a first distance image having pixel values based on a distance from a contour of a region indicating a predetermined part of the subject extracted in the first image and having a resolution lower than that of the first image; generating a second distance image having pixel values based on the distance from the contour of the region representing the predetermined part extracted in the second image and having a resolution lower than that of the second image; Department and
a first calculation unit that calculates first deformation information by aligning the first distance image and the second distance image;
a second calculation unit that calculates second deformation information by aligning the first image and the second image based on the first deformation information;
An image processing device comprising: an acquisition unit that acquires a first image and a second image obtained by imaging a subject at different times;
A first distance image having pixel values based on a distance from a contour of a region indicating a predetermined site of a subject extracted in the first image, and the predetermined site extracted in the second image. a generation unit that generates a second distance image having pixel values based on the distance from the contour of the area showing
a first calculation unit that calculates first deformation information by aligning the first distance image and the second distance image;
a second calculation unit that calculates second deformation information by aligning the first image and the second image based on the first deformation information;
with
An image processing apparatus, wherein the resolution of the first deformation information is lower than the resolution of the second deformation information. The first calculation unit and the second calculation unit are configured to calculate, of the first image and the second image, the image that was captured earlier than the one image, with respect to the one image. 3. The image processing apparatus according to claim 1, wherein the first deformation information and the second deformation information are calculated by deforming the image. 4. The image processing apparatus according to any one of claims 1 to 3, wherein the generation unit extracts a region indicating the predetermined site by extracting pixel values equal to or greater than a predetermined threshold. 5. The image according to any one of claims 1 to 4 , wherein the generating unit generates the distance image having pixel values based on a Euclidean distance from a contour of the region indicating the predetermined part. processing equipment. 6. The distance image according to claim 5 , wherein the generation unit generates the distance image such that the pixel values of at least some of the pixels in the region indicating the predetermined part increase as the Euclidean distance increases. The described image processing device. 7. The generating unit according to claim 5 , wherein the generating unit generates the distance image in which the pixels having the Euclidean distance equal to or greater than a predetermined value in the region indicating the predetermined portion uniformly have the same pixel value. Image processing device. 5. The generating unit generates the distance image such that the pixel values of at least some pixels outside the region indicating the predetermined part become smaller as the Euclidean distance becomes longer. 8. The image processing device according to any one of 7 . 9. The image processing according to claim 8 , wherein the generation unit generates the distance image in which pixels having the Euclidean distance equal to or greater than a predetermined value outside the region indicating the predetermined portion uniformly have the same pixel value. Device. 10. The image processing apparatus according to claim 1, wherein the second calculator calculates the second deformation information using the first deformation information as an initial value. At least one of a deformed image obtained by deforming the second image based on the first deformation information or the second deformation information, and a difference image indicating a difference between the first image and the deformed image. 11. The image processing apparatus according to any one of claims 1 to 10 , further comprising a display control section that causes the display section to display. 12. The image processing apparatus according to any one of claims 1 to 11 , wherein the predetermined site is at least one of a bone, liver, lung field, and brain. an image acquisition step of acquiring a first image and a second image of the subject captured at different times;
generating a first distance image having pixel values based on a distance from a contour of a region indicating a predetermined part of the subject extracted in the first image and having a resolution lower than that of the first image; a generating step of generating a second distance image having pixel values based on distances from the contour of the region extracted in the second image and having a resolution lower than that of the second image;
a first calculation step of calculating first deformation information by aligning the first distance image and the second distance image;
a second calculation step of calculating second deformation information by aligning the first image and the second image based on the first deformation information;
An image processing method comprising: The generating step generates the first distance image by reducing the resolution after generating an image having pixel values based on the distance from the contour of the region in the first image, and 2. The second distance image is generated by reducing the resolution after generating an image having pixel values based on the distance from the contour of the area. image processing method. The generating step extracts the region in the first image after reducing the resolution of the first image, and generates the first distance image having pixel values based on the distance from the contour of the region. and extracting the region in the second image after reducing the resolution of the second image to generate the second range image having pixel values based on the distance from the contour of the region. 14. The image processing method according to claim 13 , wherein: further comprising a site information acquiring step of acquiring information on a predetermined site of the subject;
In the generating step, when the acquired part of the subject is a rib or a sternum, after generating an image having pixel values based on the distance from the contour of the region in the first image, generating the first distance image by reducing the resolution, and reducing the resolution after generating an image in the second image having pixel values based on the distance from the contour of the region; The image processing method according to any one of claims 13 to 15 , wherein a second distance image is generated. In the generating step, when the acquired part of the subject is the lung field, the liver, or the pelvis, the first image includes an image having pixel values based on the distance from the contour of the area. generating the first distance image by reducing the resolution after generation, and reducing the resolution after generating an image in the second image having pixel values based on distance from the contour of the region; 17. The image processing method according to claim 16 , wherein the second distance image is generated by A method of aligning first and second images of a subject, comprising:
A first distance image generated from the first image and having pixel values based on a distance from a contour of a region indicating a predetermined portion of the subject; and the second image generated from the second image. and a second range image having pixel values based on distances from the contour of the region indicating the predetermined portion of the subject, and a lower resolution than the first and second images. a first alignment step of calculating first deformation information of resolution;
A resolution higher than that of the first deformation information by registering the first image and the second image using the first deformation information acquired by the first registration step a second alignment step of calculating second deformation information having
An image processing method comprising: A program causing a computer to execute each means of the image processing apparatus according to any one of claims 1 to 12 .

Images