JPH04341248A - Image distortion correction device, and inter-image arithmetic unit, and image motion calculating device - Google Patents

Abstract

PURPOSE:To calculate and correct the distortion even between images which are separated from each other timewise, and also, varied frequently by calculating the shift amount between each adjacent image, and also, correcting the shift of an image at the time of reference and an image after an arbitrary time by using an accumulated value of the shift amount between each adjacent image existing between both its images. CONSTITUTION:First of all, each adjacent image after the time of reference is stored in memories 5, 6, while updating it, a distortion vector of a section unit is derived, and it is stored in a memory 31. Subsequently, this distortion vector is accumulated and added by an adding part 32 with regard to an image and the adjacent one in-between. Next, by the distortion vector obtained by accumulation and addition, an image at the time of final updating is corrected, and stored in a memory 37. Thereafter, a difference of this corrected image, and the image at the time of reference stored in a memory 4 is derived by a difference part 9. A result of this difference is a result of inter-image operation, and becomes a dynamic state function image.

Description

[Detailed description of the invention]

0001

[Industrial Application Field] The present invention relates to an image distortion correction device that corrects the deviation between continuously given images, an inter-image arithmetic device that uses the corrected images, and, moreover, generally calculates image movement. The present invention relates to a calculation device for calculating data.

0002

2. Description of the Related Art In the field of medicine, a dynamic functional image of a subject is obtained and used for diagnosis. The dynamic functional image is obtained by measuring the same region of the subject subject to dynamic movement at two different times, and determining the mutual relationship between the images at the two times. The relationship between images refers to, for example, a process of determining a difference between target images and positioning the difference result as a dynamic functional image. However, in moving organs, movement information is superimposed on the desired functional information, causing a loss in the reliability of the original dynamic functional image.

[0003] The applicant has already filed an application to eliminate this unnecessary movement (Japanese Patent Application No. 1983-15).
No. 1802).

In this prior application, the distortion between two images is determined by cross-correlation, and the image is corrected using this distortion to remove the amount of motion. However, a correlation can be established in the case of distortions that are close together and have relatively little change, but a correlation cannot be established in the case of two consecutive images that are not close together and have some degree of temporal change. There are some problems.

[0004] An object of the present invention is to calculate image distortion that can be corrected even between images that are temporally distant from each other and have many changes in a large number of consecutive images that are obtained temporally continuously. The present invention provides a correction device and an inter-image calculation device that utilizes the correction results.

A further object of the present invention is to provide an apparatus for calculating the amount of motion between images that are not only temporally continuous.

[0006]

[Means for Solving the Problems] The image distortion calculation device of the present invention calculates the amount of deviation between adjacent images in temporally continuous images, and calculates the amount of deviation between images at a reference time and after an arbitrary time. The amount of deviation from the image is given by the cumulative value of the amount of deviation between adjacent images that exists between the two images, and the deviation of the image after the above arbitrary time is corrected using the cumulative value. Item 1).

Furthermore, the inter-image calculation device of the present invention performs inter-image calculation between the image obtained by this correction and the first image of the temporally continuous images (claim 2). .

Furthermore, the image distortion calculation device of the present invention includes first and second memories that mutually store adjacent images after a reference time among temporally continuous images while updating them; means for extracting adjacent images in the first and second memories within prescribed divisions and calculating cross-correlation for each corresponding division to obtain distortion vectors for each division; and updating of the distortion vectors for the adjacent images. and means for correcting the deviation of the last image at the time of any update using the cumulatively added distortion vector obtained after any update (Claim 3).

Furthermore, the inter-image arithmetic device of the present invention includes first and second memories that mutually store adjacent images after a reference time among temporally continuous images while updating them; a third memory for storing an image at a reference time;
Means for extracting adjacent images in the second memory within a prescribed section and calculating cross-correlation for each corresponding section to obtain a distortion vector for each section, and accumulating the distortion vector each time the adjacent images are updated. means for correcting the image deviation of the image at the time of the arbitrary update by the cumulatively added distortion vector obtained after the arbitrary update, and the image at the reference time in the third memory and the image after the correction. and means for performing inter-image calculations with the images (claim 4).

Furthermore, the image motion calculation device of the present invention calculates the amount of deviation between adjacent images among continuously given images, and calculates the amount of deviation between a reference image and an arbitrary image based on the amount of deviation between both images. (claim 5).

[0011]

[Operation] According to the present invention, using a large number of consecutive images,
By calculating the distortion of two adjacent images and cumulatively adding the amounts of distortion, it is possible to automatically correct the distortion of images that are not close and have temporal changes (Claim 1) , 3).

Furthermore, according to the present invention, inter-image calculations can be performed using the corrected images (claims 2 and 4).

Furthermore, according to the present invention, the movement of images between moving images is calculated (claim 5).

[0014]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 shows an embodiment of the image-to-image calculation (difference) device of the present invention. The detection system 1 detects transmitted X-rays from the subject. The preprocessing system 2 is a preprocessing circuit that can perform analog processing, and performs noise processing and waveform shaping. The AD converter 3 takes in the processing output from the preprocessing system 2 and performs AD conversion.

Memory 4, memory 5, and memory 6 are image memories; memory 4 stores image A at the earliest time (reference time), and memory 5 and memory 6 are read continuously. Images B and C that are close to each other are stored. In the field of digital subtraction angiography, the image stored in the memory 4 is called a mask image, and the first image stored in the memory 5 is the same mask image as the image stored in the memory 4, and thereafter Images stored in are called live images. Furthermore, the images stored in the memory 6 are live images.

Here, the mask image stored in the memory 4 becomes a reference image for inter-image calculation (in this embodiment, inter-image difference). Further, in this embodiment, adjacent images are images that are adjacent to each other, but in general, it is arbitrary.

In this embodiment, adjacent images after the reference time are stored in the memories 5 and 6 while being updated, and a distortion vector for each segment is obtained (memory 31). Cumulatively add for (addition unit 32)
The image at the last update is corrected using the obtained cumulatively added distortion vector (memory 37), and the difference between this corrected image and the reference image stored in the memory 4 is calculated (difference section 9).
). This difference result is the inter-image calculation result and becomes a dynamic functional image.

The configuration and operation of the cutting sections 10, 11 and subsequent sections will be explained below. (1) Cutting units 10 and 11...The cutting units 10 and 11 cut out adjacent images stored in the memories 5 and 6 according to prescribed divisions (sample windows). In Figure 2,
Two-dimensional original images R(x,
y) and an example of cutting out in m×n division D. The cutting is performed in the raster scan order from upper left to upper right and from upper left to lower left. In this way, a plurality of sections D can be cut out from one original image R (x, y). Furthermore, the cutout parts 10 and 11 are
The extraction window function W(i, j) and the image data within the extraction section are multiplied (multiplied, hereinafter the same) to perform processing to reduce discontinuities in the boundary area of the extraction section. That is, in section D, when comparing the center region of the section and the region near the boundary, the near-boundary region is near the border of the section, and since the sections separated by the border are treated independently, the data There is strong discontinuity, and the central region of the section is far from the section boundary, so there is strong continuity as data. Therefore, in order to remove the adverse effects of discontinuities outside the central region due to image cutting, it is necessary to emphasize the central region and compress the regions near the boundaries. The function for this processing is the wind function W(i,j), which has a higher level toward the center and a lower level toward the periphery, as shown in FIG. 2. This wind function W
By performing integration for each corresponding coordinate between (i, j) and the image data within the section, discontinuity in the boundary region of the cutout section can be substantially eliminated.

Expressing this mathematically, the processed cutout image data Qpq(i,j) is expressed as follows.

[Equation 1] Qpq (i, j) = W (i, j) × R (i + m
p, j+nq) Here, p and q are coordinates indicating a division, and specifically, the starting position of the division is selected as (p, q). The cropping window function W(i,j) must have small side lobes and be a finite function in order to reduce errors due to the effect of image cropping in post-processing. Now, if we express the cutout wind function W(i, j) in polar coordinates, we get

[Formula 2] W(i, j)=G(r). When the origin of polar coordinates is set at the center position of each section, r is

[Math. 3] Furthermore, when G(r) is given by Blackman's function (the maximum side lobe is -40 dB),

[Math 4] becomes. When given by trigonometric functions,

[Equation 5] However, the maximum sidelobe in this case is -28
It is dB.

The specific processing procedure of Equation 1 in the above cutout units 10 and 11 is as follows. Each coordinate (i,
j) Correspondingly, the cutout window function is stored in a memory different from the memories 5 and 6. The window function of this memory and the partition correspondence data of memories 5 and 6 are integrated for each coordinate unit. This integration result becomes the calculation result of Equation 1. The integration results are temporarily stored in a buffer memory and prepared for the next processing.

The operations of the cutout units 10 and 11 have been explained simultaneously, but the only difference between the two is that the processing target is an image in the memory 5 or 6, and the other processes are exactly the same. The same function value may be set for the extraction window function W(i, j).

(2) FFT calculation units 12, 13...In this embodiment, a cross-correlation (correlation unit 16) is taken between the image data of corresponding sections of the memories 5 and 6. FFT calculation units 12 and 13 are provided to perform the calculation in the frequency domain. Further, in the case of the frequency domain, since it is possible to easily remove the adverse effects of segmentation, FFT calculation units 12 and 13 are provided. These FFT calculation units 12 and 13
By providing this, the image data of each section is converted into the frequency domain.

(3) Spatial filter 14: Provided for the purpose of removing the adverse effects of segmentation, emphasizing the spatial frequency of interest, and removing noise. As a result, the cross-correlation unit 16
We aimed to improve the processing sensitivity. This spatial filter 14
is a first spatial filter formed by the integrating section 21, the memory 24, and the common memory 20, and the integrating section 22 and the memory 25.
and a second spatial filter formed by the common memory 20 and the common memory 20. They correspond to memory 5 and memory 6, respectively. The characteristics of these two spatial filters are the same, and have the characteristics as shown in FIGS. 3 and 4. It may have different characteristics. FIG. 3 shows a characteristic diagram in which the horizontal axis represents the frequency ω and the vertical axis represents the frequency spectrum F(ω). Nyquist on the horizontal axis means the maximum frequency that can be expressed. In this embodiment, it refers to the maximum frequency that can be expressed by the extracted pixel space. FIG. 4 shows the relationship between the cropped image and the Nyquist frequency. When r is maximum (+rmax)
The maximum frequency on the right side of is the positive Nyquist frequency.

In the characteristic F1 of FIG. 3, a region E1 represents a region for reducing the effect of the cutting window, and a region E2 represents a region for noise removal. Region E3 is a portion that emphasizes the spatial frequency component of interest. However, it is 0.1 to 0.2 LP/cm. Here, LP means line pair. Furthermore, in FIG. 3, the frequency of the cutting window is reduced. 0
This is what the bold line spectrum extending from to region E1 means.

The above spatial frequency characteristics are stored in the common memory 20, and the integration units 21 and 22 perform the FFT calculation unit 12.
, 13 and the output of the memory 20 are multiplied by a complex number. The two multiplications in the integration units 21 and 22 are to multiply the frequency spectrum, which is the result of FFT calculation, by the spatial frequency characteristic shown in FIG. Obtain the frequency spectrum. The memories 24 and 25 store the spectra of the multiplication results of the integration units 21 and 22. This storage is performed in coordinate units of divided sections. The frequency spectrum that reflects the characteristics shown in Figure 3 is one in which the cut-out wind frequency is reduced, that is, the negative influence of the cut-out window is removed, and noise is removed. have.

(4) Cross-correlation unit 16: calculates cross-correlation in the frequency domain for each section. Cross-correlation in the frequency domain means converting one of the objects of cross-correlation into a complex conjugate and integrating this with the other. The purpose of cross-correlation is
This is to use it for calculating the amount of deviation.

The cross-correlation unit 16 includes an integration calculation unit 26 that performs complex conjugation and integration, and a memory 27 that stores the result. The cross-correlation target is the result of spatial filter processing of the memories 24 and 25, and the target of complex conjugation is one of the memories 24 and 25. Multiple cross-correlation results are obtained for each segment.

(5) I-FFT calculation unit 17...This is an inverse Fourier transform unit, which calculates the partial cross-correlation results between images B and C,
This is a process of returning the frequency domain to the original two-dimensional space.

(6) Peak detection section 30...This is for finding the distortion vector for each section. That is, I-FF
Among the cross-correlation values obtained in each section of the cross-correlation between the adjacent images B and C obtained by the T calculation unit 17, the maximum one is detected. In this way, one peak value is found for each section, and the position and magnitude at that time are used as a distortion vector for each section, and this is stored in the memory 31 corresponding to the section.

(7) Cumulative addition unit 19...This is for cumulatively adding the distortion vectors found for each adjacent image every time the image is updated to obtain the cumulative addition result of the distortion vectors. 32, and a memory 33 for storing cumulative addition vectors up to the last update for accumulation. The adding unit 32 adds the current distortion vector between adjacent images to the cumulative added distortion vector stored in the memory 33 up to the previous update, and the memory 33 stores this in preparation for the next update. I did it like that. Note that this addition is vector addition that takes position and directionality into account.

(8) Interpolation calculation unit 34...This target is the cumulative distortion vector obtained at the end of all updates. here,
The cumulative distortion vector at the end of all updates refers to the cumulative distortion vector from the reference image to the final image Q. The final image Q is one image in the inter-image calculation, which is the target of the difference calculation between the final image and the reference image. The interpolation calculation unit 34 distributes the cumulative distortion vector obtained in each section to each pixel of an image equivalent to one screen (performed by so-called interpolation processing).
. The memory 35 stores the strain coordinates that are the result.

(9) Correction calculation unit 36 . . . corrects the distortion of the final image Q in the memory 6 using the distortion vector (distortion coordinates) in units of pixels in the memory 35. This distortion correction is a process of adjusting the image Q to the distortion of the reference image A. As a result, the distortions of image A and image Q are the same. The memory 37 stores this corrected image Q.

(10) Difference calculation unit 9...Performs difference calculation as inter-image calculation. That is, the difference between the reference image A and the final image Q is calculated. At this time, since distortion due to movement has been removed, a correct difference image can be obtained. This concludes the explanation of the embodiment shown in FIG.

Next, FIG. 5 shows an example of distortion vector processing. In this processing example, from the memories 24 and 25 in FIG.
This is an example of processing up to finding the distortion vector (distortion coordinates) for each pixel in step 5. In FIG. 5, the block 40 refers to the calculation unit 2 in FIG.
6 to the peak detection unit 30 are collectively displayed. In FIG. 5, the memory 24 shows 5×5 sections, and for each section, spatial filter results B11 to B55 corresponding to the section of the image B are obtained. Similarly, the memory 25 stores spatial filter results C11 to C55 corresponding to the classification of the image C.
is obtained. The memory 31 stores distortion vectors D11 to D55 between images B and C corresponding to the classification. memory 33
stores cumulative distortion vectors D'11 to D'55. The memory 35 stores distortion vectors (ie, distortion coordinates) in pixel units.

FIG. 6 shows an example of the cumulative distortion vector. For the mask image M, the image is updated as L1 → L2 → L3 → L4 → L5, and the distortion vector A1 is changed between adjacent images.
, A2, A3, A4, and A5 are obtained. This is an example of the prior application mentioned in the conventional example. In this embodiment, this distortion vector is cumulatively added, and the cumulative addition vector ZM, Z1=ZM+A
1, Z2=Z1+A2, Z3=Z2+A3, Z4=Z3
+A4, Z5=Z4+A5, are obtained one after another. Then, to perform inter-image calculation between two images M and L5,
Image L5 may be corrected by distortion vector Z5, combined with image M, and then an inter-image calculation between M and L5 may be performed. Note that L5 means the final image in the embodiment of FIG.

According to this embodiment, even if two images are separated in time and have many changes, the cumulative distortion vector can be determined, so the distortion can be removed and correct inter-image calculations can be performed.

Various modifications and applications of this embodiment are possible, and will be listed below. (1) Input to the interpolation calculation unit 34...Although the final cumulative distortion vector is used, the distortion vector A1 between adjacent images in FIG.
A2, . . . may be taken in. In this case, the cumulative distortion vectors ZM, Z1, Z2, . . . are stored in the memory 33. Furthermore, instead of A1, A2, . . . , partial cumulative values ZM, Z1, Z2, . . . may be imported. These differ depending on the processing purpose and processing mode.

(2) Adjacent images...They are not necessarily physically or image-wise adjacent, and any two-image interval or irregular interval can be adopted.

(3) Image distortion calculation: Although this calculated distortion is used for inter-image calculations, it can also be used solely for the purpose of removing temporal changes in any one image.

(4) Regarding temporal continuity...In calculating dynamic functional images, it is important to remove the influence of temporal changes, but in cases such as image communication, it is rather necessary to actively detect movement. Therefore, it is required to reduce the amount of encoded data. This embodiment can also be used for such image communication.

(5) Difference calculation: In addition to the difference, there may be various types of inter-image calculations such as addition, integration, logical processing, and mathematical processing.

(6) Distortion vector...The distortion vector is just one example, and generally the amount of deviation may be used. Also, although the classification unit is used, if there are few images, classification is not necessarily necessary.

(7) Types of images: In addition to medical images, various photographed images such as object recognition are also included.

[0044]

[Effects of the Invention] According to the present invention, it is possible to calculate and correct distortion even if there are images that are temporally distant from each other and have many changes in a large number of consecutive images that are obtained temporally continuously. Become. (Claims 1 and 3).

Furthermore, according to the present invention, correct inter-image calculation results are obtained by performing inter-image calculation using the correction results. (Claims 2 and 4). Furthermore, according to the present invention, it can also be used to calculate the motion of an image (claim 5).

[Brief explanation of the drawing]

FIG. 1 is an embodiment diagram of an inter-image calculation device of the present invention.

FIG. 2 is a diagram showing image cropping and a window function.

FIG. 3 is a characteristic diagram of spatial filtration.

FIG. 4 is a characteristic diagram of spatial filtration.

FIG. 5 is a diagram showing an example of distortion vector processing.

FIG. 6 is a diagram showing cumulative distortion vectors.

[Explanation of symbols]

1 Detection system 2 Pretreatment system 14 Spatial filter 16 Cross correlation part 19 Cumulative addition section

Claims (5)

[Claims] Claim 1: In temporally continuous images, the amount of deviation between adjacent images is calculated, and the amount of deviation between an image at a reference time and an image after an arbitrary time is determined by calculating the amount of deviation between the two images. An image distortion correction device that provides a cumulative value of the amount of deviation between adjacent images existing in the image data, and corrects the deviation of the image after the above-mentioned arbitrary time using the cumulative value. 2. An inter-image calculation device for performing an inter-image calculation between a corrected image obtained by the image distortion correction device according to claim 1 and a first image of temporally continuous images. Claim 3: A first image forming apparatus that mutually stores adjacent images after a reference time among temporally continuous images while updating them.
, a second memory, a means for cutting out adjacent images in the first and second memories within a prescribed division and calculating a cross-correlation for each corresponding division to obtain a distortion vector for each division; Image distortion comprising means for cumulatively adding vectors each time adjacent images are updated, and means for correcting a shift in the last image at the time of any update using the cumulatively added distortion vector obtained after any update. correction device. Claim 4: A first image processing system that mutually stores adjacent images after a reference time among temporally continuous images while updating them.
, a second memory, a third memory that stores the image at the reference time, and the adjacent images in the first and second memories are cut out within a prescribed division and cross-correlation is performed for each corresponding division. means for obtaining distortion vectors for each section; means for cumulatively adding the distortion vectors each time adjacent images are updated; An inter-image arithmetic device comprising: means for correcting the image shift of the third memory; and means for performing an inter-image arithmetic operation between the reference image in the third memory and the corrected image. 5. In images that are continuously given, the amount of deviation between adjacent images is calculated, and the amount of deviation between the reference image and the arbitrary image is calculated based on the amount of deviation between adjacent images that exists between the two images. An image motion calculation device that calculates a cumulative amount of shift between images.