RU2614545C1 - Device for merging medical images - Google Patents

Abstract

FIELD: physics, computer engineering.

SUBSTANCE: invention relates to information measuring devices and can be used in computer engineering, in control and signal processing systems. The device comprises: a register for storing input realisation of a first image, a register for storing input realisation of a second image, a clock generator, a unit for restoring boundaries of the first image, a unit for filtering the first image, a unit for filtering the second image, a unit for restoring boundaries of the second image, a unit for searching for control points on the first image and conformities, a unit for searching for control points on the second image and conformities, a unit for transforming the matrix of the first image, a unit for transforming the matrix of the second image, a unit for selecting contours on the images, a unit for finding image merging boundaries, a unit for excluding non-overlapping regions on the images, a unit for superimposing the gradient of the first image, a unit for superimposing the gradient of the second image, a unit for merging thee images, a register for storing the output realisation.

EFFECT: invention is characterised by providing a merged image with smoothed transition boundaries.

2 dwg

Description

The present invention relates to computer technology and can be used to form medical images obtained in radiography, tomography, fluorography and other medical research. The technical result is a combined image with smooth transition borders. The obtained panoramic alignment result is achieved under conditions of inaccurate superposition of control points under uniform illumination. To do this, in the method, the contours on overlapping images are automatically highlighted, the alignment curve is found, a mask for each image is created from the found contours, a combined image is created using masks. In FIG. 1 shows an example of combining images.

A simplified mathematical model of the result of combining images in is represented as:

- первое изображение с областью объединения;Where

- The first image with the union area;

- второе изображение с областью объединения;

- the second image with the area of the union;

- первое изображение;

- first image;

- второе изображение;

- second image;

- область объединения.

- area of association.

и

в формулу (1) и введя параметр объединения, получаем следующий вид математической модели:Substituting expressions for

and

into formula (1) and entering the union parameter, we obtain the following form of the mathematical model:

where α is the mixing coefficient of the regions.

The main problem to be solved is obtaining a combined image with smoothed transition boundaries.

A similar problem may arise: 1) in radiography; 2) in tomography; 3) in genetics; 4) in fluorography.

In those cases when the images have a large number of textures of small details, most often in practice, methods for combining images using angle detection detectors (SURF, SIFT, RANSAC) are used. As a result of a simple combination of such methods, additional boundaries of objects appear (double contours), the appearance of a border due to uneven illumination, or blurring.

The solution to the problem of the appearance of double contours is achieved by using boundary detectors to remove unnecessary areas. Using blending by applying masks to the images to be merged eliminates the unevenness of illumination and blurred images to be merged in the overlapping area.

A known method of automatic sequential planning of MP-scan [Bergmans Cornelis J.F. M (NL), Bistrov Daniel (NL), Hes Harald S. (NL), Cauvenhoven Mark (NL), Den Harder Johan M. (NL), Simonetti Aryan W. (NL), De Kock Wendy (NL). IPC class: G06T 11/20, АВВ 6/00, patent No. 2533626, 11/20/2014]. The invention relates to the field of automatic acquisition of clinical MRI images.

The method comprises: obtaining a first inspection image with a first observation field, the first inspection image has a first spatial resolution, determining the location of the first region of interest, and at least one anatomical landmark in the first examination image, the step comprising: creating a three-dimensional volume, determining a set of contours in three-dimensional volume, identification of one or more anatomical landmarks in three-dimensional volume, automatic segmentation of three-dimensional volume ; determining the position and orientation of the first region of interest using the anatomical landmark, the position and orientation of the first region is used to plan the second examination image; obtaining a second inspection image with a second observation field, the second inspection image has a second spatial resolution, the second spatial resolution is higher than the first spatial resolution; creating geometry planning for the anatomical region of interest using a second examination image; and obtaining a diagnostic image of the anatomical region of interest using geometry planning.

The features of the analog device, which coincide with the features of the claimed technical solution, are as follows: search for circuits, geometry planning.

The disadvantages of the known device are:

- the inability to work in the absence of established guidelines;

- high computing costs.

The reasons that impede the achievement of the required technical result are as follows:

- if the calculation of the image transformation matrices is incorrect, a doubling of the boundaries of the objects in the image will appear.

A known method of composite image formation [Tolstaya Ekaterina Vitalievna (RU). IPC class: G06T 7/00, G06K 9/44, patent No. 2421814, 06/20/2011], which forms a composite image from several partially overlapping images captured by such tablet devices as a scanner or multifunction peripheral device. The technical result is to increase the speed of the process of combining images. The specified technical result is achieved in that: analyze the composite image elements and identify descriptors of compatible points; perform pairwise combination of descriptors; combine descriptors with the final image and restore rotation / displacement parameters; alternately include the constituent elements in the final image based on the restored rotation / displacement parameters.

The features of the analog device, which coincide with the features of the claimed technical solution, are as follows: when combining images into a composite image on overlapping sections of the current combined image and the composite image, the optimal border passing through pixels located on overlapping sections with minimal differences is determined.

The disadvantages of the known device are:

- the appearance of a border between images as a result of combining;

- there is no preliminary processing, which greatly affects the error in determining the base points in the image;

- the appearance of double contours of the merged images.

The reasons that impede the achievement of the required technical result are as follows:

- if the images have different contrasts, then the border will be visible;

- the presence of a noise component in the images complicates the process of searching for base points and in some cases increases the number of false matches;

- if the calculation of the image transformation matrices is incorrect, a doubling of the boundaries of the objects in the image will appear.

A known method of suppressing uneven distribution of brightness of images when obtaining panoramic images of medical micropreparations [Nikitaev Valentin Grigoryevich (RU), Pronichev Alexander Nikolaevich (RU), Komarov Vladimir Vladimirovich (RU), Chistov Kirill Sergeevich (RU), Filippov Anton Aleksandrovich (RU). IPC class: G06K 9/36, patent No. 2390842, 05.27.2010]. The technical result of this invention is to increase the efficiency of determining threshold brightness values of pixels and correlation of values to predetermined limits.

The method of smoothing the uneven distribution of brightness of images when obtaining panoramic images of medical micropreparations consists in that from an external device, an image of unevenly illuminated glass of the microscope stage on the background image without micropreparation is obtained; determine the unevenness of the brightness values of the image of glass without a micropreparation (divide the image into component sections, fix the average brightness of each section, record the absolute and relative difference in brightness indicators among the sections); then the image of the micropreparation is captured from the camera; divide the resulting image into component sections, fix the average brightness of each section, fix the absolute and relative difference of the brightness indicators among the sections, for each image section of the micropreparation under study, the brightness unevenness values found earlier are directly subtracted and the edges of the image under study are smoothed out.

The features of the analog device, which coincides with the features of the claimed technical solution, are as follows: smoothing the uneven distribution of brightness of images when obtaining panoramic images.

The disadvantages of the known device are:

- in the case of receiving a pair of frames when shifting the camera fixation increases the error in the superposition of pairs of images;

- the appearance of double contours of the merged images;

The reasons that impede the achievement of the required technical result are as follows:

as a result of the displacement of the fixing camera, additional operations are required performed on the received frames to compensate for distortions;

- if the calculation of the image transformation matrices is incorrect, a doubling of the boundaries of the objects in the image will appear.

Closest to the invention is a device for combining images into a single composition with a smooth transition of contrast (patent No. 2571574, IPC G06T 5/00, 12/20/2015).

The prototype device under consideration comprises an input implementation storage register 1, an input implementation storage register 2, a clock 3, a control point search unit 4, a control point search unit 5, a best point search unit 6, a transformation matrix calculation unit 7, a first image matrix transformation unit 8, the matrix transformation unit of the second image 9, the zero filling unit 10, the contour highlighting unit 11, the curve calculating unit 12, the mask locating unit 13, the mask locating unit 14, the combining area allocation unit Images 15, 16 seam block finding unit removing unwanted areas 17, a mask superposition unit to the first image 18, the block masking the second image 19, the image combining unit 20, storage register 21 output implementation.

A device for combining images into a single composition with a smooth transition of contrast consists in the fact that the values of the input implementation go to the inputs of the device and are recorded in the storage register of the input implementation. In the blocks of search for control points, there is a correspondence between the input data using the angle detection detector, the best matches are found, and the transformation matrices are calculated. Then the matrices are transformed and filled with zeros in the missing fragments and fed into the blocks to find the mask. The joining area and the seam are found, the contours are selected along which the curve is determined in order to remove unnecessary fragments relative to the curve. A mask is applied to each image and merge. The result is written into the storage register of the output implementation and is output to the device.

The disadvantages of the known prototype device are:

- the inability to implement the known method, if the image is low resolution;

- the appearance of a noise component in the images increases the error in determining the base points and determining their correspondence on a pair of images;

- high computing costs.

The reasons that impede the achievement of the required technical result are as follows:

- to use the method it is necessary to remember the entire input implementation.

The medical imaging device is a two channel device, which is shown in FIG. 2 and comprises a first input and a second input, which are information inputs of the device, which are connected to the input of the storage register of the input implementation of the first image 1 and to the input of the storage register of the input implementation of the second image 2, respectively; the output of the storage register of the input implementation of the first image 1 is connected to the input of the filtering unit of the first image 5, the output of which is connected to the input of the border recovery unit of the first image 4, the output of which is connected to the third input of the image combining unit 17, to the third input of the gradient overlay unit of the first image 15, the second input of the matrix transformation unit of the first image 10 and to the first input of the checkpoint search block on the first image and matches 8, the output of which is connected to the second input at the checkpoint search block in the second image and matches 9 and to the first input of the matrix transform block of the first image 10, the output of which is connected to the second input of the gradient superposition block of the first image 15 and to the first input of the contouring block in images 12, the output of which is connected to the input a block for finding the boundary of the union of images 13, the output of which is connected to the input of the block for excluding non-overlapping areas in images 14, the output of which is connected to the first input of the block overlay gradient second image 16 and to the first input of the first gradient image overlay unit 15, whose output is connected to the first input image combination unit 17, whose output is connected to an output storage register 18 implementation, the output of which is a data output device; the output of the storage register of the input implementation of the second image 2 is connected to the input of the filtering unit of the second image 6, the output of which is connected to the input of the border recovery unit of the second image 7, the output of which is connected to the fourth input of the image combining unit 17, to the third input of the gradient overlay unit of the second image 16, to the second input of the block transform matrix of the second image 11 and to the first input of the block search for control points in the second image and matches 9, the output of which is connected to the second the course of the checkpoint search block in the first image and matches 8 and to the first input of the matrix transform block of the second image 11, the output of which is connected to the second input of the contour highlighting block in images 12 and to the second input of the gradient overlay block of the second image 16, the output of which is connected to the second the input of the block combining images 17; the synchronization of the device is set by the clock 3.

A device for combining medical images is implemented as follows. The values of a pair of input images having a region of overlapping a single scene are fed to the inputs of the device and recorded in the storage register of the input implementation of the first and second image. At the next step, preliminary image processing is performed, at which high-frequency interference is filtered, for example, by an image processing device based on the two-dimensional method of multiplying estimates (RF patent No. 2406130 C1, IPC G06F 17/17, applicant and patent holder of GOU VPO "YURGUES". No. 2009121964/08; claimed June 8, 2009; published December 10, 2010, Bull. No. 34). A preliminary decrease in the action of the noise component increases the accuracy of the subsequent processing associated with the search for matches in the images. At the next stage, the boundaries of objects blurred at the previous stage or obtained as a result of fixation, for example, by an X-ray machine, are restored. Restoring the boundaries of objects can be accomplished by a method of restoring boundaries, for example, restoring clarity based on the analysis of several images (Agrawal A., Xu Y., Raskar R. Multi-image deblurring: US patent 8229244 - 2012.). At the next step, a pair of images is used to search for control points with high local information content and their correspondences on a pair of studied images, for example, by the SURF method (Bay N., Tuytelaars T., Van Gool L. SURF: Speeded up robust features // Computer vision-ECCV 2006. - Springer Berlin Heidelberg, 2006.- C. 404-417.). At the next step in analyzing the position of the base points, projective transformations are performed on the images. The coefficients change until a partial (by a value of no more than 2 pixels) or full combination of 5 points in the overlap area occurs. The next step is to search for the contours of objects in the images. This operation can be performed by any boundary detector, for example, by the Canny method (Canny. J., A computational approach to edge detection / J. Canny. // IEEE Transaction on Pattern Analysis and Machine Intelligence, 8 (6): 679-698, 1986.) or a device for isolating the contours of objects on a textured background when processing digital images (RF patent No. 2522044 C1, IPC G06K 9/48, G06T 7/60, applicant and patentee GOU VPO "YURGUES". No. 20130101106/08; filed 09.01. 2013; published July 10, 2014, Bull. No. 19). The next step is to determine the boundaries of the alignment of a pair of images. This operation is based on the search for a curve located close to the center of the overlap area and separating the objects located on the frames passing along their borders. Next, the regions to the left of the overlap region and the scene fixing region are excluded for the first image, as well as to the right of the overlapping region and the scene fixing region for the second image, the exclusion boundary is set according to the curve defined in the previous step. The brightness and contrast analysis is used to determine the color transition gradients for each image, this operation can be carried out, for example, software implementations of the algorithm for combining images "Alpha-mixing" (Certificate of state registration of computer programs: ROSPATENT, 2013 - No. 20133611413) . At the final stage, combining a pair of medical images into a single content, combining a pair of modified input images and recording them in the storage register of the output implementation for further transfer to the output of the device.

A device for combining medical images works as follows. The values of a pair of input images having an overlapping area of a single scene are sent to the device inputs and are recorded in the storage register of the input implementation of the first image 1 and the storage register of the input implementation of the second image 2. In the next step, preliminary processing of the images is performed, in which the high-frequency noise is filtered in parallel with the filtering unit of the first image 5 and the filtering unit of the second image 6. The next step is to restore the boundaries of the objects smeared at the previous stage or obtained as a result of fixing by the unit for restoring the borders of the first image 4 and the unit for restoring the borders of the second image 7, for example, an X-ray apparatus. At the next step, on a pair of images, search for control points with high local information content and their correspondence between images, a search unit for control points in the first image and matches 8 and a search unit for control points in the second image and matches 9. The next step is to analyze the position of the base points projective transformations are performed on the images by the matrix transform unit of the first image 10 and the matrix transform unit of the second image 11. Coefficient We change until a partial (by a maximum of 2 pixels) or full alignment of 5 points in the overlap area occurs. At the next step, the contours of objects in the images are searched by the contour extraction unit in the images 12. This operation can be performed using any border detector. At the next stage, the boundary for combining a pair of images is determined by the block for finding the boundary for combining images 13. This operation is based on the search for a curve located close to the center of the overlap region and separating objects passing along their borders. Next, the regions to the left of the overlapping region and the scene fixing region are excluded for the first image, as well as to the right of the overlapping region and the scene fixing region for the second image by the block of excluding non-overlapping regions in images 14, the exclusion boundary is set according to the curve defined in the previous step. By analyzing the brightness and contrast, the gradients of the color transition for each image are determined by the gradient superposition unit of the first image 15 and the gradient superposition unit of the second image 16, this operation can be carried out by any image combining algorithm. At the final stage, a pair of medical images is combined into a single content, by an image combining unit 17, a pair of modified input images is combined and stored in the storage register of the output implementation 18 for further transmission to the output of the device.

Claims (1)

A device for combining medical images containing a storage register of the input implementation of the first image and a storage register of the input implementation of the second image, the inputs of which are the information inputs of the device, a storage register of the output implementation, the output of which is the information output of the device, characterized in that the output of the storage register is the input implementation of the first image connected to the input of the filtering unit of the first image, the output of which is connected to the input of the border recovery unit p the first image, the output of which is connected to the third input of the image combining unit, to the third input of the gradient overlay unit of the first image, the second input of the matrix transformation unit of the first image, and the first input of the control point search unit in the first image, the output of which is connected to the second input of the control search unit points on the second image and to the first input of the matrix transform block of the first image, the output of which is connected to the second input of the gradient overlay block of the first image and to the first input of the block for selecting contours in images, the output of which is connected to the input of the block for finding the boundary of the union of images, the output of which is connected to the input of the block for excluding non-overlapping areas in the images, the output of which is connected to the first input of the block for applying the gradient of the second image and to the first input of the block applying a gradient of the first image, the output of which is connected to the first input of the image combining unit, the output of which is connected to the storage register of the output implementation, the output otorrhea is a data output device; the output of the storage register of the input implementation of the second image is connected to the input of the filtering unit of the second image, the output of which is connected to the input of the border recovery unit of the second image, the output of which is connected to the fourth input of the image combining unit, to the third input of the gradient overlay unit of the second image, to the second input of the conversion unit matrices of the second image and to the first input of the checkpoint search unit in the second image, the output of which is connected to the second input of the control search unit lnyh points in the first image and the first input of the transformation matrix of the second image, whose output is connected to the second input of the edge enhancement in the images and to a second input of the superposition of the second gradient image, which output is connected to the second input image combining unit; the synchronization of the device is set by the clock.

Images