RU2478337C2 - Method of determining heart contour on fluorogpaphy images - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: invention relates to medicine, in particular to cardiology, and can be used in clinical and experimental research for determining contour, dimensions and position of heart on fluorography images (FI). In accordance with the method registration of frontal and left side fluorography image and generation of heart computer model are carried out. Regions of heart are determined on frontal and left side fluorography images are determined. Decomposition of heart region image into constituent parts with its further reconstruction is performed. Heart boundaries, by which correction of computer model, are marked out. Decomposition of heart region image is carried out with application of decomposition into empiric modes. Reconstruction of heart region image is performed by summing up separate empiric modes. Marking out upper heart boundary on heart region image is performed by application of segmentation, texture analysis and method of active contours.

EFFECT: automation of the process of heart geometry determination, increase of accuracy of heart contour determination, increase of accuracy of cardio-vascular system state diagnostics.

45 dwg

Description

The invention relates to medicine, in particular to cardiology, and can be used in clinical and experimental studies as a way to determine the contour, size and position of the heart in fluorographic images (FOS).

Currently, the problem of automated processing of fluorographic images has developed in a separate direction, branched into many private tasks related to various aspects of their processing: brightness adjustment. contrast, removing interference, segmentation, the allocation of individual areas and contours of the chest organs (lungs and heart) in fluorographic images [1, 2].

The task of automated allocation of the heart contour to FOS is the most relevant among those listed for the modern healthcare system. Information obtained as a result of accurate selection of the heart contour will allow: to determine its linear dimensions and geometric shape, to reconstruct the volumetric model of the patient’s heart, to obtain new diagnostic information about the state of the heart and to identify pathological deviations together with the analysis of the electrocardiogram.

A known method for determining the main functional indicators of myogeodynamics of the heart [3], which consists in registering the frontal and left-side fluorographic images of the patient’s heart, determining the geometric parameters of the patient’s heart by applying, superimposing and non-linear scaling of the computer model of the heart.

From the analysis of the claims of the known method, it follows that the determination of the geometric parameters of the heart is carried out manually. A radiologist reviews and analyzes fluorographic images on a monitor screen, identifies pathologies and makes a conclusion, guided by a description template, and stores the information in a database.

The disadvantages of "manual" determination of the size and position of the heart by fluorographic images are:

1. The inability to automatically determine the geometry of the patient’s heart by fluorographic images.

2. Low throughput of manual processing of fluorographic images.

Closest to the proposed invention is a method for determining the opaque region on the chest x-ray [4], which consists in registering the front fluorographic image, generating a computer model of the heart, selecting the left and right borders of the heart based on the borders of the lungs, highlighting the lower border of the heart, regulation of the upper and lower borders of the computer model of the heart based on the selected borders of the heart and the conclusion of the contour of the heart.

As follows from the claims, in the known method for determining an opaque region on the chest x-ray, the boundaries of the lungs are already known. Then, a computer model of the heart (CCM) is generated, which is a flat elliptical shape that combines with the known boundaries of the lungs. In the process of combining, the CMC is deformed to fill the area between the lungs. Thus, the allocation of the left and right borders of the heart. Then, the lower border of the heart is extracted using the Hough transform [5]. Then, the upper and lower boundaries of the computer model of the heart are regulated by deforming it under the selected borders of the heart to obtain a heart contour and the output of the heart contour.

The figure 1 shows a diagram of a known method for determining the opaque region on the chest radiograph.

The figure 2 (a, b) shows the frontal x-ray photograph (a) and the location of the organs (b) on it. The numbers indicate: 1 - heart; 2 - the left lung; 3 - the right lung; 4 - trachea; 5 - thymus gland; 6 - border of the heart.

The figure 3 shows a computer model of the heart of a known method for determining the opaque region on the chest radiograph.

The figure 4 shows the result of the regulation of the computer model of the heart under the already known boundaries of the right and left lung.

Figure 5 shows the result of a threshold conversion of a fluorographic image using the Sobel operator.

Figure 6 (a, b) illustrates the Hough transform: the Hough space (p, u) (a) with “reference points” forming the lower border of the heart (b) is shown.

The figure 7 shows a x-ray photograph with highlighted contours of the heart and lungs. The numbers indicate: 7 - the region of the heart; 8 - region of the right lung; 9 - region of the left lung.

According to the authors of the alleged invention, in the known method for determining an opaque region on a chest x-ray, the term “chest x-ray” can be replaced without loss of essence by the term “fluorographic image” (FOS). A X-ray photograph is a chest x-ray taken by an X-ray examination of the chest organs [6].

According to figure 1, in the known method for determining the opaque region on the chest radiograph, the following actions are performed:

1. Registration of frontal FOS. For registration of FOS, specialized X-ray equipment is used, including a luminescent screen, an electronic camera, an optical system, a computer, a tablet, a light-protective casing, and an X-ray transparent screen [7]. FOS recorded in several standard projections [8]. In the known method, frontal FOS is used. Figure 2a shows the frontal FOS recorded with a sampling frequency of 1000 Hz, figure 2b shows the frontal FOS with the location of organs on it, where the numbers indicate: 1 - heart; 2 - the left lung; 3 - the right lung; 4 - trachea; 5 - thymus gland; 6 - border of the heart. FOS of the patient’s chest is stored digitally in computer memory.

2. Generation of CCM. At this stage, the computer model of the heart (CCM) is generated in the frontal projection. The CMC of the known method is a flat figure of elliptical shape, consisting of 26 points, the width of which is equal to the width of the average human heart 9 cm and corresponds to the average shape of the human heart (see figure 3).

3. The allocation of the left and right borders of the heart based on the boundaries of the lungs. At this stage, the left and right borders of the heart are selected based on the regulation of the CCM under the already known borders of the right and left lung (see figure 4). First, the CCM by positioning in the image in the lung area is combined with the known boundaries of the lungs. Then, in the process of regulating the CCM, it is deformed so that it fills the area between the lungs. The KMS deformation process can be represented as a change in the shape of a flexible closed area of a fixed area, which is subjected to force and, according to this effect, takes a form that limits it. Thus, the allocation of the left and right borders of the heart.

4. The selection of the lower border of the heart. At this stage, the selection of the lower border of the heart on FOS is carried out using the Hough transform [5]. The Hough transform allows you to define “reference points” on a binary image that form the contour of the object of interest - the contour of the lower border of the heart. To perform the Hough transform, it is necessary to perform threshold processing of the FOS using the Sobel operator to obtain a binary image. The threshold processing of FOS using the Sobel operator is carried out by calculating the average brightness of the image and the threshold for transmitting brightness values [1, 5, 9].

The average value of the brightness of the image is calculated by the formula

,

,

where m is the ordinate;

n is the abscissa;

Gm, Gn - 3 × 3 matrices calculated by the following formulas:

,

,

,

,

where X is the initial FOS;

^∗ - denotes a two-dimensional convolution operation.

Based on the calculated average value of the brightness of the image, a threshold for transmitting brightness values for the FOS is selected. The threshold value is selected manually. The threshold value for transmitting the brightness values of the image is selected so that it is possible to separate the object of interest (the region of the lower border of the heart) from other areas in the image. The result of the threshold processing of FOS using the Sobel operator is shown in figure 5.

According to the results of threshold processing of FOS, a binary image is obtained, the lower border of the heart on which is partially separated from other objects in the image, its breaks are traced. This is due to the fact that the area of the lower border of the heart is overlapped by the diaphragm. To increase the accuracy of highlighting the lower border of the heart, the Hough transform is used. The Hough transform defines the "reference points" that form the lower border of the heart. A “reference point" is a point that is located between the break points of the lower border of the heart, is connected with them by one equation of the line and belongs to this border [5].

The definition of "reference points" forming the lower border of the heart is as follows.

First, the binary image obtained as a result of the threshold transformation with coordinates (m, n) is transferred to the Hough space with coordinates (p, u), as shown in Figure 6a. Then, in the resulting space, a 5 × 5 grid is introduced, dividing the Hough space into equal cells, the cell size is one pixel (see figure 6a). Each cell in the Hough space corresponds to a point, with coordinates (p _i u _i ,). According to the Hough transform, the distance between the points by tears of the lower border of the heart should not exceed 5 pixels [5]. Therefore, a prerequisite for the transformation is to hit at least two points x1 and x2 in the grid, forming a boundary discontinuity (see figure 6a). Then, curves are constructed for each point in the Hough space (see figure 6b)

,

,

where u is the angle between the external normal of the line and the abscissa axis, u Є [0, 2π];

p is the distance from the line to the origin, p≥0;

m _i is the value of the ith cell point along the abscissa axis;

n _i is the value of the ith cell point along the ordinate axis.

According to the Hough transform, a “reference point” between break points of the lower border of the heart is a point that is associated with break points by a single equation of the curve. The line that connects the break points of the lower border of the heart is the break line of the lower border of the heart. Figure 6b shows the Hough space with three points: x1 and x2 - points belonging to breaks in the lower, border of the heart; x3 - "reference point" lying between the points of rupture of the lower border of the heart and belonging to this border. Line g is the break line of the lower border of the heart.

Thus, as a result of threshold processing of FOS by the Sobel operator and determination of “reference points” that make up the lower border of the heart on FOS using the Hough transform, the lower border of the heart on FOS is selected.

5. Regulation of the upper and lower boundaries of the computer model of the heart based on the selected borders of the heart. The regulation of the lower border of the heart is carried out as a result of the deformation of the CCM, fixed by the lateral borders of the heart, under the lower border of the heart found at the previous stage. Thus, in the known method of deformation, the lower boundary and the lateral boundaries of the CMC are subjected. The upper part of the CCM (what remains after deformation), “not attached” to any border of the heart in the picture, is considered the upper border of the patient’s heart (see figure 7). As a result, according to the authors of the known method, the determination of the contour of the heart.

6. Conclusion of the contour of the heart. The result of this stage is the conclusion of FOS, on which the contour of the heart is highlighted (see figure 7). The figure 7 shows the FOS with the organs of the chest highlighted on it, where the numbers indicate: 7 - heart; 8 - the left lung; 9 - the right lung. The contour of the heart determines the location of the heart in the chest.

Thus, from the description of the actions of the known method for determining the opaque region on the chest radiograph, it follows that the definition of the heart contour is carried out according to the already known boundaries of the right and left lungs by:

- highlighting the lower border of the heart on FOS using the Hough transform;

- CMS deformations under the borders of the right and left lungs;

- regulation of the lower and upper borders of the heart.

Moreover, the upper border of the patient’s heart is determined, so to speak, “by the residual principle”, i.e. what remains of the upper border of the CCM after deformation in the known method is the upper border of the patient’s heart, and it is not clear how it is consistent with the real upper border of the patient’s heart. The upper border of the CCM in the known method, in principle, cannot accurately describe the upper border of the heart, which is an important component of the heart region for setting a diagnostic conclusion. CCM in the known method in the upper border only connects the broken curve of the left and right borders of the patient’s heart. The CMR used in the known method consists of only 26 points, so the number of points subjected to deformation in the region of the upper border of the heart will be limited. A limited number of CCM points in the region of the upper border does not allow a smooth description of the obtained upper border of the heart (see figure 7).

According to the authors of the alleged invention, the use of a closed area as a CCM is not effective from the point of view of accurately determining the contour of the heart on FOS. The average heart shape, proposed in the known method as a CMS, allows you to determine only the location of the heart in the chest, but not to solve the problem of accurately highlighting the contour of the heart, in particular its upper border.

Thus, the known method for determining the opaque region on the chest x-ray [4] does not provide an accurate allocation of the heart contour to FOS.

A disadvantage of the known method for determining the opaque region on the chest x-ray is the impossibility of accurately highlighting the upper border of the heart in a fluorographic image, as a result, the impossibility of an unambiguous interpretation of the state of the cardiovascular system and the exact formulation of the diagnostic conclusion.

The invention is aimed at improving the accuracy of determining the contour of the heart in fluorographic images (FOS) by highlighting the upper border of the heart.

This is achieved by the fact that in the method for determining the contour of the heart in fluorographic images, which consists in registering the frontal fluorographic image, generating a computer model of the heart, extracting the left and right border of the heart based on the borders of the lungs, highlighting the lower border of the heart and deriving the heart contour, additionally carry out registration of the left-side fluorographic image, determining the region of the heart in the frontal and left-side fluorographic images, decomposing the image of the area the heart into its components using decomposition into empirical modes, restoring the image of the heart region by summing up individual empirical modes, highlighting the upper border of the heart in the image of the heart region using segmentation, texture analysis and the active contour method, adjusting the computer model of the heart according to the selected heart contours on the front and left-sided image of the region of the heart.

The essence of the proposed method lies in the fact that the selection of the heart contour on the FOS is carried out by registering the frontal and left-sided FOS, determining the region of the heart on the FOS, decomposing the image of the region of the heart into its constituent parts, restoring the image of the region of the heart by summing up individual empirical modes. On the reconstructed image of the region of the heart, the upper border is selected. Then carry out the adjustment of the computer model of the heart and the conclusion of the heart contour.

The main distinguishing feature of the proposed method for highlighting the contour of the heart in fluorographic images is the decomposition of the image of the heart region selected on the frontal and left-sided images into components using decomposition into empirical modes (DEM). The subsequent texture analysis of the “graininess” of the reconstructed image of the region of the heart allows us to separate the region of the heart from other organs in the image. This, according to the authors of the alleged invention, provides a more accurate selection of the heart contour on the FOS.

The figure 8 shows a diagram of the proposed method for highlighting the contour of the heart in fluorographic images.

The figure 9 (a, b) shows the left-side FOS (a) and the location of the organs (b) on it. The numbers indicate: 10 - heart; 11 - trachea; 12 - thymus gland; 13 - border of the heart; 14 - diaphragm of the lungs.

The figure 10 shows a computer model of the heart of the proposed method for highlighting the contour of the heart in fluorographic images.

The figure 11 (a, b, c) shows options for the position of the heart in the human chest: oblique (a), horizontal (b) and vertical (c).

The figure 12 (a, b) shows the image of the region of the heart, highlighted on the frontal (a) and left-sided (b) FOS.

The figure 13 shows a diagram of the decomposition algorithm of the image of the region of the heart into empirical modes.

Figure 14 (a, b, c, d, d) shows the result of decomposition of the frontal image of the heart region into its constituent parts - a set of empirical modes of image of the heart region (a, b, c, d) and the remainder - the global trend of the image of the heart region (d )

Figure 15 (a, b, c, d, d) shows the result of decomposition of the left-side image of the heart region into its constituent parts — a set of empirical modes of the image of the heart region (a, b, c, d) and the remainder — the global trend of the image of the heart region (d )

The figure 16 (a, b) shows the restored frontal (a) and left-side (b) image of the region of the heart, obtained by summing up individual empirical modes.

The figure 17 (a, b) shows the segmentation of the frontal (a) and left-sided (b) image of the region of the heart.

Figure 18 (a, b) shows the frontal (a) and left-side (b) image of the region of the heart with a separate upper part of it.

The figure 19 shows the algorithm for texture analysis of the image of the region of the heart.

The figure 20 (a, b) shows the frontal (a) and left-side (b) image of the region of the heart with the highlighted upper region of the heart.

Figure 21 (a, b) shows the distribution of an arbitrary point of the "active circuit" of the total energy near the region of the heart (a) and the distribution of an arbitrary point of the "active circuit" of the smoothing energy near the region of the heart (b).

The figure 22 (a, b) shows the frontal (a) and left-side (b) image of the region of the heart with a highlighted upper border of the heart.

The figure 23 shows the result of adjusting the computer model of the heart according to the analysis of images of the heart region.

The figure 24 (a, b) shows the frontal (a) and left-sided (b) FOS with highlighted contours of the heart.

A comparison of the algorithms shown in figures 1 and 8 shows that the proposed method for determining the contour of the heart in fluorographic images is based on a completely different approach to highlighting the contour of the heart, which eliminates this drawback of the known method [4].

The authors suggest instead of:

- the average model of the heart, selected as the CCM in the known method, use a three-dimensional model of the heart;

- determining the boundaries of the heart using the Hough transform, use the decomposition of the image of the region of the heart into FOS into its component parts and their restoration;

- determine the upper boundary of the heart region by deforming the middle model of the heart using texture analysis of the image of the restored areas of the heart FOS.

Consider the features of the proposed method for determining the contour of the heart on fluorographic images (see figure 8). The first stage "Registration of the frontal FOS" is similar to the stage of the known method [4].

The next stage is "Registration of the left-sided FOS." Registration of the left-sided FOS is carried out in order to obtain additional information about the size and position of the heart. In addition, the projection of the heart on the left-side FOS in conjunction with the projection of the heart on the frontal FOS will allow for the adjustment of the computer model of the heart (CMS) in order to obtain the patient's CMS (IMSC). Figure 9a shows the left-side FOS, figure 9b shows the left-side FOS with the location of organs on it, where the numbers indicate: 10 - heart; 11 - trachea; 12 - thymus gland; 13 - heart tenderloin; 14 - diaphragm of the lungs. The frontal and left-side FOS of the patient are stored digitally in the computer memory and are used for further processing and analysis.

The next stage "Generation of CCM" is similar to the stage of the known method [4], only as a CCM is used a volumetric model of the heart, close to the shape of the human heart (see figure 10). The three-dimensional model of the human heart is a tool for the visual study of the internal structure of the heart, modeling the processes taking place in the heart and for visualizing the results of this simulation. Volumetric model of the heart is a distinctive feature of the proposed method for determining the contour of the heart.

The next stage is “Determining the region of the heart in the frontal and left-side fluorographic images.” This stage is carried out on the registered frontal and left-sided FOS. The determination of the region of the heart in the recorded images is carried out in order to reduce the computational cost and time for the allocation of the heart contour. As part of this stage, it is proposed to analyze only that part of the image into which the region of the heart falls.

The determination of the region of the heart in the images is carried out according to the principle of selecting only that region into which the heart of any patient can fall. The position and size of the heart depends on gender, age, body weight and height, chest structure, working conditions and life of the patient. According to the position of the heart, three types are distinguished (see figure 11 (a, b, c)): oblique (a), horizontal (b) and vertical (c).

The definition of the region of the heart on FOS is as follows. First, the pictures are scaled to the smallest possible size format of 500 × 500 pixels, without distorting the details of the pictures. Scaling to this size is carried out in order to reduce the images to a single format, as the formats of images obtained on different radiographic systems may be different. According to [10], the approximate center of the middle human heart in the images will be located relative to the center of the image itself to the right of 1/20 of the image width for the frontal FOS and to the left of 1/20 of the image width for the left-side FOS. Regarding the centers of the heart on FOS, knowing the average size of a person’s heart and possible variants of its deviations, the area in the images is determined, in which the heart of any patient will surely fall. According to [10], the size and position of a person’s heart can change (increase or decrease in size) relative to their average size (length - 150 mm, width - 90 mm) and position on the front and left-side images on average 30 mm to the right, left, up and down. Thus, on the frontal and left-side FOS from the center of the human heart, 75 mm are laid to the right and left vertically (half the width of the middle human heart is 45 mm, plus 30 mm is the average deviation of the size of the heart), up and down horizontally 105 mm (half the length of the middle human heart 75 mm, plus 30 mm - the average deviation of the size of the heart).

In addition, it should be noted that the stage of "Determining the region of the heart on FOS" is automatic. Since all registered images arriving at the input are scaled to a single size, the region of the position of the heart of any patient will be located at the same coordinates for all FOS. Therefore, when scaling images to a format of 500 × 500 pixels, the region of the heart on the front FOS will be along the abscissa in the range from 200 to 400 pixels, along the abscissa in the range from 120 to 420; on the left-side FOS will be on the abscissa in the range from 120 to 320, on the abscissa in the range from 50 to 350 (see figure 12 (a, b)).

Thus, the result of this stage is the automatically selected region of the heart on the frontal (see figure 12a) and left-side FOS (see figure 12b).

The next step is “Decomposition of the image of the heart region into its constituent parts”. At this stage, the decomposition into empirical modes (DEM) of the image of the region of the heart on the frontal (see figure 2a) and left-sided (see figure 9a) FOS. Decomposition of the image of the heart region into empirical modes is an adaptive method of data analysis. The basis used for decomposition (a set of empirical modes) is constructed directly from the analyzed image. This allows one to take into account all local features of the image [11, 12].

Empirical modes are monocomponent image components modulated in amplitude and frequency for each row and column of the image, i.e. their amplitude and frequency change over time. Mods do not have a strict analytical description, but must satisfy two conditions [11, 12]:

- the total number of extrema and the number of zero crossings should differ by no more than one;

- the average value of two envelopes: the upper one, interpolating local maxima, and the lower one, interpolating local minima, should be approximately equal to zero.

The DEM algorithm diagram of the heart region is shown in figure 13 and includes the following steps.

1. Determination of local extrema and minima of the image of the region of the heart f _j (m _i n _i ):

- the value of the ith sample f _j (m _i n _i ) is a local maximum if the condition f _j (m _i-1 , n _i-1 ) <f _j (m _i , n _i ) ≥f _j (m _{i +1} , n _{i + 1} );

- the value of the ith sample f _j (m _i n _i ) is a local minimum if the condition f _j (m _i-1 , n _i-1 )> f _j (m _i , n _i ) ≤f _j (m _{i +1} , n _{i + 1} ),

where j is the decomposition step.

2. Determination of the upper e _j (m _i n _i ) and lower g _j (m _i n _i ) envelopes of the image of the heart region using cubic spline interpolation [11, 13] based on the found local extrema f _j (m _i , n _i )

,

,

,

,

where a _in , b _in , c _in , d _in are the coefficients for each value of the i-th reference of the upper envelopes of the noisy FOS;

and _n , b _n , c _n , d _n are the coefficients for each value of the i-th reference of the upper envelopes of the noisy FOS.

3. The calculation of the average value of the envelopes of the image of the region of the heart in accordance with the expression

,

,

where h _j (m _i n _i ) is the average value of the envelopes of the noisy FOS;

e _j (m _i , n _i ) and g _j (m _i , n _i ) the upper and lower envelopes of the noisy FOS, respectively.

4. The calculation of the remainder of the image of the heart region by the formula

,

,

where s _j (m _i , n _i ) is the remainder of FOS.

5. The calculation of the value of the stopping criterion. As a criterion for stopping decomposition, the value of the normalized quadratic difference, defined as [11, 12], is used

,

,

6. Checking the shutdown condition. At this stage, the value of the remainder of the image of the region of the heart is compared with the value of the normalized quadratic difference. If:

- SD> s _j (m _i , n _i ), then proceed to step 1;

- SD <s _j (m _i n _i ) and h _j (m _i n _i )> s _j (m _i n _i ), then proceed to the next step.

7. The conclusion of the empirical modes of the image of the region of the heart on the frontal and left-sided FOS. At this stage, the mode m _k (m _i , n _i ) and the remainder s _j (m _i , n _i ,) of the heart region are output, where k is the number. Figure 14 (a, b, c, d, d) and 15 (a, b, c, d, d) show the results of decomposition of the image of the heart region on the frontal and left-side FOS into its components - a set of empirical modes of image of the heart region (a , b, c, d) and the remainder is the global trend of the image of the region of the heart (e). The empirical modes shown in Figures 14 and 15 are obtained by applying the DEM method sequentially for each image, starting from the original image, respectively, in the frontal and left-side projection. From each image (see figure 2a and 9a) the first mode is obtained (see figure 14a and 15a), and the next one is obtained from it (see figure 14b and 15b) and so on. Each empirical mode shown in figures 14 and 15 was obtained by decomposing the previous one. The components of the images shown in Figures 14d and 15d are global trends of images and are not subject to further decomposition. The global trend is a monotonic function, which cannot be further decomposed into modes, according to the conditions of signal decomposition into empirical modes [11, 12].

The next stage is "Restoring the image of the heart region." At this stage, the image of the region of the heart is restored by summing the individual empirical modes m _k (m _i , n _i ). The purpose of the “Restoration of the image of the region of the heart” stage is to obtain an image with clear textural differences in the region of the heart relative to other regions in the image.

The criterion for evaluating the reconstructed image of the heart region is the value of the fractal dimension. The fractal dimension reflects the degree of “graininess” of the texture in the images (fine-grained and coarse-grained). The numerical value of the fractal dimension characterizes the degree to which the window is filled with elements in which the fractal system exists [14].

The calculation of the fractal dimension of the images is as follows. First, restored empirical modes are converted from shades of gray to black and white images. Then, a grid with a square cell 1 × 1 pixel in size was superimposed on the image and the number of cells into which only black sections fall was counted. Then, the dependence of the number of cells occupied by black or white pixels on the image size is determined, i.e. the fractal dimension of the image is determined by the following formula:

,

,

where D is the fractal dimension;

N is the minimum number of square cells that completely cover the fractal set, i.e. plot only black;

ε is the side length of the square grid cell of the image.

According to [14], the fractal dimension can be 2, i.e. coincide with the topological dimension of the image plane, and for an image in which there is only white color, the fractal dimension will be 1. An image having a gradation of black and white, the fractal dimension will be fractional and varies from 1 to 2. Thus, in According to expression (10), for each reconstructed image of the heart region, the value of the fractal dimension D will change. So, the value of the fractal dimension of the image of the region of the heart varies within 1 <D <2.

Thus, the values of the fractal dimension for the image of the region of the heart are calculated. The value of the fractal dimension of the image of the region of the heart in the region of the heart is different from the fractal dimension of other organs and tissues in the image and is in the range of 1.3 <D <1.6. So, in the interval of values of the fractal dimension 1 <D <1.3 and 1.6 <D <2, the restored images of the heart region fall, the texture of which is more enlarged or finer in the region of the heart itself. Such images are not used for further research. The value of the fractal dimension of the reconstructed image of the region of the heart, falling in the range 1.3 <D <1.6, is optimal.

The result of restoring the image of the region of the heart by summing up individual empirical modes is shown in figure 16 (a, b) for the frontal and left-sided image of the region of the heart.

The next stage “Highlighting the upper border of the heart” is carried out on the basis of segmentation of the image of the heart region, texture analysis of the upper region of the heart and analysis of its upper border by the active contour method.

First, segmentation of the image of the heart region in the images is performed by finding the center of the image (m _s , n _s ). Then, relative to the center of the image (m _s , n _s ), a central vertical line is drawn.

Then, a vertical line, from image center back down the up and down _^1/4 of the average length of the human heart, i.e. 30 mm up and down respectively in a vertical line. Find additional vertical points of the heart region in the pictures

,

,

the coordinate of the second upper and second lower points of the heart region

,

,

lying on the center vertical line. Through the additionally found upper vertical point of the heart region in the image, we draw a horizontal line A, which divides the image into two parts, the upper part of which contains the region with the upper border of the heart, and the lower part contains the region with the lower and lateral borders of the heart. The location of all points in the image is shown in figure 17 (a, b) in the frontal (a) and left-sided (b) image of the region of the heart.

Then, the upper part of the heart region is separated in the images along line A. A matrix is formed in size that matches the size of the selected region of the heart, in which only the upper part of the selected region of the heart is present, and the remaining part is empty, filled with zeros (see figure 18 (a, b )).

The following is a texture analysis of the upper region of the heart in the reconstructed frontal and left-sided image of the region of the heart. The purpose of texture analysis is to highlight a region of the heart. The criterion for the selection of the upper region of the heart is a homogeneous texture, which allows you to highlight the borders of the heart and separate them from other organs in the image. The texture analysis algorithm is shown in figure 19.

A texture analysis of the reconstructed region of the heart on FOS is performed using the sliding window method [15]. The essence of the method is that the area in the picture is selected, which is a square section that processes all sections of the image in a certain sequence, from left to right, from top to bottom. In this case, at each next step, the window is shifted by one element (pixel), i.e. a new window overlaps the previous one. The sliding window first shifts horizontally from the left edge of the image to the right. Then, when the right border is reached, the window is shifted 1 pixel vertically from the top to the bottom of the image and returns to the left edge of the image. Thus, line-by-line crawl of the image is carried out until the window reaches the lower right border of the image.

Progressive crawl of the image is necessary in order to determine those areas in the image that correspond only to the region of the heart. For this, it is necessary to select the optimal texture coefficients for the upper border of the heart: the size of the sliding window, the step of its displacement along the image, and the evaluation criterion for each window. It is proposed to use the same criterion as the evaluation criterion that was used at the stage of "Restoring the image of the heart region" - fractal dimension. The values of the texture coefficients of the heart region in the image of the heart region were selected as follows: the value of the fractal dimension 1.48, the size of the sliding window 12, the step of the sliding window 1. The values of the texture coefficients of the heart region in the image of the heart region were selected as follows: the value of the fractal dimension 1, 34, the size of the sliding window 8, the pitch of the sliding window 1. The result of this operation is shown in figure 20 (a, b), respectively.

On the found upper region of the heart, the allocation of its upper border is carried out. The upper boundary is selected by the active contour method [16, 17]. To select the upper border of the heart by the method of active contours, the algorithm “Region Based Active Contour Segmentation” is used [18].

By “active circuit” is meant a variable circuit [16, 17], which consists of M points in two-dimensional space

,

,

where ν _i = [m _i , n _i ], i = [1, ..., M].

Each point of the "active circuit" iteratively approaches the border of the region of the heart. For each point close to the peak ν _i , the value of the total energy of the "active circuit" E _{i is considered}

,

,

where E _int (ν _i ) is the energy component, depending on the shape of the active circuit,

E _ext (ν _i ) - energy component, depending on the properties of the image - the gradient;

a , b - weighting coefficients, providing the contribution of each of the energies;

E _i E _int , E _ext - square matrices.

The value in the center of each of the energy matrices corresponds to the energy at the point v _i (i-th vertex of the "active circuit"). The remaining values in the energy matrices correspond to the energy at each point surrounded by ν _i .

Each vertex ν _i can potentially go to any point ν ' _i corresponding to the minimum value of energy E _i . This process is depicted in figure 21a. If the energy function is configured correctly, the vertices of the "active circuit" V iteratively move and stop near the borders of the heart region.

The internal energy E _{int is} calculated by the following formula [16]:

where c is the weight coefficient;

E _con (ν _i ) is the square energy matrix of the smoothing contour;

E _bal (ν _i ) is the square energy matrix of the bursting circuit.

The internal energy E _int (ν _i ) consists of two components: energy “active circuit” E _con (ν _i ) (smoothing component) and bursting “active circuit” energy E _bal (ν _i ) [16, 17].

The smoothing energy of the closed "active circuit" E _con (ν _i )) can be calculated by the following expressions:

,

,

where p _jk (ν _i ) are the points (x, y) that correspond to the points in the image in the energy matrix;

n is the number of points in the active circuit.

This component of the "active contour" E _con tightens the contour and does not allow individual vertices to strongly separate from the rest. The minimum value in the smoothing energy matrix E _con (ν _i ) will correspond to the value p _jk (ν _i ), which is as close as possible to the ideal surface passing through two adjacent vertices of the contour ν _i-1 and ν _{i + 1} (see figure 21b).

Bursting energy (E _bal (ν _i )) causes the circuit to deform in one direction. The elements of the matrix of bursting energy E _bal (ν _i ) are calculated by the following formula:

,

,

where n _i is the unit normal vector to the contour V;

p _jk (ν _i ) - points (x, y) that correspond to points in the image in the energy matrix.

Vector n _i can be found by rotating the vector t _i 90 degrees

.

.

External energy E _ext (ν _i ) is calculated by the following formula [16]:

,

,

where b, m, g - weighting factors;

E _mag (ν _i ) is the square matrix of the image energy;

E _grad (ν _i ) is the square matrix of the gradient energy.

The external energy E _ext (ν _i ) also consists of two components: the image energy E _mag (ν _i ) and the gradient energy E _grad (ν _i ) [16, 17]. The image energy matrix E _mag (ν _i ) consists of image brightness values

,

,

where I (p _jk (ν _i )) is the brightness function at the point p _jk (ν _i ).

The gradient energy matrix E _grad (ν _i ) is calculated by the formula

,

,

where ν-I (p _jk (ν _i )) is the gradient function of the image (the first derivative with respect to brightness).

According to the algorithm [18], to improve the accuracy of highlighting the contour of the heart in the image of the region of the heart, it is necessary to select the optimal number of iterations for E _i .

Thus, the result of the work “Highlighting the upper border of the heart region” is the selected upper border of the heart (see figure 22) in the heart region in the frontal (a) and left-side (b) image of the heart region, based on segmentation of the heart region, texture analysis of the upper areas of the heart and analysis of its upper border by the method of active contours.

The next two stages, “Isolation of the left and right border of the heart based on the borders of the lungs” and “Isolation of the lower border of the heart” are similar to the steps of the known method for determining the opaque region on the chest radiograph. The determination of the lower, left, and right borders of the heart on FOS is carried out in the same way as in the “Highlighting the upper border of the heart” stage based on segmentation of the heart region, texture analysis of the upper heart region and analysis of its upper border by the active contour method.

Highlighting the lower border of the heart is carried out by segmenting the image along a vertical line B drawn through a point in the image (x _s , y ₁₁ ) (see figure 17 (a, b)), followed by texture analysis of the lower region of the heart and highlighting the lower border of the heart active contour method.

As a result of the separation of the upper and lower parts of the heart region along the vertical lines A and B in the image, only the central region with the left and right border of the heart remained, which is also divided along the central vertical line into two regions of the heart (see figure 17).

The selection of the left and right borders of the heart is based on the segmentation of the image along a horizontal line drawn through the center point in the image (x _s , y _s ) to lines A and B (see figure 17 (a, b)). As a result, the remaining part of the heart region is divided into two parts, in each of which the border of the heart is highlighted by texture analysis and the active contour method.

A distinctive feature of this stage from the stage “Highlighting the upper border of the image of the heart region” are the values of texture coefficients. The values of texture coefficients in the frontal image of the heart region for the lower border of the heart region are equal: the value of the fractal dimension 1, 37; size of the sliding window 12; step of the sliding window 1. The values of texture coefficients in the frontal image of the heart region for the lateral borders of the heart region are equal: the value of the fractal dimension 1, 34; the size of the sliding window is 8; sliding window step 1.

Thus, according to the previous steps, areas of the heart in the images were separately identified.

The final stages of the proposed method for isolating the heart contour in fluorographic images are “Correction of the CCM according to the selected contours of the heart in the frontal and left-sided image of the heart region” and “Output of the heart contour”.

After separating the region of the heart from other organs in the image and highlighting its borders, we proceed to the stage “Correction of the CMR according to the selected contours of the heart in the frontal and left-sided image of the region of the heart”. To do this, use the "Method for automatically determining the size and position of the heart by fluorographic images" [19], which includes first forming a projection of a three-dimensional CMR on a plane, corresponding to the projection of the heart on the frontal and left-side FOS. Then an array of contours of the CCM projections is obtained, obtained by successive turns of the model with a given angle step along the three coordinate axes. For a sufficiently accurate determination of the position of the patient's heart, the maximum deviation of the rotation angle of the model is proposed to be taken no more than 5 ° degrees.

Then, the image of the projection of the heart model is superimposed on the image of the heart contour highlighted on the FOS, the geometric center of the contour image of the heart is determined, the geometric center of the projection of the heart model is combined with the geometric center of the heart contour image using the shift operation along the coordinate axes. The direct and left lateral FOS of the patient are simultaneously compared with the contours of the corresponding projections of the model. Depending on the ratio of the areas of the image of the heart on the FOS and the projection of the heart model, the scaling factors of the heart model along each of the coordinate axes are calculated [19]

,

,

,

,

,

,

where Kx, Ku, Kz are the scaling factors along the X, Y, Z axes, respectively;

SF1 is the area of the heart in the frontal image of the region of the heart;

SF2 is the frontal area of the heart model;

SL1 is the area of the heart in the left-side image of the region of the heart;

SL2 is the area of the left-side projection of the heart model.

After determining the angles of rotation of the patient’s heart, nonlinear scaling of the images of the contours of the heart model is performed so that they more closely correspond to the images of the heart contours on the patient’s FOS. Scaling of the heart model is carried out using affinity transformations. An affine transformation is a linear transformation followed by a shift transformation; it can be represented as follows:

,

,

where T is an affine transformation;

x is the coordinate vector of a point in space;

О - matrix of linear transformation of coordinates;

q is the shift vector.

The same transformation is often written in matrix form

where T is an affine transformation;

x, y, z - coordinates of a point in space;

and 1 .. and 3 are the coefficients of the linear transformation of coordinates;

b1..b3 - shift coefficients.

The obtained scaling factors Kx, Ku, Kz are used when adjusting the three-dimensional IMSC. This method allows you to adjust the CCM on the selected contours of the heart, as shown in figure 23.

Performing the stage "Correction of the CCM according to the selected contours of the heart in the frontal and left-sided image of the heart region" allows you to adjust the boundaries of the heart region in the frontal and left-side FOS.

Thus, the result of the proposed method for highlighting the contour of the heart in fluorographic images is the selected contour of the heart on the frontal and left-sided FOS (see figure 24).

The above description of the method for determining the contour of the heart on fluorographic images, according to the authors of the alleged invention, shows that the proposed method for determining the contour of the heart on fluorographic images eliminates the disadvantage of the known method for determining the opaque region on the chest radiograph, namely, to increase the accuracy of determining the contour of the heart, for by highlighting its upper border in the frontal and left-side fluorographic images, and to improve the accuracy of diagnosis of conditions of cardio-vascular system.

Moreover, in the proposed method for determining the contour of the heart on fluorographic images, new properties are manifested that allow you to automate the process of accurately determining the contour of the heart by FOS and then synthesize a computer model of the patient’s heart.

Information sources

1. Gonzalez R., Woods R., Eddins S. Digital image processing in Matlab. Moscow: Technosphere, 2006.616 p.

2. Rangayyan P.M. Analysis of biomedical signals. Practical approach / Per. from English under the editorship of A.P. Nemirko. M .: FIZMATLIT, 2007.414 s.

3. A method for determining the main indicators of myohemodynamics of the heart / Bodin O.N. / Burukina I.P. / Mitin A.A. / Ogonkov V.V. / Mitroshin A.N. / Bondarenko L.A. / Rudakova L.E. // RF patent №2264786, IPC АВВ 5/0402, 6/00.

4. US Patent No. 20080181481. Method and system for locating opaque regions in chest X-ray radiographs / Ling Hong / Ying Li / Hong Shen // 07/31/2008.

5. Degtyareva AE, Hough transform (Hough transform). Computer Graphics and Multimedia, No. 1 (2). 2003 URL: http://cgm.computergraphics.ru/content/view/36 .

6. Kochura V.I., Laptev V.Ya. Possibilities of digital fluorography for lung examination URL: http://www.medafarm.ru.

7. Apparatus for obtaining computer x-ray images and a method for obtaining such images / Arapov N.A. / Kornev A.N. / Kulakov V.I. / Nikonov I.A. / Santalov B.F. / Ustinin M.N. / Fokin V.A. / Yashin V.A. // RF patent №2134450, IPC ⁶ G06T 1/00, G03B 42/02, H05G 1/00, АВВ 6/00, G06F 19/00, G06F 159: 00.

8. Roitberg G.E., Strutinsky A.V. Laboratory and instrumental diagnosis of diseases of internal organs. M .: Medicine. 2003.89 p.

9. Jane B. Digital image processing. Moscow: Technosphere, 2007.584 s.

10. Tsvetkov V.D. The heart, the golden ratio and symmetry. Pushchino: PSC RAS, 1997.170 p.

11. Huang N.E., Shen S.S. The Hilbert-Huang transform and its applications - World Scientific Publication, 2005. pp.323.

12. Huang, N.E., Attoh-Okine Nii O. The Hilbert-Huang transform in engineering -Taylor and Francis, 2005. pp. 95.

13. Korneichuk N.P. Splines in approximation theory - M .: Nauka, Main Edition of the Physics and Mathematics Literature, 1984. P.352.

14. Fractal analysis of processes, structures and signals. Collective Monograph / Ed. R.E. Pashchenko // Kharkov. NEO EcoPerspective. 2006.348 s.

15. Mitsel A.A., Kolodnikova N.V., Protasov K.T. Nonparametric algorithm for texture analysis of aerospace images. TPU News, No. 1, 2005. S.65-70.

16. Kaas M., Witkin A., Terzopoulos D. Snakes: Active Contour Models. Int. Journal of Computer Vision. 1987. p. 312-331.

17. Petrov V.O., Privalov O.O. Modification of the active contour algorithm to solve the problem of interactive segmentation of raster images. Modern problems of science and education. 2008. S.14-19.

18. Active Contour Matlab Code Demo. Regionbased_seg_demo.zip.2008. URL: http: // www.shawnlankton.com/tag/computer-vision.

19. A method for automatically determining the size and position of the heart by fluorographic images / Bodin ON / Kuzmin A.V. / Semenkin M.A. / Moiseev A.E. // RF patent №2372844, IPC АВВ 6/00/107.

Claims (1)

The method for determining the heart contour in fluorographic images, which consists in registering the frontal fluorographic image, generating a computer model of the heart, extracting the left and right border of the heart based on the borders of the lungs, highlighting the lower border of the heart and deriving the heart contour, characterized in that they register left-side fluorographic image, determining the area of the heart in the frontal and left-side fluorographic images, decomposing the image of the heart area into composite parts Partitions using decomposition into empirical modes, restoring the image of the heart region by summing up individual empirical modes, highlighting the upper border of the heart in the image of the heart region using segmentation, texture analysis and the active contouring method, adjusting the computer model of the heart according to the selected heart contours in the frontal and left-sided image areas of the heart.

Images