FR2800491A1 - METHOD FOR PROCESSING IMAGES IN THE PRESENCE OF STRUCTURED NOISE AND NON-STRUCTURED NOISE - Google Patents

Abstract

The invention relates to a method for processing a 2D or 3D image representative of an object whose contours are disturbed by structured noise and unstructured noise, characterized in that it consists in extracting the image, the active contour of the object by the following steps: a) analysis of unstructured noise, that is to say background noise, b) structured noise analysis, that is to say detection of multiple reflections and the signature of the objectc) recognition of the edges of the object from the image obtained in b); andd) recognizing the surface of the object from the raw image (or initial image) and the images obtained in steps b) and c).

Description

IMAGE PROCESSING METHOD

IN THE PRESENCE OF NOISE STRUCTURE

AND NO STRUCTURE NOISE

DESCRIPTION

  Field of the Invention The invention relates to a method for processing representative images of objects whose contours are disturbed by structured noise and unstructured noise. This method consists in extracting and controlling the surface information of objects immersed in an opaque liquid medium from images

  two-dimensional or three-dimensional.

  The invention finds applications in the fields of ultrasound imaging and electromagnetic imaging for analyzing 2D or 3D images and providing representations of the image object, which do not require any particular knowledge and

  no interpretation on the part of the user.

  For example, the invention can be applied in the medical field, for the diagnosis, by ultrasound, of a disease or

  malformation or for medical supervision.

  The invention can also be applied in the nuclear field, for the inspection of a fast neutron reactor vessel in service, in order to visualize the submerged metal structures, to control the mechanical manipulations and the research

  of objects lost in the reactor vessel.

  STATE OF THE ART In the field of ultrasound imaging, there are currently numerous devices for viewing or taking images of an immersed object.

in an opaque liquid.

  One of these devices is described in particular in the French patent application No. FR-A-2,755,242, filed in the name of the applicant. This device makes it possible to form digital volumes containing the surface information relating to the immersed object, after acquisition of ultrasound echoes by an orthogonal multi-element focusing device.

the show and at the reception.

  The image acquired by such a device must then be interpreted by the user for

diagnose the state of the object.

  For example, in the case of a fetus visualized by ultrasound, the sonographer must know how to interpret the ultrasound images to detect a malformation of the fetus or any other anomaly. An error in the interpretation of the user may, in some cases, have

dramatic consequences.

  It would therefore be interesting to be able to offer the user representations of the immersed object that does not require any particular knowledge of the imaging system. However, there is currently no method for extracting the ultrasonic image object surface information to provide a reliable representation of the object without requiring interpretation from the user. There are, of course, methods for analyzing unstructured noise, i.e., background noise, on an ultrasound or electromagnetic image. These methods generally consist of a filtering, linear or not, to eliminate the high frequency noise and to extract the contours of the structures in the image. They can also consist of a thresholding to eliminate the continuous components. There are, moreover, known methods for analyzing structured noise, that is to say artifacts, images. One of these methods consists of a mathematical morphology, as described in G. MATHERON, entitled "Random sets and integral geometry", WILEY, New York, 1975, or in the book entitled "Image analysis and In a few words, this method consists in using a simple form, called a structuring element, which is displaced throughout the structure of the image, in order to extract

shapes, 2D or 3D, interpretable.

  Another method of structured noise analysis is the HOUGH transform, which is described in particular by H. MAITRE in "a panorama of the transformation of HOUGH", signal processing, 2, n 4, 1984, or in "A This paper method consists in transforming a pattern recognition problem into a research problem, from A. IANNINO and SD SHAPIRO, Pattern Recognition and Image Processing, pages 32 - 38, 1978. maxima. None of these methods can both analyze structured and unstructured noise and recognize relevant information.

  relating to the structure of the object being imaged.

  In addition, the methods described above are suitable for processing grayscale quantified images. They are not suitable for

  process images of non-digitized intensity.

  SUMMARY OF THE INVENTION The purpose of the invention is precisely to propose a method for processing representative intensity images of an object whose contours are disturbed by structured noise and nonstructured noise. We recall, for all intents and purposes, that structured and unstructured noises are

  consisting, in particular, of different types of echoes.

  In an ultrasonic or electromagnetic image, we can find the following echoes: - specular echo: for a transceiver pair, the specular point on a given surface is the one that defines the shortest path between the transmitter and the receiver passing through this point of the surface (FERMAT principle). This point is actually associated with a characteristic spot of the imaging system (opening of the antennas) and the geometry of the object's surface at the theoretical specular point, creating a cloud of points in the volume; - diffracted echo: when the object has sharp edges, a diffraction phenomenon occurs. The reflected wave propagates in privileged directions, thus contributing locally to highlights; - echo backscattered: this echo only exists for rough structures, whose roughness scale is close to that of the wavelength; - ghost echo (artifacts): its presence is related to sufficiently close faces (relative to the acquisition window) to introduce false surfaces into the image by multiple reflections

between these faces.

  These four types of echoes are likely to end up in the digital volume formed after the acquisition of the image. Some of these echoes are semantically and visually rich in information: it is they that allow the human brain to reconstruct the scene taken in image. This is, in

  in particular, diffracted and backscattered echoes.

  The image processing proposed by the invention is precisely based on the identification of

these different types of echoes.

  The method of the invention seeks to eliminate background noise while retaining backscattered echoes, to eliminate ghost echoes (ie structured noise) while retaining relevant information from backscattered and diffracted echoes and to properly dissociate the echoes

  specular diffracted echoes.

  More specifically, the invention relates to a method for processing a 2D or 3D image representative of an object whose contours are disturbed by structured noise and unstructured noise, characterized in that it consists in extracting image, the active contour (or relevant contour) of the object by the following steps: a) elimination, on the initial image, unstructured noise having a magnitude close to that of the contours of the object to extract, by sewing, by a maximum of entropy, a unimodal histogram representative of the distribution of the amplitudes in the image, thus providing a binarized image; b) analyzing the structured noise of the image, from the binarized image, by eliminating the fine structures and filling the holes of the thick structures extracted in step a) to provide an image with significant structures; c) recognition of the edges of the object by characterizing, on the image with significant structures, the structured noise associated with diffraction echoes and the structured noise associated with specular echoes, by extracting and locating the diffraction echoes of the structures significant of the image, thus forming a set of characteristic points, then modeling all these characteristic points according to a linked list of points; and d) recognizing the surfaces of the object, from the image obtained in c), by determining additional characteristic points within the linked list of points, these additional characteristic points being amplitude extrems determined from of the initial image and the image with significant structures, then optimizing the contour formed by all the characteristic points. Advantageously, the characterization of the structured noise associated with diffraction echoes is performed from a parameter a measuring the rate of variation of the minimum depth rmin extracted for each azimuth Oi of a connected component CC, with i Z oieccrmin (Oi ) - inf (rmin (i, SOi E CC) N sup (rmin (e), ii E c) - inf (rmin (i (i O i E CC) where N is the number of points CC, and one parameter 1 indicating the orientation of the characteristic points with ia - amin amax 1 amin where ia is the azimuth of the point of minimum depth of CC, iamin and iamax are, respectively, the azimuths

minimum and maximum of DC.

  According to one embodiment of the method of the invention, step d) contour optimization is performed by the following function: E (p (e)) = 1 (+ ó (e <- _ XI (p ( 0)) d, op and 0 are the polar coordinates of the point considered, I is the intensity at this point, X is a weighting term for taking into account the intensity of the points in the optimization process, and oa and P have different values depending on whether the considered point is in a structured noise zone or in a

an unstructured noise zone.

  According to one embodiment of the method of the invention, step a) consists in applying the KAPUR method (described later) to amplitude images, in which the entropies are

  determined from a number of states fixed a priori.

Brief description of the figures

  - Figure 1 shows schematically the functional diagram of the method of the invention; FIG. 2 represents the structures to be extracted in step b) from the process of the invention; FIG. 3 represents an example of an object immersed in an acquisition volume, the object of which is sought to determine the surface representation; and FIGS. 4A 4F represent the different images obtained, for the object of FIG.

  3, during the process of the invention.

Detailed description of modes of

  Embodiment of the Invention The method of the invention generally consists in allowing the automatic detection of the surface defects of the object being imaged and then the reconstruction of this object from a model base. . In other words, the method of the invention consists, from a raw digital image, in invalidating zones to perform a structural analysis of the volume and prepare the environment for a fine analysis by active contours. After optimization, the active contours model the high-level visual forms to be introduced into the process. This requires breaking down image elements, called "cells", into three categories: background cells, characteristic cells,

invalidated cells.

  This decomposition of cells is

also called "labeling".

  Bottom cells essentially contain background noise (unstructured low frequency noise) and can not be used to characterize object surfaces. information

  in these cells is therefore irrelevant.

  The set of characteristic cells

  constitutes relevant but partial information.

  The invalidated cells do not characterize the visually and semantically rich ultrasound information, but may nevertheless contain it. The method of the invention therefore proposes to identify the different types of echoes by characterizing and classifying the different cells of the digital volumes obtained after

  acquisition of the ultrasound image of the studied object.

  This is achieved through the following four steps: a) unstructured noise analysis, ie background noise; b) a structured noise analysis, ie a detection of the multiple reflections and the signature of the object; c) recognition of the edges of the object from the image obtained in b); and d) a recognition of the surfaces of the object from the raw image (or initial image)

  and images obtained in steps b) and c).

  As mentioned above, the images processed by the method of the invention may be two-dimensional or three-dimensional. They can

  be ultrasonic or electromagnetic type.

  However, to simplify the description, the invention

  will be described only in the case of a two-dimensional ultrasound image described in

polar.

  In addition, the images processed by the method of the invention are intensity images or, more precisely, amplitude images in the sense that what is displayed corresponds to an intensity which encodes the amplitude. Figure 1 shows schematically the

  different steps of the process of the invention.

  Block a corresponds to the analysis of unstructured noises; after this step we obtain a picture Il on which the echoes diffracted and

  specular are physically characterized.

  Block b corresponds to the structured noise analysis which makes it possible to obtain, on an image I2, the geometrical characterization of these diffracted and specular echoes. Block c represents the operation of recognizing the edges of the object by determining the invalid areas of the image I2 and points

  characteristics of the object's outline.

  The images obtained at the output of blocks a and c are then used to perform, in block d, the recognition of the surfaces of the object, with

  the determination of the active contour of the object.

  Finally, block e represents the model

  geometric representation of the surface of the studied object.

  Steps a) to d) will now be

described in more detail.

  Step a) consists of physically characterizing the unstructured noise. This can be of two types: - either of high frequency: in this case, it intervenes on the contours of the structures in the image; - or low frequency: in this case, it appears roughly as a component

continue in the picture.

  This step a) is intended to characterize the background cells, that is, the cells that essentially contain background noise, namely unstructured low frequency noise; these cells can not be used for the characterization of

surfaces of the studied object.

  According to a preferred embodiment of the invention, this step a) consists of a binarization adapted to a digital volume. This binarization is performed by unimodal histogram thresholding by maximum entropy, cut by section. More specifically, the invention proposes using the known KAPUR method for eliminating background noise in grayscale quantized images and adapting it to intensity images. The method of KAPUR is described in the document entitled "a new method for Gray-Level Picture Thresholding using the Entropy of the Histogram", by J. N. KAPUR et al., Published in computer vision, graphics, and image

processing, 29, 273-285, 1985.

  In accordance with the invention, the method of KAPUR has been modified so as to adapt to

  intensity images, not quantified in grayscale.

  The modification relates to the intra-class posterior entropy which is calculated on a number of states fixed a priori, so as to modify the quantification of the intra-class histograms according to the threshold of

binarization of intensities.

  Thus, at the end of step a), a binarized image is obtained, that is to say an image of which all

the cells are at 0 or 1.

  Step b) of the method of the invention consists of a structural analysis of the image of the object, or structured noise analysis. Unlike unstructured noise that is physically characterized, structured noise is geometrically characterized. Structured noise analysis is therefore done from the binarized image obtained in step a). Structural analysis can be carried out according to two distinct methods: by a morphology

  mathematical or by a HOUGH transform.

  The HOUGH transform is a method for detecting the presence of shapes belonging to a family of simple parametric curves (in 2D) or simple parametric surfaces (in 3D) from a binary image or from a set of characteristic points previously extracted. It is a statistical method to transform a pattern recognition problem into

  a problem of finding maxima.

  In mathematical morphology, a simple form, called the structuring element, is used a priori to extract two-dimensional or three-dimensional forms that can be interpreted. It's an inductive process of exploiting what you're looking for. The displacement of this structuring element in the structured set supposes an inclusion relation between the definition spaces of the two sets. In the preferred embodiment of the invention, the structural analysis method used is mathematical morphology, which makes it possible to detect the generally triangular structures associated with edge echoes and the rather rectangular structures.

  associated with specular echoes.

  In addition, the mathematical morphology allows, by an opening operation, to soften the contours and, in particular, to remove small structures. This opening operation conventionally consists of eroding a set of cells by a structuring element, then dilating the assembly resulting from this erosion using the same structuring element. In parallel, a conventional morphological closure operation consists of a dilation,

  then an erosion by a structuring element.

  Closing can fill gaps; the resulting image of this transformation contains the initial form. After binarization of the digital volume, the remaining backscattering information is very fragmented and especially thin compared to the volume sections. Therefore, the method of the invention proposes to use erosion by a small structuring element to remove both the background noise remaining and the backscattering information subsequently processed. Only the structures associated with the diffraction streaks and the streaks of the specular remain. In order to plug the holes observed mainly in the diffraction streaks, the invention proposes to use a

  dilation by a larger 3D structuring element.

  Taking such a structuring element makes it possible to extract solid and significant structures for the purpose of discriminating specular echoes - diffracted echoes. The expected result of this sequence of morphological transformations is described in Figure 2, where the structures of the image are highlighted with a clear line. The practical results are shown in FIGS. 4B and 4C (described in more detail later), figures for which the small structuring element chosen is composed of three azimuths and three depths, whereas the large structuring element is composed of three azimuths and five depths. . Thus, by implementing a modified aperture (changing structuring element between erosion and expansion), it is possible to detect the structured noise contained in the digital volumes of the binarized image. In addition, it is possible to locate backscatter information and eliminate noise.

  unstructured who stayed after binarization.

  The process of the invention is continued by

  step c) recognition of the edges of the object.

  It is recalled that the analysis of unstructured and structured noise of steps a) and b) allowed the detection of a set of structures related to the presence of echoes diffracted by the edges of

  the object, either specular echoes.

  However, diffracted echoes contain topological information, unlike specular echoes. It is therefore important to be able to discriminate the structures resulting from specular echoes of the structures resulting from diffracted echoes in order to be able to determine, among all the structures, those which contain important information, namely information relating to the edges.

of the object.

  According to the invention, this recognition of the edges of the object is achieved by the detection of depth minima which correspond to characteristic points, then the closure, or modeling, of all these characteristic points. However, as explained in step b), a diffraction structure is generally triangular while a specular structure is generally rectangular. It is therefore important to properly characterize triangular structures. Each triangle is characterized by its vertices: the vertex of minimum depth corresponds to a corner of front face of object, the other vertex of adjacent polar angle corresponds to the rear face corner located on the same lateral face as the front corner. The position of the third vertex relative to the other two indicates

  which side to look for the surface information.

  FIG. 2 precisely represents such rectangular R and triangular structures T1 and T2. The triangular structure T1 shows that information must be sought to the right of the image with respect to point A; the triangular structure T2 shows that it is necessary to look for the information towards the left of the image with respect to the point B. The discrimination between triangular and rectangular components is then carried out thanks to a first parameter a measuring the rate of variation of the minima at depth rmin extracted for each azimuth Oi of a connected component CC: i E 6, crmin (ei) - inf (rmin (Oi), Oi E CC) sup (rmi (Oi), ri e CC) - inf (rmin (O), 1 E CC)

o N is the number of CC points.

  If a is close to 0, the front face of the redundancy structure (vis-à-vis the incident waves) is almost flat and the related component is interpreted as the significant drag of a specular echo. If a is close to 0.5, the front is inclined and corresponds to the hypotenuse of

desired diffraction triangle.

  It is then necessary to know the overall orientation of the triangle in order to indicate on which side relative to the azimuth of the point extracted from the connected diffraction component the contour is to be sought. For this, a second parameter i has been introduced; it indicates whether the lower bound in depth of a related diffraction component is on the azimuth side that is less than or greater than the median azimuth of the component: a -iain amax amiin where ia is the azimuth of the point of minimum depth of CC, iamix and iamax are, respectively, the azimuths

  minimum and maximum of the studied component.

  Thus, a and f are the two characteristic parameters of 2D diffractive connected components from which it is possible to extract the minimum depth point labeled at + 1 according to whether the surface to be extracted is on the right or on the right.

left of the point.

  Finally, a linked list of labeled and ordered points in increasing azimuth is constructed. All other points (or associated cells) belonging to the redundancy structures

are invalidated.

  The method of the invention proposes, then,

  a step d) of recognizing the surfaces of the object.

  This surface recognition is established from the image obtained in c) the edges of the object and the image of the significant structures obtained after step b). In 2D, this surface recognition results in the detection of plane curves from snap points (or characteristic points) and, in 3D, by the detection of surfaces from curves

left hanging.

  More precisely, this step d) consists first of all in an initialization of the "active" contours of the object, in other words, the actual contours of the object. For this, we consider the characteristic points of the linked list determined in step c) as the ends of curves to be extracted and we search for other points of attachment of the initial contours, also called interior points, which

  do not belong to the edges determined in step c).

  First, the points of the list

  chained to match must be opposite labels.

  The succession of two points of the same label means that an outline hides another. This simplification results in the exclusive treatment of pairs of points, opposite labels, successive in the linked list. On the other hand, if the first edge encountered indicates that the object is near the edge of the numeric volume, the missing edge is initialized by the point

of shallower depth.

  Once the pair of points has been detected, the search for internal hooking points is done by detection of order extrema (for example, order 20) non-invalid. Finally, we add an intermediate azimuth point of the polar grid: that which is just below the intersection of the segment connecting two points of attachment (inner or end) by the given azimuth radius. This set of points is subject to movement, each according to its line in depth (at fixed azimuth) and according to a process

  optimization that will now be described.

  The optimization of the active contours consists in choosing a deformable model that integrates the structure of the volume sections: an active contour is sought in the form of a 2D polar graph with coordinates (p, 0). The optimization method, according to the invention, is based on the active contour expressed from the EULER-LAGRANGE theorem with the following functional: 6, Dp2 a2P E (P ()) = (O + (0) - I (p (", e0o ae 80 op and 0 are the polar coordinates of the point in question, I is the intensity at this point, x is a weighting term for taking into account the intensity of the points in the process of optimization, and oa and I have different values depending on whether the considered point is in a structured noise zone or in

an unstructured noise zone.

  The discretization adopted constrains

  model points to stay on their line in depth.

  Only a local approximation of the intensities on the lines in depth will guide the displacement of the points against the regularizing internal force, which acts more on a regularity according to the azimuths. The interest of this constraint is to obtain a good distribution of the points even if the difference between

  the points are not strictly constant.

  By applying the EULER-LAGRANGE theorem for the functional E (p (8)) given above, we obtain the following functional equation: - () 5 ep + P (O) I2p ai (p (0))

- + = δp y 5 2 ap Boundary conditions are fixed by p (eo), p '(0o),

  p (0i), p '(i), which guarantee uniqueness

  of the solution of the differential system of order 4.

  The larger the ratio x / 5, the more rigid and short the curve, which is needed in invalid areas. In addition, so that the internal energy is more taken into account in the invalid areas relative to the external energy, rather than taking 3 constant and increase a, either in practice: - if the point is valid> a = 1, 3 = 10

  - if the point is invalid => a = 1, 3 = 100.

  During diffraction, the time dependence of the active contour appears as kinetic energy in the differential equation: 2 Yp ap + ap a (p (e)) at 502 f04 Sp o Y plays the role of weight of the curve . According to a discretization with finite differences in 0 and t, the previous differential equation is transformed into a matrix system: (A + - IdN) R (t) = R (t - At) - XF (t - At), At o R (t) = [p (O, t), p (AG, t), ..., p (AON, t)], aI F (t) = is the derivative at each point of the ap active contour of a cubic Bspline approximating each line in depth traversed by the curve, and A is a pentadiagonal matrix in the case of open active contours (with free or fixed ends), of symmetrical band: [lai, i-2; ai, i-1 ; ai, i; ai, i + 1; ai, i + 2] with: Ii-1 ai, i-2 = h4 aiil = 4 (-h2a-2i1-2ij) aii = - (h2 (ai + i i + l) 4j + i- + 4i + i + 1) h4 aii + l = h4 (-h2 (ai + 1) - 25i - 20i + 1) Ii + 1 ai, i + 2 h4 valid from lines 2 to N - 2. The remaining lines are determined by the conditions at the ends of the curve. Moreover, h is the average distance between two

  points of the active contour at time t.

  When steps a), b), c) and d) are completed, we obtain a 2D active contour model that integrates the structural analysis into its parameters.

stiffness and elasticity.

  Figures 3 and 4A to 4F show an example of application of the method of the invention. More specifically, FIG. 3 represents a flat plate

  which one seeks to obtain a good representation.

  This flat plate has a length of 0.8 m, a width of 0.7 m and a thickness of 0.02 m. She is willing,

  for the example considered, 2 m from the imaging system.

  The digital volumes shown are restricted to areas of good performance of the imaging system, namely over an area ranging from +15 in azimuths with a pitch of 0.25, from -5 to +1 in sites with also a step of 0 , 25, and a depth field ranging from

  1.98 m to 2.1 m discretized on 101 points.

  Given the shape of the plate, when taking an image by an ultrasound imaging system, multiple reflections between the different faces of the plate will create different echoes. It is these echoes that the method of the invention seeks to characterize to obtain a good representation of

the plaque.

  Figures 4A to 4F show the different representations of the object obtained during the process. More precisely, FIG. 4A represents the ultrasonic image of the flat plate, viewed from above. FIG. 4B represents the image obtained

  after step a) unstructured noise analysis.

  FIGS. 4C and 4D represent the binarized images obtained, respectively, during step b) and at the end of this step b) of analysis of the

structured noise.

  FIG. 4C corresponds to the image obtained after the erosion operation of the image 4B by small structuring elements, and FIG. 4D represents the image obtained after expansion by a large structuring element. In these two figures 4C and 4D, it is now clear that the object has two edges, represented by substantially triangular shapes, and a direct path represented by the substantially rectangular shape. FIG. 4E shows the plane plate after step c) of recognizing the edges of the object and FIG. 4F shows the active contour of the plate

  plane, obtained at the end of the process of the invention.

Claims (4)

  1. A method for processing a 2D or 3D image representative of an object whose contours are disturbed by structured noise and nonstructured noise, characterized in that it consists in extracting from the image, the relevant contour of the object by the following steps: a) elimination, on the initial image, of the unstructured noise having a magnitude close to that of the outlines of the object to be extracted, by thresholding, by a maximum of entropy, a unimodal histogram representative of the distribution of the amplitudes in the image, thus providing a binarized image; b) analyzing the structured noise of the image, from the binarized image, by eliminating the fine structures and filling the holes of the thick structures extracted in step a) to provide an image with significant structures; c) recognition of the edges of the object by characterizing, on the image with significant structures, the structured noise associated with diffraction echoes and the structured noise associated with specular echoes, by extracting and locating the diffraction echoes of the structures significant of the image, thus forming a set of characteristic points, then modeling all these characteristic points according to a linked list of points; and d) recognizing the surfaces of the object, from the image obtained in c), by determining additional characteristic points, interior to the linked list of points, these additional characteristic points being amplitude extremes determined from of the initial image and the image with significant structures, then optimizing the contour formed by all the characteristic points. 2. Method according to claim 1, characterized in that the characterization of the structured noise associated with diffraction echoes is made from a parameter a measuring the rate of variation of the minimum depth rmin extracted for each azimuth ei of a connected component CC, with N eeccrmin (ei) - inf (rmin (ei) Oi e Cc) N sup (rmin (Oi), ei E CC) - inf (rm, .n (,) ,, e CC) 'o N is the number of points CC, and a parameter D indicating the orientation of the characteristic points with = a 1ami ni amax amin O ia is the azimuth of the point of minimum depth of CC and iamin and iamax are, respectively, the round minimum and maximum of DC.   3. Method according to claim 2, characterized in that step d) contour optimization is performed by the following function: E (p (e)) = j (e + D (e8_2P - xI (p (e ) "d, op and e are the polar coordinates of the point considered, I is the intensity at this point, X is a weighting term for taking into account the intensity of the points in the optimization process, and oa and X have different values depending on whether the considered point is in a structured noise zone or in a an unstructured noise zone.   4. Process according to any one of   Claims 1 to 3, characterized in that step a)   consists in applying the KAPUR method to amplitude images, in which the entropy is determined at   from a number of states fixed a priori.