FR3112012A1 - Method for generating at least one generalized detection of a large source - Google Patents

Abstract

The invention relates to a method for generating at least one generalized detection of a broad source present in an image comprising the successive steps of: - from at least one elementary detection of said broad source, generating a segmentation mask associated with said detection, - generate at least one segmentation region such that each generated segmentation region is defined so as to encompass: either a single segmentation mask, or several overlapping segmentation masks, - determine for each region segmentation generated generalized detection. FIGURE OF THE ABRIDGE: Fig. 2

Description

Method for generating at least one generalized detection of a large source

The invention relates to a method for generating a generalized detection of a broad source present on an initial image by processing said image or for generating several generalized detections, each generalized detection relating to a distinct broad source on the image initial by processing said image.

BACKGROUND OF THE INVENTION

In the field of the navy, it is known to have recourse to systems for acquiring and processing at least one image to target objects such as, for example, surface ships.

There illustrates a possible architecture of an existing system. Such a system comprises different modules: a module for determining detections 101, a module for merging detections 120 and a module for classifying and/or tracking merged detections 103.

In fact, the detection determination module 101 can comprise two blocks:

• one 104 for large sources, also called extended source (i.e. the source is represented by at least ten pixels on the image), and
• the other 105 for so-called point sources (i.e. the source is represented by a few pixels at most in the image).

Each block 104, 105 can implement an algorithm which assumes that the background of the image is Gaussian and is therefore characterized by at least two parameters which are the “average” and the “variance”. Thus the algorithm can evaluate the following criterion(s):

• an average of the background surrounding an object in the image (the average of the background will be called “I _m B” below), ie the average level of the pixels constituting the background surrounding an object;
• a variance of the noise of said background (subsequently the standard deviation of the background will be denoted “σ _B ” and its variance “σ _B ² ”).

For example, the algorithm determines the average level of the pixels constituting an object (said average level will be called “I _m T” below). Alternatively, the algorithm imposes I _m T on the value of the central pixel of the object.

Then the algorithm estimates the signal to noise ratio (hereinafter called "SNR") by the formula

SNR = (I _m T - I _m B)/ σ _B

In case of management of positive contrasts

Or alternatively according to the formula (in case of management of negative contrasts):

SNR = |I _m T - I _m B|/ σ _B

Based on the SNR value and a nominal threshold value Th1, value predetermined based on detection performance and/or false alarm tolerance:

- if SNR is less than Th1 then there is no detection,

- if SNR ≥ Th1 then there is detection and the block 104, 105 outputs the position of the detection and/or one or more information characteristic of the detection (morphology, entropy, etc.).

According to a variant, the algorithm is based on a segmentation by Markov fields: the algorithm thus works on the image by successive iterations to obtain homogeneous zones that can be modeled by Gaussians, the contours of which it then analyzes to obtain detections.

Regardless of the algorithm used, a recurring problem is poor analysis of broad sources. Thus, it regularly happens that the system ends up deducing from an image that there are several detections for the same broad source.

This of course harms the proper counting of sources but also disrupts the processing of information related to deductions, especially when one wishes to track a particular source, which affects the performance of the whole.

OBJECT OF THE INVENTION

An object of the invention is to propose a method limiting a risk of multiple detections of the same broad source present on an image.

With a view to achieving this goal, according to the invention, a method is proposed for generating at least one generalized detection of a broad source present on an image comprising the successive steps of:

• from at least one elementary detection of said broad source, generate a segmentation mask associated with said detection,
• generate at least one segmentation region such that each generated segmentation region is defined to encompass:

either a single segmentation mask,

either several segmentation masks that overlap with each other,

• determining for each segmentation region generated a generalized detection.

In this way, the invention allows a fusion dedicated to large sources.

When the invention is used in a system for acquiring and processing at least one image to target objects, the invention thus limits the risk of obtaining two targets for the same broad source.

The invention therefore makes it possible to improve the counting of targets. The invention also makes it possible to limit errors in the subsequent processing of the image and/or of the target, for example at the level of a module for tracking said target.

More generally, the invention makes it possible to improve the detection of a target, the tracking stability of a target, the stability of counting targets, the stability of the classification and/or identification of targets, etc.

The invention thus proves to be relatively effective. In particular, a device implementing the invention proves to be relatively efficient, in particular in terms of false alarms (or False Alarms).

Advantageously, the invention can quickly and easily be implemented in existing devices.

The invention is therefore inexpensive to implement.

Furthermore, the invention does not excessively increase the calculation time in the event of implementation in an existing device.

For the present application, it is recalled that a broad source (or extended source) is a source represented by at least ten pixels on an image and which thus opposes the notion of a point source represented by a maximum of a few pixels on a picture.

Optionally, a cumulative histogram is determined on a portion of the initial image centered on the position of the elementary detection of the wide source considered.

Optionally, a local detection threshold is determined based on gray levels of the pixels of said portion by thresholding the accumulated histogram by an intermediate threshold of predetermined value.

Optionally, the intermediate threshold has a value between 80 and 90%.

Optionally, the initial image is thresholded in gray levels by the local detection threshold in order to obtain a first image presenting one or more segmentations associated with the elementary broad source detection.

Optionally, the first image is dilated and/or eroded to obtain a second image.

Optionally, the first image or the second image are labeled to extract the at least one segmentation mask.

Optionally, a new image is created including the at least one segmentation mask and said image is labeled to extract the at least one segmentation region.

Optionally, a position of the at least one generalized detection of the broad source associated with the at least one segmentation region is calculated.

Optionally, the position of the at least one generalized detection of the broad source is considered to be the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level.

The invention also relates to a device making it possible to implement the aforementioned method.

The invention also relates to an optronic system comprising a device as mentioned above.

The invention also relates to a marine vehicle comprising an optronic system as mentioned above.

Other characteristics and advantages of the invention will become apparent on reading the following description of a particular implementation of the invention.

The figure has already been described in relation to the prior art of the invention.

Furthermore, the invention will be better understood in the light of the following description with reference to the appended figures, including:

there schematically illustrates in part a system allowing a particular implementation of the invention, said system comprising an image processing device;

there is a schematic representation of a first block of a third module of the image processing device illustrated in ;

there is a schematic representation of an image processed during a first step by the first block illustrated in ;

there is a schematic representation of a second block of the third module of the image processing device illustrated in ;

there is a schematic representation of a binary image generated by the second block shown in ;

there is a schematic representation of another binary image generated by the second block shown in .

DETAILED DESCRIPTION OF THE INVENTION

With reference to the , a system allowing a particular implementation of the invention will now be described. Such a system is for example an optronic system. It is recalled that an optronic system relies on both optics and electronics to fulfill its function(s).

In the present case, the optronic system comprises at least one image pickup element (optical sensor type) as well as a device for processing said images 1 in order to allow the detection of targets. Preferably, the system also comprises one or more modules arranged downstream of the image processing device 1 such as for example a module for classifying and/or identifying said targets, a tracking module for said targets, an electro-optical firing control …For example, the system includes a target classification module 2.

Optionally, the images acquired by the image pickup element are infrared images.

In addition, the images acquired by the imaging element have a monomode background. As a variant, the images acquired by the image pickup element have a multimodal (i.e. textured) background.

In the present case, the system is arranged on a marine vehicle (such as a surface ship or even a submarine).

The system is for example an optronic marine watch system.

The image processing device 1 is for example a calculator, a controller, a computer, etc. incorporating various modules which will be described later. Each module can for example be implemented by a reprogrammable logic circuit (such as an FPGA circuit) implanted in the image processing device 1.

The image processing device 1 comprises a first module 3 for point source detections and a second module 4 for elementary detections of broad sources. Said sources are in the present case surface objects such as surface ships and/or even aircraft.

In a manner known per se, the function of the first module 3 is to segment each image in order to obtain detections of point sources and the function of the second module 4 is to segment each image in order to obtain elementary detections of large sources. For example each module 3, 4 detects on each image the points of interest by evaluation of the SNR as described in the part “Technological background of the invention”. These points of interest correspond to "contrasted" points or even "salience" points, i.e. "hot" points if the image processing is based on positive contrast or "cold" points if the image processing is based on a negative contrast.

The first module 3 and the second module 4 are well known from the prior art and will not be further detailed here.

It is noted that at the input of the first module 3 and of the second module 4 there is an initial image acquired by the optical sensor and at the output of the first module 3 detections of point sources are recovered and at the output of the second module 4 elementary detections of broad sources.

The image processing device 1 comprises a third module 5 for merging the elementary detections of large sources. This third module 5 is therefore arranged downstream of the second module 4 so as to have as input the elementary detections of wide sources determined by the third module 4.

With reference to the , the third module 5 comprises a first block 11 calculating from said elementary detections of wide sources, segmentation masks associated with each of said detections.

Preferably, the first block 11 retains information transmitted by the second module 4 only the position of the detections of large sources (positions expressed in line-column).

Optionally the first block 11 carries out the following loop.

Beginning of the loop.

For each of the detections elementary s of wide source provided by the second module 4 , the different rungs following are implemented.

According to a first stage of a first level, the first block 11 determines a cumulative histogram on a portion of the initial image centered on the position of the elementary detection of wide source considered. This portion of the initial image is of predetermined shape and dimensions identical to all the elementary detections of wide sources. For example the portion of the image is a rectangle. There represents such a portion of an image. As for the cumulative histogram, it is a local representation of the distribution of the levels of gray (also called “intensities”) of the pixels around the position of elementary detection of wide source considered. A cumulative histogram associated with an elementary detection of broad source considered can therefore be totally different from that evaluated for another detection of broad source yet belonging to the same broad source.

Thus, during the first stage, a local analysis of gray levels around each detection elementary widely considered source taken from the initial image .

Then, during a second stage of the first level, the first block 11 determines a local detection threshold Thp based on the gray levels of the pixels of the portion of the image by thresholding the cumulative histogram by an intermediate threshold in percentage Th_% of predetermined value.

Optionally, we choose Th_% has a value between 80 and 90%. For example, Th_% is chosen at 85%. This means that it is sought to retain only 15% of the pixels present in the portion of the image, independently of their positions in said portion. For each portion of the initial image associated with a broad source elementary detection , the local detection threshold Thp depends on Th_% and the cumulative histogram : said local detection threshold Thp will therefore depend directly on the local neighborhood of the Elementary wide source detection associated e and will be different from one portion to another .

During a second step, the initial image is in turn thresholded in gray levels (whereas at the level of the first module 3 and of the second module 4 the initial image was thresholded in SNR) by the local detection threshold thp.

A binary image I2p is thus obtained presenting one or more segmentations associated with the elementary broad source detection.

We note that the threshold at the local level of detection Thp is specific to each broad source elementary detection so that the different I2p binary images each associated with one of the elementary detections of large sources are different from each other.

A local segmentation is thus performed around each of the detections elements from broad sources after analysis of the distribution of gray levels around said detections wide source elements are. I a segmentation performed in the third module 5 is therefore finer than in the second module 4 thanks in particular to use of the cumulative histogram.

Optionally, the first block 11 expands and/or erodes the binary image I2p for:

• delete isolated segmentations, for example if said segmentations are too small and/or inconsistent in view of the other segmentations; and or
• group close segmentations together.

To this end, the first block is based on predetermined criteria to define a "small" segmentation, an "incoherent" segmentation, an "isolated" segmentation, "close" segmentations... For example for "close" segmentations, the first block can be based on a proximity criterion expressed in row-column distance.

A binary image I3p is thus obtained having a main segmentation (and preferably which is the only segmentation of said binary image I3p) associated with the elementary broad source detection.

Said main segmentation preferably includes the broad source elementary detection. Furthermore, said main segmentation is chosen so as to be the most important segmentation in terms of area which includes the broad source elementary detection.

Preferably, the first block 11 labels the binary image I3p to extract a zone in the binary image I3p associated with the elementary wide source detection. Preferably, said zone is defined so as to contain the main segmentation without including other segmentations of the I3p binary image. Said zone is also called “segmentation mask” below.

Preferably, the segmentation mask follows the contours of the main segmentation and has a larger area but as close as possible to said main segmentation.

The segmentation mask therefore advantageously has a larger area than the broad source elementary detection while being associated with said broad source. This ensures better segmentation with the third module 5 than at the end of the second module 4.

An image (of the same dimensions as the initial image) comprising said segmentation mask can thus be generated. Such an image is denoted binary image I4p.

The other zones of the binary image I3p are thus considered as being uninteresting, which is logical since the local detection threshold Thp has no meaning for these other zones, the local detection threshold Thp having been evaluated locally around the elementary detection of the wide source.

End of the loop

The output of the first block 11 thus provides a segmentation mask for each elementary wide source detection transmitted at the input of the first block 11.

Preferably, if the first block 11 cannot determine a segmentation mask for at least one of the elementary broad source detections, the first block 11 outputs a segmentation zone associated with said elementary broad source detection which has been determined by the second module 4. In this way, elementary wide source detection is not lost through the first block 11.

Optionally, if the main segmentation does not include the elementary broad source detection (for example because of erosion and/or dilation), an additional segmentation is sought in the vicinity of said elementary broad source detection, segmentation which would include said detection. The neighborhood is for example a zone of 3 pixels by 3 pixels centered on the elementary broad source detection (or a central pixel of said elementary broad source detection). The corresponding segmentation mask will then be defined vis-à-vis this additional segmentation (and not the main segmentation). If no annex segmentation can be determined then, as indicated above, it is considered that the first block 11 cannot determine a segmentation mask for this elementary detection of wide source and the first block 11 outputs an associated segmentation zone to said elementary broad source detection which has been determined by the second module 4. The third module 5 also comprises a second block 12 arranged downstream of the first block 11 and receiving from the latter the segmentation masks and/or the I4p images (and possibly segmentation zones associated with one or more elementary detections of broad sources, elementary zones which have been determined by the second module 4 and/or images comprising such zones) to provide generalized detections of broad sources as output.

To this end, and according to a first level, the second block 12 creates a binary image I4 which is in reality the union of all the binary images I4p (and optionally of the segmentation zones calculated by the second module 4 and/or images including such areas).

For example, the binary image I4 is created as follows:

• for each pixel belonging to a segmentation mask in an I4p image (and optionally to a segmentation zone calculated by the second module 12), the second block sets to 1 the value of the pixel having the same row-column position in the image binary I4;
• if several segmentation masks (and optionally a segmentation zone calculated by the second module 12) include the same pixel, the value of the corresponding pixel in the binary image I4 is maintained at 1;
• otherwise the value of the pixel of the binary image I4 is taken equal to 0.

An example of an I4 binary image is visible at .

According to a second step, the second block 12 labels the binary image I4 to extract segmentation regions in the binary image I4 which are independent of each other and to consequently associate the segmentation masks which overlap, the segmentation masks overlapping thus jointly forming the same region of segmentation.

The second block 12 thus constitutes one (or more) segmentation region(s) grouping together (each) the segmentation masks associated with the various elementary detections of the same broad source.

An example of a labeled I4 binary image is visible at .

We note that each segmentation region is defined in such a way as to encompass:

• either a single segmentation mask,
• or several segmentation masks which overlap with one another.

Preferably, the segmentation region follows the contours of the associated segmentation mask(s) and has a larger area but as close as possible to said mask (or to the overlapping of said masks).

Moreover, it is thus possible to define a generalized detection of the broad source associated with each segmentation region.

This definition is made here during a third level.

Optionally, the second block 12 considers that the position of said general detection is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level. The second block can use the following formula for this purpose:

with

X = the number of the column (or row) considered in the segmentation region,

i = a pixel of the segmentation region,

level = the gray level of the considered pixel, and

Mask = the set of pixels constituting said segmentation region.

Thus, the second block 12 provides at its output (which here also corresponds to that of the third module 5) the generalized detections of wide sources and in particular their positions.

These generalized detections of broad sources are thus defined so as to group elementary detections of broad sources associated with the same broad source by the second module.

Advantageously, these generalized detections of large sources are moreover more refined than the elementary detections of large sources with regard to their morphological characteristics (position, contour, etc.)

With reference again to the , the image processing device 1 comprises a fourth module 6 for merging the generalized detections of large sources generated by the third module 5 with the detections of point sources generated by the first module 3. This fourth module 6 is therefore arranged downstream of the first module 3 and the third module 5.

The fourth module 6 operates like a prior art broad source-point source fusion module (with the difference that the input data of the fourth module 6 will therefore be the generalized detections of broad sources coming from the third module 5 and no longer the elementary detections of broad sources generated by the second module 4) so that it will not be detailed here.

At the output of the fourth module 6 (which here also corresponds to the output of the image processing device 1), the targets are recovered which will then be processed by the rest of the system.

An image processing device 1 comprising two modules 4, 5 dedicated to the detection and segmentation of large sources has thus been proposed. The second module 4 allows the detection and segmentation of the points of interest on which to focus then the third module 5 performs a local analysis of the cumulative histogram around all the points of interest provided by the second module 4. The third module 5 therefore makes it possible to adapt the local detection threshold for each of the points of interest and to obtain segmentation masks associated with said points of interest (more finely segmented than if one were content to use the second module 4), masks which are then merged by proximity criterion, in order to develop a single detection by wide source.

More generally, an image processing device 1 has been proposed based on a segmentation of large sources controlled by local multi-statistical criteria with the aim of associating them with a single detection.

Advantageously, the second module 4 and the third module 5 are relatively independent.

Thus the second module 4 can be relatively rustic and/or an already existing module of the prior art.

The use of the second module 4 - third module 5 duplicate therefore advantageously avoids the third module 5 having to resort to a complex process of segmentation around the broad source since the third module 5 takes directly as input the elementary detections of broad sources provided by the second module 4 (which is a relatively rustic module) which thus indicates to it the zones to be favored for the segmentation.

Similarly, the first module 3 and the fourth module 6 can be already existing modules of the prior art.

The third module 5 can thus easily be installed in existing image processing devices of the prior art.

The third module 5 therefore improves the performance of the image processing device 1 and therefore of the system.

The image processing device 1 thus described improves the segmentation of large sources and their counting while maintaining the requirement for an acceptable false alarm rate (rate linked to the second module 4). Advantageously, the image processing device 1 thus described can even reduce this rate of false alarms thanks to the groupings of the multi-detections on the sources (which can make it possible to rule out unwanted detections such as for example waves on the sea, edges of clouds….).

Of course, the invention is not limited to the implementation described and variant embodiments may be added thereto without departing from the scope of the invention.

In particular, the invention is applicable to many other applications than the military maritime field. The invention can thus be applied in the civil field, in the military field, in the land field, in the aeronautical field, in the airborne field, in the field of transport in general, etc.

It is also possible, in addition or as a replacement, to work in other wavelength bands than infrared and for example in the visible.

In the same way, we can use different types of optical sensors: camera (infrared, visible, etc.); radar (synthetic aperture radar usually designated by the English acronym SAR, side aperture radar, etc.); …

The third module may be different from what has been indicated and for example fulfill additional functions. Thus if a segmented region and/or a segmentation mask is inconsistent (for example in terms of morphology), the third module may decide to associate with the elementary detection(s) of large sources the characteristics (in particular the position) provided by the second module. The detection of the inconsistency can for example be based on one or more of the following parameters: obtained number of pixels in the segmented region and/or the segmentation mask on total number of pixels, maximum sizes allowed in predetermined width and height in the segmented region and/or the segmentation mask, width to height ratio in the segmented region and/or the segmentation mask...

Claims (13)

Method for generating at least one generalized detection of a broad source present on an initial image comprising the successive steps of:
- from at least one elementary detection of said broad source, generate a segmentation mask associated with said detection,
- generate at least one segmentation region such that each generated segmentation region is defined so as to encompass:
either a single segmentation mask,
either several segmentation masks that overlap with each other,
- determining for each segmentation region generated a generalized detection. Method according to claim 1, in which a cumulated histogram is determined on a portion of the initial image centered on the position of the elementary detection of wide source considered. Method according to claim 2, in which a local detection threshold is determined based on gray levels of the pixels of said portion by thresholding the accumulated histogram by an intermediate threshold of predetermined value. Method according to Claim 3, in which the intermediate threshold has a value of between 80 and 90%. Method according to one of Claims 3 to 4, in which the initial image is thresholded in gray levels by the local detection threshold in order to obtain a first image exhibiting one or more segmentations associated with the broad source elementary detection. Method according to claim 5, in which the first image is expanded and/or eroded to obtain a second image. Method according to Claim 5 or Claim 6, in which the first image or the second image is labeled in order to extract the at least one segmentation mask. Method according to one of Claims 1 to 7, in which a new image is created including the at least one segmentation mask and the said image is labeled in order to extract the at least one segmentation region. Method according to one of Claims 1 to 8, in which a position of the at least one generalized detection of the broad source associated with the at least one segmentation region is calculated. Method according to claim 9, in which it is considered that the position of the at least one generalized detection of the broad source is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level. Image processing device making it possible to implement the method according to one of Claims 1 to 10, the device comprising a first module (3) for detecting point sources, a second module (4) for elementary detection of broad sources , a third module (5) for merging elementary detections of broad sources generated by the second module and a fourth module (6) for merging generalized detections of broad sources generated by the third module with the detections of point sources generated by the first module. Optronic system comprising a device according to claim 11. Marine vehicle comprising an optronic system according to claim 12.