WO2022002583A1

WO2022002583A1 - Method for generating at least one pervasive detection of a broad source

Info

Publication number: WO2022002583A1
Application number: PCT/EP2021/066127
Authority: WO
Inventors: Dominique MALTESE
Original assignee: Safran Electronics & Defense
Priority date: 2020-06-29
Filing date: 2021-06-15
Publication date: 2022-01-06
Also published as: FR3112012A1

Abstract

The invention relates to a method for generating at least one pervasive detection of an image-based broad source, comprising the successive steps of: - from at least one basic detection of the broad source, generating a segmentation mask associated with the detection, - generating at least one segmentation region such that each generated segmentation region is defined so as to include: either a single segmentation mask or several segmentation masks which overlap with one another, - determining, for each generated segmentation region, a pervasive detection.

Description

Method for generating at least one generalized detection of a large source

The invention relates to a method for generating a generalized detection of a wide source present in an initial image by processing said image or generating several generalized detections, each generalized detection relating to a distinct wide source on the image. initial by processing said image.

BACKGROUND OF THE INVENTION

In the field of the military navy, it is known to have recourse to systems for acquiring and processing at least one image in order to target objects such as, for example, surface ships.

Figure 1 illustrates a possible architecture of an existing system. Such a system comprises different modules: a module for determining the detections 101, a module for merging the detections 120 and a module for classifying and / or tracking the merged detections 103.

In fact, the detection determination module 101 can comprise two blocks:

- one 104 for wide sources, also called extended source (i.e. the source is represented by at least ten pixels on the image), and

- the other 105 for so-called point sources (ie the source is represented by a few pixels at most in the image).

Each block 104, 105 can implement an algorithm which assumes that the background of the image is Gaussian and is therefore characterized by at least two parameters which are the “mean” and the “variance”. Thus the algorithm can evaluate the following criteria (s):

- an average of the background surrounding an object on the image (the average of the background will be called “I _m B” hereafter) ie the average level of the pixels constituting the background surrounding an object;

- a variance of the noise of said background (subsequently the standard deviation of the background will be denoted "s _B " and its variance "s _B ² ").

For example, the algorithm determines the average level of the pixels constituting an object (said average level will be called “I _m T” hereafter). Alternatively, the algorithm imposes I _m T on the value of the central pixel of the object.

Then the algorithm estimates the signal to noise ratio (hereinafter called “SNR”) by the formula SNR = (I _m T - I _m B) / o _B

In case of management of positive contrasts

Or as a variant according to the formula (in case of management of negative contrasts):

SNR = | I _m T - I _m B | / o _B

Based on the SNR value and a nominal threshold Thl value, a predetermined value based on detection performance and / or false alarm tolerance:

- if SNR is less than Thl then there is no detection,

- If SNR ³ Th1 then there is detection and the block 104, 105 outputs the position of the detection and / or one or more information characteristic of the detection (morphology, entropy, etc.).

According to one variant, the algorithm is based on segmentation by Markov fields: the algorithm thus works on the image by successive iterations to obtain homogeneous zones that can be modeled by Gaussians, the contours of which it then analyzes to obtain their detections. Whatever algorithm is used, a recurring problem is the poor analysis of broad sources. Thus, it often happens that the system ends up deducing from an image that there are several detections for the same large source. This of course impairs the correct counting of sources but also disrupts the processing of the information (s) linked to the deductions, in particular when it is desired to track a particular source which adversely affects the performance of the assembly.

OBJECT OF THE INVENTION

An aim of the invention is to propose a method limiting a risk of multiple detections of the same large source present in an image.

SUMMARY OF THE INVENTION

With a view to achieving this aim, according to the invention there is proposed a method for generating at least one generalized detection of a wide source present in an image comprising the successive steps of:

- from at least one elementary detection of said wide source, generating a segmentation mask associated with said detection,

- generate at least one segmentation region such that each segmentation region generated is defined so as to include: either a single segmentation mask, or several segmentation masks which overlap with each other,

- determining for each segmentation region generated a generalized detection.

In this way, the invention allows a fusion dedicated to wide sources.

When the invention is used in a system for acquiring and processing at least one image in order to target objects, the invention thus limits a risk of obtaining two targets for the same large source. The invention therefore makes it possible to improve the count of targets. The invention also makes it possible to limit the errors in the subsequent processing of the image and / or of the target, for example at the level of a module for tracking said target.

More generally, the invention makes it possible to improve the detection of a target, the stability of tracking a target, the stability of the count of targets, the stability of the classification and / or identification of the targets, etc. The invention thus proves to be relatively efficient. In particular, a device implementing the invention proves to be relatively efficient, in particular in terms of false alarms (or false alarms).

Advantageously, the invention can quickly and easily be implemented in existing devices. The invention is therefore inexpensive to implement.

In addition, the invention does not excessively increase the computation time in the event of implementation in an existing device.

For the present application, it is recalled that a wide source (or extended source) is a source represented by at least ten pixels on an image and which is thus opposed to the notion of point source represented by at most a few pixels on an image. a picture.

Optionally, a cumulative histogram is determined on a portion of the initial image centered on the position of the elementary wide source detection considered. Optionally, a local detection threshold based on the gray levels of the pixels of said portion is determined by thresholding the cumulative histogram by an intermediate threshold of predetermined value.

Optionally, the intermediate threshold has a value between 80 and 90%.

Optionally, the initial image is thresholded in gray levels by the local detection threshold in order to obtain a first image exhibiting one or more segmentations associated with the elementary detection of a wide source. Optionally, the first image is dilated and / or eroded to obtain a second image.

Optionally, the first image or the second image is labeled in order to extract the at least one segmentation mask. Optionally we create a new image including the au minus one segmentation mask and labels said image to extract the at least one segmentation region. Optionally, a position of the at least one generalized detection of the wide source associated with the at least one segmentation region is calculated.

Optionally, it is considered that the position of the at least one generalized detection of the wide source is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level. The invention also relates to a device making it possible to implement the aforementioned method.

The invention also relates to an optronic system comprising a device as mentioned above.

The invention also relates to a marine vehicle comprising an optronic system as mentioned above.

Other characteristics and advantages of the invention will emerge on reading the following description of a particular implementation of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 has already been described in relation to the prior art of the invention.

Furthermore, the invention will be better understood in the light of the description which follows with reference to the appended figures, among which:

FIG. 2 schematically illustrates in part a system allowing a particular implementation of the invention, said system comprising an image processing device;

FIG. 3 is a schematic representation of a first block of a third module of the image processing device illustrated in FIG. 2;

FIG. 4 is a schematic representation of an image processed during a first step by the first block illustrated in FIG. 3; FIG. 5 is a schematic representation of a second block of the third module of the image processing device illustrated in FIG. 2;

FIG. 6 is a schematic representation of a binary image generated by the second block represented in FIG.

5;

Figure 7 is a schematic representation of another binary image generated by the second block shown in Figure 5.

DETAILED DESCRIPTION OF THE INVENTION

With reference to FIG. 2, a system allowing a particular implementation of the invention will now be described. Such a system is for example an optronic system. It will be recalled that an optronic system relies on both optics and electronics to fulfill its function (s).

In the present case, the optronic system comprises at least one image-taking element (optical sensor type) as well as a device for processing said images 1 in order to allow the detection of targets. Preferably, the system also comprises one or more modules arranged downstream of the image processing device 1 such as, for example, a module for classifying and / or identifying said targets, a module for tracking said targets, an electro-optical firing line. ... For example, the system includes a target classification module 2.

Optionally, the images acquired by the imaging element are infrared images.

Further, the images acquired by the imaging element have a single-mode background. As a variant, the images acquired by the imaging element have a multimodal (i.e. textured) background.

In the present case, the system is arranged on a marine vehicle (such as a surface ship or even a submarine). The system is, for example, an optronic marine watch system.

The image processing device 1 is for example a computer, a controller, a computer, etc. incorporating various modules which will be described below. Each module can for example be implemented by a reprogrammable logic circuit (such as an FPGA circuit) installed in the image processing device 1.

The image processing device 1 comprises a first module 3 for detecting point sources and a second module 4 for elementary detections of wide sources. Said sources are in the present case surface objects such as surface ships and / or aircraft.

In a manner known per se, the function of the first module 3 is to segment each image in order to obtain detections of the point sources and the function of the second module 4 is to segment each image in order to obtain elementary detections of the wide sources. For example, each module 3, 4 detects the points of interest on each image by evaluating the SNR as described in the section “Technological background of the invention”. These points of interest correspond to “contrasting” points or even “salience” points, that is to say “hot” points if the image processing is based on positive contrast or “cold” points if the image processing is based on positive contrast. on a negative contrast.

The first module 3 and the second module 4 are well known from the prior art and will not be further detailed here. We retain that at the input of the first module 3 and of the second module 4 there is an initial image acquired by the optical sensor and at the output of the first module 3 we recover point source detections and at the output of the second module 4 elementary detections of wide sources.

The image processing device 1 comprises a third module 5 for merging the elementary detections of sources wide. This third module 5 is therefore arranged downstream of the second module 4 so as to have as input the elementary detections of large sources determined by the third module 4.

With reference to FIG. 3, the third module 5 comprises a first block 11 calculating from said elementary detections of wide sources, segmentation masks associated with each of said detections.

Preferably, the first block 11 keeps information transmitted by the second module 4 only the position of the detections of large sources (positions expressed in row-column).

Optionally, the first block 11 carries out the following loop.

Beginning of the loop.

For each of the elementary broad source detections provided by the second module 4 _r, the following different steps are implemented.

According to a first stage of a first step, the first block 11 determines an accumulated histogram on a portion of the initial image centered on the position of the elementary wide source detection considered. This portion of the initial image is of predetermined shape and dimensions identical to all the elementary detections of large sources. For example the portion of the image is a rectangle. FIG. 4 represents such an image portion. As for the cumulative histogram, it is a local representation of the distribution of the gray levels (also called “intensities”) of the pixels around the elementary detection position of the wide source considered. A cumulative histogram associated with an elementary wide source detection considered can therefore be totally different from that evaluated for another wide source detection which nevertheless belongs to the same large source.

An analysis is thus carried out during the first stage. local gray levels around each elementary detection of a considered wide source extracted from the initial image.

Then, during a second stage of the first step, the first block 11 determines a local detection threshold Thp based on the gray levels of the pixels of the portion of the image by thresholding the cumulative histogram by an intermediate threshold in percentage. Th_% of predetermined value.

Optionally, Th_% is chosen to have a value between 80 and 90%. For example we choose Th_% to 85%. This means that one seeks to keep only 15% of the pixels present in the portion of the image, independently of their positions in said portion. For each portion of the initial image associated with an elementary detection of a large source, the local detection threshold Thp depends on Th_% and on the cumulative histogram: said local detection threshold Thp will therefore depend directly on the local neighborhood of the detection. elementary wide source associated and will be different from one portion to another.

During a second step, the initial image is in turn thresholded in gray levels (while at the level of the first module 3 and of the second module 4 the initial image was thresholded in SNR) by the local detection threshold Thp.

An I2p binary image is thus obtained having one or more segmentations associated with the elementary detection of a wide source.

It is noted that the local level detection threshold Thp is specific to each elementary detection of a wide source so that the different binary images I2p each associated with one of the elementary detections of wide sources are different from each other.

A local segmentation is thus carried out around each of the elementary detections of large sources. after analysis of the distribution of the gray levels around said elementary detections of large sources. The segmentation carried out in the third module 5 is therefore finer than in the second module 4 thanks in particular to the use of the cumulative histogram. Optionally, the first block 11 expands and / or erodes the binary image I2p in order to: remove isolated segmentations, for example if said segmentations are too small and / or inconsistent in view of the other segmentations; and / or group together segmentations that are close to one another.

For this purpose, the first block is based on predetermined criteria to define a “small” segmentation, an “incoherent” segmentation, an “isolated” segmentation, “close” segmentations ... For example for “close” segmentations. , the first block can be based on a proximity criterion expressed in row-column distance.

A binary image I3p is thus obtained having a main segmentation (and preferably which is the only segmentation of said binary image I3p) associated with the elementary detection of a wide source.

Said main segmentation preferably includes elementary broad source detection. In addition, said main segmentation is chosen so as to be the largest segmentation in terms of area which includes elementary wide source detection.

Preferably, the first block 11 labels the binary image I3p to extract an area in the binary image I3p associated with the elementary detection of a wide source. Preferably, said zone is defined so as to contain the main segmentation without including other segmentations of the binary image I3p. Said zone is also called “segmentation mask” hereinafter.

Preferably, the segmentation mask follows the contours of the main segmentation and has an area greater than but as close as possible to said main segmentation.

The segmentation mask is therefore advantageously of a larger area than the elementary wide source detection while being associated with said wide source. Better segmentation is thus ensured with the third module 5 than at the end of the second module 4. An image (of the same dimensions as the initial image) comprising said segmentation mask can thus be generated. Such an image is denoted binary image I4p. The other areas of the binary image I3p are thus considered to be uninteresting, which is logical since the local detection threshold Thp does not make sense for these other areas, the local detection threshold Thp having been evaluated locally around the elementary detection of the wide source.

End of the loop

The output of the first block 11 thus provides a segmentation mask for each elementary detection of a wide source transmitted at the input of the first block 11.

Preferably, if the first block 11 cannot determine a segmentation mask for at least one of the elementary wide source detections, the first block 11 outputs a segmentation zone associated with said elementary wide source detection which has been determined by the second module 4. In this way, elementary detection of a wide source is not lost through the first block 11.

Optionally, if the main segmentation does not include the elementary broad source detection (for example because of erosion and / or dilation), an ancillary segmentation is sought in the vicinity of said elementary broad source detection, segmentation which would include said detection. The neighborhood is for example an area of 3 pixels by 3 pixels centered on the elementary detection wide source (or a central pixel of said elementary wide source detection). The corresponding segmentation mask will then be defined with respect to this annex segmentation (and not to the main segmentation). If no additional segmentation can be determined then, as indicated above, it is considered that the first block 11 cannot determine a segmentation mask for this elementary wide source detection and the first block 11 outputs an associated segmentation zone to said elementary detection of a large source which has been determined by the second module 4. The third module 5 also comprises a second block 12 arranged downstream of the first block

11 and receiving therefrom the segmentation masks and / or the I4p images (and possibly segmentation zones associated with one or more elementary detections of large sources, elementary zones which have been determined by the second module 4 and / or images comprising such areas) to output generalized detections of wide sources.

For this purpose, and according to a first step, the second block

12 creates a binary image 14 which is in reality the union of all the binary images I4p (and optionally of the segmentation zones calculated by the second module 4 and / or of the images comprising such zones).

For example, binary image 14 is created as follows:

- for each pixel belonging to a segmentation mask in an I4p image (and optionally to a segmentation area calculated by the second module 12), the second block sets to 1 the value of the pixel having the same row-column position in the binary image 14;

- if several segmentation masks (and optionally a segmentation zone calculated by the second module 12) include the same pixel, the value of the corresponding pixel in the binary image 14 is maintained at 1; - otherwise the value of the pixel of binary image 14 is taken equal to 0.

An example of binary image 14 is visible in FIG. 6. According to a second step, second block 12 labels binary image 14 to extract segmentation regions in binary image 14 which are independent from one another and to associate in Consequently, the segmentation masks which overlap, the segmentation masks overlapping thus jointly forming the same segmentation region. The second block 12 thus constitutes one (or more) segmentation region (s) grouping (each) the segmentation masks associated with the different elementary detections of the same broad source.

An example of labeled binary image 14 is visible in Figure 7.

We retain that each segmentation region is defined so as to include:

- either a single segmentation mask,

- or several segmentation masks which overlap with each other.

Preferably, the segmentation region follows the contours of the associated segmentation mask (s) and has an area greater than but as close as possible to said mask (or to the overlap of said masks). Furthermore, it is thus possible to define a generalized detection of the broad source associated with each segmentation region. This definition is made here during a third level. Optionally, the second block 12 considers that the position of said general detection is the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level. The second block can use the following formula for this:

with

X = the number of the considered column (or row) of the segmentation region, i = a pixel of the segmentation region, level = the gray level of the pixel considered, and

Mask = the set of pixels constituting said segmentation region.

Thus, the second block 12 provides at its output (which also corresponds here to that of the third module 5) the generalized detections of large sources and in particular their positions.

These generalized detections of large sources are thus defined so as to group together elementary detections of large sources associated with the same large source by the second module.

Advantageously, these generalized detections of large sources are also more refined than the elementary detections of large sources with regard to their morphological characteristics (position, contour, etc.)

With reference again to FIG. 2, the image processing device 1 comprises a fourth module 6 for merging the generalized detections of wide sources generated by the third module 5 with the detections of point sources generated by the first module 3. This fourth module 6 is therefore arranged downstream of the first module 3 and of the third module 5.

The fourth module 6 operates as a wide source - point source merging module of the prior art (with the difference that the input data of the fourth module 6 will therefore be the generalized detections of wide sources coming from the third module 5 and no longer the elementary detections of large sources generated by the second module 4) so that it will not be detailed here.

At the output of the fourth module 6 (which also corresponds here to the output of the image processing device 1), the targets are recovered which will then be processed by the rest of the system.

An image processing device 1 has thus been proposed comprising two modules 4, 5 dedicated to the detection and segmentation of wide sources. The second module 4 allows the detection and the segmentation of the points of interest on which to focus then the third module 5 performs a local analysis of cumulative histogram around all the points of interest provided by the second module 4. The third module 5 consequently makes it possible to adapt the local detection threshold for each of the points of interest and to obtain segmentation masks associated with said points of interest (more finely segmented than if one were satisfied to have recourse to the second module 4), masks which are then merged by proximity criterion, in order to develop a single detection by wide source.

More generally, an image processing device 1 has thus been proposed which is based on a segmentation of large sources controlled by local multi-statistical criteria with the aim of associating them with a single detection.

Advantageously, the second module 4 and the third module 5 are relatively independent.

Thus the second module 4 can be relatively rustic and / or an already existing module of the prior art.

The use of duplicate second module 4 - third module

5 therefore advantageously avoids the third module 5 having to resort to a complex process of segmentation around the wide source since the third module 5 takes as input directly the elementary detections of wide sources supplied by the second module 4 (which is a relatively rustic) which thus indicates to him the zones to be privileged for the segmentation.

Likewise, the first module 3 and the fourth module

6 can be already existing modules of the prior art. The third module 5 can thus easily be installed in existing image processing devices of the prior art. The third module 5 therefore improves the performance of the image processing device 1 and therefore of the system.

The image processing device 1 thus described improves the segmentation of large sources and their counting while maintaining the requirement for an acceptable false alarm rate (rate linked to the second module 4). Advantageously, the image processing device 1 thus described can even reduce this false alarm rate thanks to the groupings of the multiple detections on the sources (which can make it possible to rule out unwanted detections such as, for example, waves on the sources. the sea, edges of clouds ....).

Of course, the invention is not limited to the implementation described and it is possible to provide variant embodiments without departing from the scope of the invention.

In particular, the invention is applicable to many applications other than the military maritime field. The invention can thus be applied in the civil field, in the military field, in the land field, in the aeronautical field, in the airborne field, in the field of transport in general ...

It is also possible, in addition or as a replacement, to work in bands of wavelengths other than infrared and for example in the visible.

In the same way, different types of optical sensors can be used: camera (infrared, visible, etc.); radar (synthetic aperture radar usually designated by the acronym SAR, side-opening radar, etc.); ...

The third module may be different from what has been indicated and for example fulfill additional functions. Thus if a segmented region and / or a segmentation mask is inconsistent (for example in terms of morphology), the third module may decide to associate the elementary detection (s) of large sources with the characteristics (in particular the position) provided by the second module. Detection of inconsistency can for example be based on one or more of the following parameters: obtained number of pixels in the segmented region and / or the segmentation mask on the total number of pixels, maximum sizes allowed in predetermined width and height in the segmented region and / or the segmentation mask, width to height ratio in the segmented region and / or the segmentation mask ...

Claims

1. Method for generating at least one generalized detection of a large source present on an initial image comprising the successive steps of:

- determining for each segmentation region generated a generalized detection.

2. Method according to claim 1, in which a cumulative histogram is determined on a portion of the initial image centered on the position of the elementary wide source detection considered.

3. Method according to claim 2, in which a local detection threshold is determined based on the gray levels of the pixels of said portion by thresholding the cumulative histogram by an intermediate threshold of predetermined value.

4. Method according to claim 3, in which the intermediate threshold has a value of between 80 and 90%.

5. Method according to one of claims 3 to 4, in which the initial image is thresholded in gray levels by the local detection threshold in order to obtain a first image exhibiting one or more segmentations associated with the elementary source detection. large.

6. Method according to claim 5, in which the first image is dilated and / or eroded to obtain a second image.

7. Method according to claim 5 or claim 6, in which the first image or the second image is labeled in order to extract the at least one segmentation mask.

8. Method according to one of claims 1 to 7, in which a new image is created including the at least one segmentation mask and the said image is labeled in order to extract the at least one segmentation region.

9. Method according to one of claims 1 to 8, in which a position of the at least one generalized detection of the wide source associated with the at least one segmentation region is calculated.

10. Method according to claim 9, in which the position of the at least one generalized detection of the wide source is considered to be the barycenter of all the pixels constituting the associated segmentation region, weighted by their gray level.

11. Image processing device making it possible to implement the method according to one of claims 1 to 10, the device comprising a first module (3) for point source detection, a second module (4) for elementary detection of wide sources, a third module (5) for merging elementary detections of large sources generated by the second module and a fourth module (6) for merging generalized detections of large sources generated by the third module with the detections of point sources generated by the first module.

12. An optronic system comprising a device according to claim 11.

13. Marine vehicle comprising an optronic system according to claim 12.