EP1149310A1

EP1149310A1 - Method and device for detecting objects

Info

Publication number: EP1149310A1
Application number: EP99963806A
Authority: EP
Inventors: Per Börjesson
Original assignee: Saab Dynamics AB
Current assignee: Saab AB
Priority date: 1998-12-11
Filing date: 1999-12-07
Publication date: 2001-10-31
Also published as: WO2000039607A1; SE9804285L; SE9804285D0; SE514646C2

Abstract

The invention presented here relates to a method for detecting the presence of objects by means of an optical sensor in a scene within the sensor's field of view. The sensor is made record images of the scene so that each individual image element is assigned a value corresponding to the detected heat radiation within the infrared wavelength range. The method is characterised in that the sensor records (1, 2) at least a pair of consecutive images of the scene, where one of the images in each pair is recorded (2) under illumination of the scene by means of a radiation source. Thereafter, a comparison image is created (4) such that each image element in the comparison image is assigned a value corresponding chiefly to a differential or quotient between the radiation values for corresponding image elements in the images recorded (2) under illumination, and the radiation values for corresponding image elements in the images recorded (1) without illumination. The image elements of the created comparison image are scanned, and the image element values that exceed a certain threshold level are marked (5) as an optical aperture within the infrared wavelength range. The invention also comprises a device for executing the method above.

Description

Method and device for detecting objects

The invention presented here relates to a method for detecting the presence of objects by means of an optical sensor in a scene within the sensor' s field of view, whereby the sensor records images of the scene so that each individual image element is assigned a value corresponding to the detected radiation within the infrared wavelength range.

The invention also relates to a device for the execution of the above method.

One purpose of the presented invention is to achieve a reliable method and device for discovering optical apertures within the infrared wavelength range. An additional purpose of the present invention is to allow a user to indicate, in a readable way, the position of the detected reflecting apertures.

This has been accomplished by means of a method, the introductory part of which is already known, characterised in that; using the sensor, at least a pair of consecutive images of the scene are recorded, where one of the images in each pair is recorded when the scene is illuminated by a radiation source; a comparison image is created so that each image element in the comparison image is assigned a value corresponding chiefly to a differential or quotient between the radiation values for corresponding image elements in the images recorded under illumination, and the radiation values for corresponding image elements in the images recorded without illumination, and that the image elements of the created comparison image are scanned, and the image element values that exceed a certain threshold level are marked as an optical aperture within the infrared wavelength range.

The preferred versions have one or more of the characteristics stated in the sub-claims 2-4. The above method can be realised by means of a device of the above mentioned type, which is characterised in that; the device comprises: a storage means connected to the sensor designed to store images recorded by the camera; a device designed to illuminate the scene by means of a radiation source, a laser for example ; control means designed to control the sensor and the illumination means so that the sensor records and stores two consecutive images of the scene at the same time as the illumination means i lluminates the scene on recording one of the i mages ; first image processing means designed to create, from the recorded i mages, a comparison i mage such that each image element in the comparison i mage i s assigned a value corresponding to the differential or the quotient between the radiation values of the corresponding image elements in both of the recorded images , and second image processing means designed to scan the image elements of the comparison image and mark the i mage element values that exceed a certain threshold value as a detected optical aperture within the infrared wavelength range

The preferred embodiments have one or more of the characteristics stated in the sub-claims 6- 10

The device and method in accordance with the invention have several advantages . It is possible, for example, to measure difficult to discover, passive optical apertures within a large area (IR camera beam width) with very good resolution. Present-day high-performance IR cameras can be used for the measurements and specially produced hardware is not required. For the detection, the laser should be reasonably powerful and when detecting at very long distances the beam width of the laser can be reduced in order to increase the range.

The invention will be described i n more detail below with the aid of examples of embodiments of the method and the device according to the present invention with reference to the accompanying drawings, wherein: Fig 1 shows a flow chart in accordance with an example of a method according to the present invention and

Fig 2 shows schematically an example of the construction of a device according to the present invention

The flow chart in Fig 1 describes in broad outline the complete procedure for detecting the presence of reflecting objects in the form of optical apertures or reflectors, chief ly within the infrared wavelength range In step 1 a first image of a scene i s recorded Recording is per- formed by an optical sensor, for example in the form of a sensitive IR camera, which in Fig 2 is denoted by reference number 7 The size of the scene corresponds in one example to the IR camera' s field of view During image recording the IR camera assigns each separate image element a value corresponding to the detected radiation, which in turn is proportional to the heat radiation emitted from the object in the scene For each individual image element there is a IR detector element ai ranged in such a way that, for a specific integration time, say 20 μm, the photons that hit the detector surface are transformed to electrons The value of each image element corresponds to the number of electrons and thus indirectly to the number of photons that hit the detector surface during the integration time The recorded first image is stored in a memory of the type read-write memory The memory is denoted by reference number 1 1 in Fig 2

In step 2 a second image of the scene is recorded in the same way On recording of the second image the IR camera 7 is positioned in the same place as in the first image recording It may be essential, depending on the characteristics of the scene, that the time interval between the first and the second image recording is short, typically a few tens ms During recording of the second image the whole scene is illuminated The illumination is such that its wavelength is within the optical range of the IR camera Illumination is provided by a radiation source, for example in the form of a laser, in front of which is placed a lens of a type designed to disperse the laser beam over the whole scene. In another example, the laser would sweep across the scene during the recording. In Fig. 2 the laser is denoted by reference number 8. The laser need only be moderately powerful. To achieve adequate illumination in the case where the depth of the scene is great, one can, instead of increasing the laser power, choose the scene so that it is narrower than the field of view of the camera, thus allowing the spread of the laser beam to be restricted. The second recorded image is also stored in the read-write memory 1 1 . In another embodiment the first image is recorded under illumination by means of laser and the second without illumination. The method can be repeated, whereby the detection capability increases according to known theories of signal processing for time integration etc.

As mentioned previously, the time interval between the first and the second image recording should be as short as possible. This is because it is desirable to avoid large movements in the scene between the two image recordings. Furthermore, the IR camera should be held motionless by, for example, being mounted on a stand, so as to avoid shaking and any consequent blurring and displacements in the image. Even if a short time interval is used and shaking is minimised, displacements in the image can still occur. With the aim of reducing the impact of these displacements in the image, in step 3 differences between the two images as a result of the displacements are detected and removed. In one example, the impact of the displacements could be reduced by investigating the correlation between the two images so as to detect the displacements, and based on the said detected displacements to update one of the images. In this way a simultaneous recording of both images is simulated.

With a view to filtering out any optical apertures present, in step 4 a discrepancy image for every image element is created between the recorded images, so that each image element in the discrepancy image is assigned a value corresponding to the difference between the radiation values of the equivalent image elements in both of the recorded images. The image element values of the image taken under illumination are generally higher than those for the non-illuminated image. However, at an optical aperture, as a result of light produced reflections, the image element values of the illuminated image will be considerably higher than those of the non-illuminated image. Because of this, the image element values will be significantly higher at optical apertures than at places where such do not occur. The anomalous values in the discrepancy image are either positive or negative depending on which image was subtracted from which.

In order to filter out the optical apertures, a quotient image can, in step 4, be created instead of the above described discrepancy image. The quotient image is created by forming a quotient between the two recorded images.

In step 5 the values in the discrepancy image or the quotient image over a certain threshold level are marked as optical apertures. This threshold level can be a predetermined value or an adaptive value, which is set, for example, according to the mean level in the discre- pancy image or quotient image. The image elements of the discrepancy image / quotient image are scanned in detail, whereupon the image elements exceeding the threshold level are marked as an optical aperture within the infrared wavelength range. In one embodiment, groups of marked image elements adjacent to each other are assumed to be one and the same optical aperture and are marked accordingly. In another example, all image elements with values below the threshold level are reset to zero and the remaining image element values are taken to constitute markings of detected optical apertures. In order to increase the reliability, the procedure involving steps 1 - 5 can be repeated to see if the marked optical apertures are also marked in the subsequent measurements. In another example, discrepancy or quotient images can be created for several measurements according to steps 1 -4. Thereafter, a final discrepancy or quotient image is created, in which, in one example, the image element values of the various discre- pancy or quotient images have been added for every image element, whereupon the image element values of the final discrepancy image greater than a second threshold value, based on the number of measurements, are marked as optical apertures. In another embodiment, the discrepancy / quotient image is created when a mean radiation value for each image element is formed for images recorded both with and without illumination, after which the discrepancy or quotient image is created either by subtracting the mean radiation values for each image element in both images, or by forming a quotient between these values. In the case described above, with repeated measurements, differences between the two images in each pair of images and differences between the pair of images can be removed in step 3.

Irrespective of how the optical apertures are marked out in step 5, in step 6 the markings of the optical apertures are shown on a presentation image. In a preferred embodiment, the presentation image comprises either one of the two recorded images, on which symbols have been placed at the places where the markings were made in the discrepancy / quotient image. Furthermore, the optical apertures can be indicated by showing the coordinates of their positions in relation to, for example, the position of the IR camera. In an example embodiment, these coordinates could be automatically sent to a main unit, which can be positioned elsewhere.

The described method can be implemented in a conventional, high- performance IR camera. As long as the camera has sufficient memory and programmable circuits, the only additional requirement, apart from the IR camera 7, is the laser 8 with some type of device for dispersing the laser beam so that the entire scene is illuminated. The laser in the example is a carbon dioxide laser. In an automatic device there is also a control device for controlling the IR camera and the laser.

With reference to Fig. 2, an example of a device according to the invention will now be described. The IR camera 7 is in communica- tion with the memory 1 1 which is designed to store images recorded by the camera as well as the discrepancy / quotient image. Preferably, the memory capacity is such that the recorded images and the discrepancy / quotient images from the successive measurements can be stored in order to render it possible for the method in step 5 to increase the reliability of the measurements. In an example embodiment, separate memories are used for the recorded images and for the discrepancy / quotient images, in which case the IR camera only needs to have communication with the memory for the recorded images. Moreover, the device comprises the laser 8 having the means for dispersing the laser beam over the entire scene. A control means 9 is designed to operate both the IR camera 7 and the laser 8, so that the IR camera automatically records and stores two consecutive images of the scene at the same time as the illumination means illuminates the scene for at least a part of the recording of one of the images. The control device 9 can be arranged to minimise the illumination time so that it is no longer than that required for the image recording. In this way the risk of detection is reduced.

The IR camera 7 is also in communication, via the control device, with a signal processor 10, in an embodiment designed to create, from the recorded images and on receiving a signal from the control device 9, a discrepancy image such that each image element in the discrepancy image is assigned a value corresponding to the difference between the radiation values of the corresponding image element in the two recorded images (step 4). In another embodiment, a quotient image is created from the recorded images, such that each image element in the quotient image is assigned a value equivalent to the quotient between the radiation values of the corresponding image element in both of the recorded images. The signal processor is also designed to scan the image elements of the discrepancy / quotient image, and, in the case of the discrepancy image, mark the image element values exceeding a predetermined absolute value as a detected optical aperture within the infrared wavelength range, as well as, in the case of the quotient image, if the illuminated image is a numerator and the non-illuminated image is a denominator, mark the image element values exceeding a certain quotient value and vice versa (step 5). In this case, the signal processor can be arranged to mark groups of adjacent marked image elements as one and the same optical aperture. Furthermore, the signal processor 10 is, in an example embodiment, designed to perform the resetting of the image element values to zero, which is described in connection with the method. The signal processor can also be designed to perform the detection of the displacements and the updating of one of the recorded images (step 3). In an example embodiment, the signal processor is designed to execute, on receiving a signal from the control device 9, the operations in sequence in accordance with steps 3-5.

A display 12, for example in the form of some type of viewing screen, is designed to show a presentation image consisting of one of the recorded images in the memory 1 1 , and the markings of the optical aperture superimposed on this image. In one embodiment, the signal processor 10, the memory 1 1 and the display 12 are integrated in the IR camera. The display 12 can also be designed to show the coordi- nates of the marked optical apertures, whereby the said coordinates can be calculated by the processor 10.

Claims

1. A method for detecting the presence of objects by means of an optical sensor (7) in a scene within the sensor's field of view, whereby the sensor (7) records images of the scene so that each individual image element is assigned a value corresponding to detected radiation within the infrared wavelength range, c h arac te ri ze d i n that:

the sensor (7) records (1, 2) at least a pair of consecutive images of the scene, where one of the images in each pair is recorded (2) at least partly under illumination of the scene by means of a radiation source,

a comparison image is created (4) such that each image element in the comparison image is assigned a value corresponding chiefly to a differential or quotient between the radiation values for corresponding image elements in the images recorded (2) under illumination, and the radiation values for corresponding image elements in the images recorded (1) without illumination, and

the image elements of the created comparison image are scanned, and the image element values that exceed a certain threshold level are marked (5) as an optical aperture within the infrared wavelength range.

2. A method according to claim 1, c h arac teri z e d i n that groups of marked adjacent image elements are marked as one and the same optical aperture.

3. A method according to any of the preceding claims, c h arac te ri z e d i n that before the comparison image is created, displacements in parts of the scene between the two recorded images in each pair are detected (5) by determining how the image element values of the two images are correlated, whereupon one of the images is updated so as to remove the differences between the recorded images due to the displacements.

4. A method according to any of the preceding claims, c h a r a c t e r i z e d i n that the marked optical apertures are indicated (6) on a displayed presentation image of the scene.

5. A device in an optical sensor (7) for detecting the presence of objects in a scene within the sensor' s field of view, whereby the sensor (7) operates to record images of the scene so that each image element is assigned a value corresponding to detected radiation within the infrared wavelength range, c h a r a c t e r i z e d i n that said device comprises:

storage means ( 1 1 ), connected to the sensor (7) and arranged to store at least the images recorded by the sensor,

means (8) arranged to illuminate the scene with a radiation source, for example in the form of a laser,

control means (9) arranged to operate the sensor and the illumination means, so that the sensor records and stores two consecu- tive images of the scene at the same time as the illumination means illuminates the scene on recording one of the images,

first image processing means ( 10) arranged to create a comparison image from the recorded images such that each image ele- ment in the comparison image is assigned a value corresponding to a differential or quotient between the radiation values for corresponding image elements in both of the recorded images and second image processing means (10) arranged to scan the image elements of the comparison image, and mark the image element values that exceed a certain threshold level as an optical aperture within the infrared wavelength range.

6. A device according to claim 5, c h arac te ri z e d i n that said first image processing means (10) are arranged to reset the unmarked image elements in the comparison image to zero.

7. A device according to any of claims 5 to 6, c h arac te ri zed i n that said first image processing means (10) are arranged to detect, before the comparison image is created by means of a correlation procedure, displacements in the scene between the two recorded images and update one of the images so as to remove the differences between the recorded images due to the displacements.

8. A device according to any of the claims 5 to 7, c h arac te r i z e d i n that said second image processing means (10) are arranged to create from any of the recorded images a presentation image, in which the detected optical apertures are indicated.

9. A device according to any of the claims 5 to 8, c h a rac te r i z e d i n that the illumination means (8) comprise a laser mounted on the sensor and equipped with means for dispersing the laser light over the entire scene.

10. A device according to any of the claims 5 to 9, comprising means for presenting the coordinates of the detected apertures, for example in relation to the position of the sensor (7).