WO2007138025A1

WO2007138025A1 - Electro-optical device for counting persons, or other, based on processing three-dimensional images, and relative method

Info

Publication number: WO2007138025A1
Application number: PCT/EP2007/055136
Authority: WO
Inventors: Bruno Crespi
Original assignee: Neuricam Spa
Priority date: 2006-05-30
Filing date: 2007-05-28
Publication date: 2007-12-06
Also published as: ITUD20060138A1

Abstract

Electro-optical device (10) for counting persons or other, comprising at least a unit for the acquisition (11) of three-dimensional images by means of the indirect measurement of the time of flight of light impulses emitted towards the persons, or other, to be counted, at least a unit for the emission (12) of the light impulses, and at least a processing unit (13) able to process the three-dimensional images detected by the acquisition unit (11), to count the persons present in the field of vision of the acquisition unit (11) and to determine the directions of the streams of movement of the persons.

Description

"ELECTRO-OPTICAL DEVICE FOR COUNTING PERSONS, OR OTHER, BASED ON PROCESSING THREE-DIMENSIONAL IMAGES, AND

RELATIVE METHOD"

* * * * * FIELD OF THE INVENTION

The present invention concerns an integrated electro-optical device for counting persons, objects or bodies in movement, crossing a passage or standing in a delimited zone.

The counting device according to the invention is suitable to reconstruct the three-dimensional profile of an area subject to visual control by means of artificial vision techniques. The persons, or other moving bodies, for example animals or objects, are counted by separating them from the background, by discriminating the three-dimensional form of the bodies from the profiles generated by objects or structures, and by reconstructing the trajectory of the movement of the bodies inside the detection zone. The invention is applied preferably in the field of public transport (buses, trains, subways) or in the control of areas or buildings that are open to the public where it is necessary to count the number of persons present and/or who transit through a passage. The invention also concerns the counting method performed by said device. BACKGROUND OF THE INVENTION

Devices are known for counting persons transiting through passages, which are normally achieved by means of photocells, pyroelectric sensors, ultrasounds or combinations thereof. However, these devices have a considerable disadvantage: given the point-by point nature of the detection, they do not allow to determine the shape of the object detected and hence to discriminate one object from another. Consequently, the performance and reliability of these systems are compromised in crowded situations, in which two or more persons in close contact enter and/or exit through a passage, or by the presence of bulky objects such as suitcases, rucksacks, trolleys, umbrellas, etc. Moreover, systems consisting of multiple point-by-point sensors have considerable problems of calibration and pointing, which determine high installation and maintenance costs.

Surveillance systems are also known, based on one or more TV cameras. Systems based only on the detection of movement are affected by errors caused by moving shadows or reflections, or by sudden changes in the light conditions. Moreover, such systems are not able to detect the presence of static objects. Recognition systems based only on two-dimensional analysis of the images are not yet able to discriminate a given object from the background in typical real situations. This is because the variation in the possible two-dimensional forms resulting from different perspectives or positions of the object, in particular when there are several moving objects, makes it extremely complex to analyze the scene. US-A-6.323.942 discloses a vision system comprising a detection device which, by measuring the time of flight of a determinate signal emitted towards the objects to be identified, detects the position of said object with respect to the point of emission of the signal.

This type of device is able to acquire some information concerning the characteristics of the object visualized, but it is not able to follow its movement and direction, that is, the trajectory of travel, in the zone of interest.

Purpose of the present invention is to achieve an electro-optical device for counting persons, or other moving forms, which overcomes the limits and problems of devices currently available. In particular, purpose of the invention is to achieve a counting device able to offer good reliability and accuracy, which allows to follow the movement and direction of determinate objects in a specific zone of interest, which has limited sizes, which is easy to install and configure, and which can be connected to external control systems by means of digital protocols. The Applicant has devised, tested and embodied the present invention to obtain these and other purposes and advantages.

SUMMARY OF THE INVENTION

The present invention is set forth and characterized in the respective main claims, while the dependent claims describe other innovative characteristics of the invention.

In accordance with these purposes, the electro-optical device according to the present invention comprises at least the following components: a unit to acquire three-dimensional images by means of indirect measurement of time of flight; a lighting unit with monochromatic light impulses, emitted for example by LED or laser; a processing unit dedicated to processing the temporal flow of the three- dimensional images; and a unit suitable to enable communication between the device and the outside. These main components are suitably inter-connected and each has specific characteristics and functions.

According to a first characteristic of the invention, the image acquisition unit comprises at least an optical sensor, consisting of a matrix of photo-sensitive elements, able to convert the luminous radiation of the image detected into an electric signal and able to estimate the distance of the objects illuminated by light impulses by means of the indirect measurement of the time of flight of the light reflected by said objects. The image acquisition unit also comprises an illuminator device which allows to emit, inside a suitable solid angle, light impulses synchronized with the optical sensor, and an optical system able to project onto the optical sensor the image of the same zone of space illuminated by the illuminator device.

According to another characteristic of the invention, the processing unit is configured to perform the following operations: discrimination of persons present in the field of vision of the sensors, counting thereof, and possible verification of their trajectory f passage through pre-determined thresholds.

In another form of embodiment of the present invention, the device comprises a neural network, implemented in software or hardware mode, able to classify the three-dimensional profiles obtained by processing the images. The use of the neural network allows to learn, for example, the three-dimensional profiles corresponding to persons filmed from a determinate perspective and to discriminate these profiles from those of objects or other bodies. The capacity of recognizing and classifying three-dimensional forms, thanks to the use of the neural network, allows to reduce counting errors due to the simultaneous presence of extraneous persons and objects in the control zone.

In a first form of embodiment, the device according to the invention is implemented by means of the combination of discreet components, that is, the sensors, the processing unit and the external connection unit are independent parts mounted on cards and made to communicate with each other by means of suitable electric connections.

In an alternative form of embodiment, the different units which make up the system are integrated on a single silicon chip, that is, a System on Chip, using VLSI technology. This technology of extreme integration allows to considerably reduce sizes, costs and consumption.

Even if the present invention has been described until now as an independent unit, it is clear that various devices according to the invention can be inter- connected through a line of digital communication so as to produce a network of sensors that covers several passages. In this way, the present invention can be used to control the stream of persons through areas or buildings with different access passages.

Similarly, according to another evolution, the device according to the invention can be coupled with biometric security control devices such as devices that recognize the face, voice, fingerprints and/or the iris.

Even though the present invention was devised specifically for counting persons, it comes within the field of the invention to provide the presence of a trainable recognition system which allows to configure the device for counting different classes of objects.

Even though the present invention was devised specifically for counting persons, it comes within the field of the invention to provide the possibility to configure the system to detect the presence of extraneous objects in the access zone and/or to signal movements or behavior not in conformity with the security norms of the persons present there.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other characteristics of the present invention will become apparent from the following description of two preferential forms of embodiment, given as a non-restrictive example with reference to the attached drawings wherein: - figs. Ia-Ib show, respectively with a front view and from above, a device according to the invention in an assembly position;

- fig. 2 shows a block diagram of the device according to the present invention in a first form of preferential embodiment; - fig. 3 shows the flow chart describing the processing of the time sequence of the images with the purpose of counting persons.

DETAILED DESCRIPTION OF SOME PREFERENTIAL

FORMS OF EMBODIMENT In fig. 1 , the device 10 is shown assembled above a passage, denoted in its entirety by the reference number 20, so that the movement in and out of a person 21 passing through it is ideally perpendicular to the line that joins the two perspectives and so that the plane of viewing is parallel to the floor.

In the configuration step for counting persons, the neural network of the device 10 is trained to recognize the three-dimensional form of a person seen from above.

With reference to fig. 2, the device 10 according to the present invention comprises, as essential parts, a unit to acquire three-dimensional images 1 1, a light impulse lighting unit 12, a unit to control the synchronism between the image acquisition and light impulse generation 13, a processing/calculating unit 14 to recognize the three-dimensional forms based on neural networks and to calculate the trajectory of movement of the persons, and a communication interface 15.

The image acquisition unit 11 is formed in this case by a CMOS optical sensor 16 able to execute acquisitions synchronized with the emission of light impulses and to estimate the distance of the object that reflects the light impulse as a function of the quantity of light received. Each optical sensor 16 consists of a matrix of 50x30 active pixels, and every pixel is able to independently execute the calculation of the distance. The images are digitalized inside the optical sensor 16 by means of an analog-digital converter. The parameters of regulation of the optical sensor 16 are controlled by a control mechanism that allows to optimize the quality of the images as the reflectivity and the distance of the objects present in the field of vision of the sensor vary.

The lighting unit 12 is dedicated to the emission of light impulses with a frequency and a duration that allow to detect the radiation reflected by objects that are at a maximum distance of about ten meters from the sensor and in a zone of space subtended by a solid angle of suitable amplitude.

The radiation emitted by the illuminator, laser or LED, is in the spectrum of near infrared (from about 700 nm to 1 100 nm) and supplies uniform lighting to the passage and ensures a correct and stable functioning of the device 10 even in conditions of poor or zero lighting.

The regulation of the parameters of the optical sensor 16 stabilizes in real time the response of the optical sensors 16 to the variations in reflectivity and distance of the objects present in their field of vision.

The calculation of the distance map, that is, of the image in which with every pixel a number is associated that indicates the distance traveled by the light reflected by an object and detected by said pixel, is carried out by measuring the ratio between light emitted and light reflected and detected by the sensor in time windows suitably chosen and synchronized with the emission of the light impulses.

The processing unit 14 comprises a processor with a RAM type memory, or

CPU. The processing unit 14 is able to effect a so-called high-level or cognitive processing of the images, and comprises the recognition of three-dimensional forms, identification of the trajectory of movement thereof, counting, temporary memorization of the results and communication with the outside.

The present invention uses neural networks to identify the shape of a person so as to discriminate it from other objects. In fact, the great variability of possible poses of the human body with respect to the TV camera makes the recognition procedure very difficult to describe in mathematical and/or geometrical terms.

On the contrary, by using neural techniques, it is sufficient to train the network with a sufficient number of typical examples, without having to have recourse to any specific algorithm. The communication unit 15 preferentially consists of an Ethernet channel at 100 Mbit/sec, a USB 2.0 channel, an RS485 serial and four opto-isolated ports for communication with industrial devices.

With reference to fig. 3, the device 10 according to the present invention comprises, as an integral part, a program to achieve, on the electronic architecture previously described, the function of person counter. The flow chart of the program provides the following method steps:

- acquisition of three-dimensional images;

- calculation of the average intensity of the images and regulation of the parameters of the optical sensors 16 in order to obtain images of constant quality;

- identification of the presence of shapes of persons by means of neural processing techniques; - determination of the trajectories of the persons present in the passage zone, that is, the temporal evolution of the movement of the person;

- counting the persons that transit through the passage;

- temporary memorization and communication to the outside of the results of the count; and - reading of messages arriving from outside.

In an alternative preferential embodiment, the hardware architecture of the device 10 is integrated in a System on Chip of the VISOC type (Vision System On Chip, for example as described in EP-A- 1.353.498 in the name of the present Applicant). This micro-electronic device consists of various blocks integrated on silicon, suitably connected with each other and each with a specific function. In particular, the device comprises:

- an optical sensor 16 with ITOF technology consisting of a matrix of photosensitive elements with active pixel, which provide to convert the light radiation into an electric signal, and calculation elements which allow to estimate the distance of the reflected light as a function of the optical energy detected at suitably chosen time intervals;

- an analogical-digital converter;

- a sequential microprocessor of the Von Neumann type; - a parallel processor of the neural type;

- a volatile memory able to memorize data and programs in execution (RAM);

- a non-volatile memory able to memorize programs and regulation and calibration parameters (FLASH);

- an interface for communication of the device with other external devices. The VISOC device is coupled with an illuminator, consisting of laser or LED, with a wavelength in the portion of the spectrum of the near infrared (from 700 nm to 1100 nm) which emits light impulses synchronizable with the acquisition of the images of the sensor. The CMOS optical sensor 16 of the electronic device is coupled with an optical system which focuses the radiation emitted by the illuminator and reflected by objects present in the field of vision. The optical device can consist of lenses and/or prisms and/or mirrors and/or optical fibers. The VISOC device is programmable by means of high-level languages, for example C/C++, Basic.

In this case, the VISOC device is programmed following the flow of operations required to count the persons.

The VISOC device is coupled with a set of high luminosity laser modules or LED, with a wave length in the portion of the spectrum of the near infrared (from 700 nm to 1 100 nm) which ensures a correct and stable functioning of the device even in conditions of poor or zero lighting.

Modifications and variants may be made to the device and method for counting persons based on three-dimensional vision by means of time of flight as described heretofore, without departing from the field of the invention as defined by the attached claims.

Claims

1. Electro-optical device for counting persons or other, characterized in that it comprises at least a unit for the acquisition (1 1) of three-dimensional images by means of the indirect measurement of the time of flight of light impulses emitted towards said persons, or other, to be counted, at least a unit for the emission (12) of said light impulses, and at least a processing unit (13) able to process said three-dimensional images detected by said acquisition unit (1 1), in order to count the persons present in the field of vision of said acquisition unit (11) and to determine the directions of the streams of movement of said persons.

2. Device as in claim 1, characterized in that said acquisition unit (11) of three- dimensional images comprises at least an optical sensor (16) able to convert the luminous radiation of the image detected into an electric signal, and an optical system able to project onto the at least one optical sensor (16) the image of the same zone of space illuminated by said light impulses, so as to be able to extract from said three-dimensional images the information on the three-dimensional profile of the objects present.

3. Device as in claim 2, characterized in that said optical sensor (16) consists of a matrix of photo-sensitive elements.

4. Device as in claim 1, characterized in that said processing unit (13) comprises a neural network able to discriminate and classify the three-dimensional profiles obtained by processing said three-dimensional images.

5. Device as in claim 4, characterized in that said neural network is implemented in software mode.

6. Device as in claim 4, characterized in that said neural network is implemented in hardware mode.

7. Device as in claim 1, characterized in that it comprises at least a unit (15) suitable for communication between the device and the outside.

8. Device as in any claim from 2 to 7, characterized in that said at least one optical sensor (16), said processing unit (13) and said communication unit (15) with the outside are integrated on a single silicon support made using CMOS technology.

9. Device as in any claim hereinbefore, characterized in that it is inter-connected with a plurality of devices of the same type so as to allow to control the stream of persons through a mating plurality of passages (20).

10. Device as in any claim hereinbefore, characterized in that it is able to be associated with means for the recognition of the person obtained by means of techniques to identify the face of the person, and/or the voice of the person.

1 1. Method to count persons using the device as in any claim hereinbefore, characterized in that it comprises at least a first step to acquire a three- dimensional image of objects present in a detection area (20) by means of the indirect measurement of the time of flight of determinate light impulses emitted towards said persons to be counted, and a second step to discriminate and count the persons present in said detection area, and to detect the directions of the streams of movement of said persons.

12. Method as in claim 11, characterized in that it comprises a step to verify the passage of said persons through determinate thresholds.

13. Method as in claim 11 or 12 characterized in that at least said step of discrimination and counting is carried out by a neural network.