EP1891577A1

EP1891577A1 - Method and system for counting passers-by passing an object

Info

Publication number: EP1891577A1
Application number: EP06757803A
Authority: EP
Inventors: Stefan Douwe Constantijn Molenaar; Johannes Gerardus Maria Schavemaker; Jan Baan
Original assignee: Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek TNO
Current assignee: Nederlandse Organisatie voor Toegepast Natuurwetenschappelijk Onderzoek TNO
Priority date: 2005-06-17
Filing date: 2006-06-19
Publication date: 2008-02-27
Also published as: WO2006135235A1; NL1029275C2

Abstract

The invention concerns a method and systeem counting the number of passers-by passing an object (1, 31) with the help of an optical detection system (2), e.g. a camera system (3, 34), which determines a detection area (6). The method comprises the steps of the detection of one or more objects (4) by the detection system within the detection area, which detection system generates a representation code and the counting of the number of detected objects in the representation code for the generation of the number of passers-by. The object may be an advertisement object and the counted number of passers-by a measure for the object's advertising value or effectiveness.

Description

Title: Method and system for counting passers-by passing an object

Field

The invention concerns a method and system for counting the number of passers-by passing an object, for example - but certainly not restricted to that - an advertisement, billboard or something like that.

Background

There are several application areas for a method and system for counting the number of passers-by passing an object. One application area is pointed to the need to determinate the "advertising value" for outdoor advertising, e.g. by means of billboards etc. However there are many more areas in which a method and system for counting the number of passers- by passing an object may be applicable, e.g. in the area of monitoring people flows e.g. during events (crowd control, interesting for the civil government and event organizers), monitoring how many people pass a certain premises (interesting for real estate brokers and developers) or monitoring the drift of animals (interesting for biologists) etc. So, although hereinafter there will — as an example - be a focus on the monitoring of public advertisements, it is emphazised that the field of the invention is, in general, the determination of the number of passers-by (people or even animals) passing an object.

Below the issue will be discussed using the example of advertisements and the determination of their effectiveness or their "reach".

Nowadays people are confronted with advertisements via radio, television, newspapers and magazines many times a day. Also outdoor advertising on billboards, shops and on advertising boards in e.g. bus shelters and scaffoldings etc. occur more and more often. An important factor for the determination of the fee for advertising space is the reach of the advertisement. Therefore both the providers of advertising space and the advertisers will appreciate when accurate information about the number of people that (may) see an advertisement (the reach of the advertisement) is acquired.

For radio and television use may be made of panels of which the listening or watching behaviour respectively can be recorded automatically. The fee for advertisements in newspapers and magazines mostly is determined by their circulation figures.

The determination of the advertising value for outdoor advertising is more difficult. Usually the number of persons passing a particular location is counted manually (e.g. by students) at random during a limited period. Subsequently, the counting results are extrapolated, however, without taking into account factors as weather conditions, holidays, extra opening hours, special events etc. Such a way of determining the attention value of an advertisement thus is inaccurate which, in consequence, results in an inaccurate calaculation of the fee an advertiser has to pay for.

Summary

It is a objective of the invention to provide a method for a more accurate determination of passers-by that e.g. may see an advertisement.

This objective is realized by providing a method for the counting the number of passers-by passing an object with the help of an optical detection system, e.g. a camera (system) which is arranged to cover a certain detection area, comprising the steps of: a) detection of one or more objects by the detection system within the detection area, which detection system generates a representation code, and b) counting the number of detected objects in the representation code.

The continuous presence of the detection system makes inaccurate extrapolation of random manual counts superfluous, whereas modern camera techniques make it possible to count persons with a great accuracy. Also when large groups of people pass more or less at the same moment the detection area of the detection system, as for example in a shopping street. Consequently, accurate number information can be obtained about the number of passing persons as a result of which e.g. the reach of an advertisement can be determined. Also, circumstances that may influence the number of passers-by may be detected.

In a preferred embodiment of the invention the detection system is arranged in such way that detection of objects occurs by at least one camera at an angle: "skimming". Such a skimming orientation of the camera enables counting of larger groups of people in e.g. a shopping street.

In a preferred embodiment of the invention, preceding to step b) at least one image (or picture) interpretation step is carried out at the representation code which is arranged to distinct objects. Preferably use is made of predefined object models. Such an image interpretation step increases the accuracy of the number information.

In a preferred embodiment of the invention the representation code of the camera is removed or not saved after the number information has been generated. For the determination of numeric information it is after all not necessary that the images with recognizable objects, as people, are saved. This method thus prevents invasion of the privacy of passers-by in the detection area and with that it meets the need for privacy of the public and the relevant legislation.

In a preferred embodiment of the invention the detection system is connected to a network and the number information is collected via the network. In this way the number information for the determination of- in the exemplary case - the reach of an advertisement can simply be gathered. Obviously it is also possible that the detection system comprises a, preferably releasable, module in which the number information is saved. This module can be read out on site from the detection system or be taken along to be read out at a later moment. A counting system can also be shared by a number of detection systems.

The invention also concerns a systeem for counting the number of passers- by passing an object, comprising at least one camera positioned under an angle with a vertical for the detection of objects within a cover reach of the camera and a counting system arranged for counting the number of detected objects, e.g. forming the basis for the object's advertising value.

The continuous presence of the systeem prevents the need for (inaccurate) extrapolation of random manual counts, whereas modern camera techniques and the skimming orientation of the camera make it possible to count with great accuracy persons, also when large groups of people pass at the same time the detection area of the detection system, as for example in a shopping street. Consequently, accurate number information can be obtained about the number of persons that observe the advertisement and thus e.g. the advertising value of the advertisement can be calculated.

In a preferred embodiment according to the invention the systeem comprises at least one first camera with a first detection area and a second camera with a second detection area, the first detection area and the second detection area overlapping each other at least partially, further comprising a data processing unit for the processing of the representation codes of the first and second camera for the counting the number of objects. The application of two camera's increases the accuracy of the number information, particularly on sites where large groups of people pass the detection area. Use can be made for example of two stereo camera's or of a conventional camera combined with an infrared camera.

It should be noticed that counting of persons by means of a camera is known as such for example from WO 01/52545. However, in the prior art no suggestion is given for applying a systeem for counting the number of passers-by which pass an object like an advertisement, billboard or something like that. The invention will be discussed hereinafter supported by some figures, which show preferred embodiments according to the invention. Obviously, the invention is not in any way restricted by these specific and preferred embodiments.

Exemplary Embodiments

Figs. IA and IB show schematically a camera configuration at an advertisement for the performing of the method according to an embodiment of the invention.

Figure 2 shows a flow diagram that illustrates an embodiment according to the invention of a method for the determination of the advertising value of an advertisement.

Figs. 3A-3C show schematically an example of an image interpretation step from Figure 2 for the distinction of detected objects.

Figs. 4A-4D show different embodiments for the collection of the number information.

Figure 5 shows a billboard — camara configuration with passing people.

Figure 6 gives an illustration of a preferred way of processing the camera images in order to count the passing people.

Figs. IA and IB show a combination of an advertisement 1, e.g. an advertisement on a scaffolding banner, and an optical detection system 2 comprising camera's 3 for the detection of objects 4 and a counting system 5. The objects 4 are e.g. people or vehicles that pass the advertisement. The counting system 5 is communicatively connected to the camera's 3, the counting of the number of by the camera or camera's detected objects 4 for the generation of number information for the determination of the advertising value of the advertisement 1.

The camera's 3 are adjusted in such a way that a detection area 6 is covered from which the advertisement is clearly visible. In Figs. IA and IB the detection area 6 is three-dimensionally positioned in front of the advertisement 1, so that the camera's 3 are positioned in skimming way, i.e. not perpendicular downwards. It may be clear to the expert that a same detection area 6 can be realized by another positioning of the camera's 3, e.g. straight above the detection area 6. The detection area 6 in Figure 1 of both camera's converges entirely.

The camera's 3 can e.g. be two stereo camera's or a conventional camera combined with an infrared camera. The type of camera's 3 and/or the camera configuration is e.g. dependent on the type of location, the pixel resolution necessary for an accurate counting of the objects 2. Stereovision is obtained by applying several camera's 3 or a stereo camera 3. In the case of several camera's the counting system can be equipped with a processing unit 7 for the combination of the video signals of the camera's.

Figure 2 shows a flow diagram that illustrates an embodiment according to the invention of a method for the determination of the advertising value of the advertisement 1. In step 10 the passers-by 4 in the detection area 6 are detected by the camera or camera's 3. The representation codes (video signals) get then, if necessary, digitalized in step 11. Then in step 12 an image interpretation step is carried out at the representation code for the distinction of objects 4, as will be amplified further hereinafter with reference to Figs. 3A-3D. The modified representation code after all is supplied in step 13 to the counting system 5 for the counting of the number of detected objects 4 for

the generation of number information for the determination of the advertising value in step 14. It should be noticed that the advertising value can simply be the number of counted objects 4 (the number information corresponds to the advertising value) or a value in which the number information is considered.

The representation code is removed or not saved after the number information has been obtained. For the determination of the advertising value of the advertisement it is after all not necessary that the images with recognizable objects 4, as people, get saved. This method thus prevents invasion of the privacy of passers-by of the detection area and thus meets the need for privacy of people.

The accuracy of the counting of the objects 4 is of importance in order to come to a reliable advertising value of the advertisement 1. Various techniques exist for the accurate counting of objects 4 by means of camera's 3. It is e.g. possible to group changing pixels, indicate for movement of an object 4. Each group of changing pixels represents one object 4. Usually, for that purpose a shot of the detection area 5 is made first, which shot is used as reference.

Objects 4 can however be positioned or seem to be positioned close to each other as a result of angle a that the camera 3 makes with the vertical V, as shown in Figure 3A. 3A. In such a situation one single groups of pixels can represent several objects 4 which will result in an incorrect counting of the number of objects 4.

More advanced counting systems at which camera's are applied are described in Tracking Multiple Humans in Complex Situations, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 26, No. 9, September 2004 of T. Zhao and R. Nevazia and Evaluation of People Counting Systems of T. Schlδgl, B. Wachmann, W. Kropatsch, and H.

Bischof. The contents of these publications are esteemed to be included in the current request. It is preferable in an embodiment to make use at the image interpretation step of beforehand defined object models M, as shown in Figure 3B. Various sizes of the object model M are shown.

At the image interpretation step 12 use can be made of different image characteristics in the video signal of the camera's 3. Changing pixel information in the video signal can e.g. indicate the proportions of a group of objects 4 and a movement direction of such a group. Subsequently, within a group and by means of profiling P, e.g. two heads of person can be distinguished, whereas the bodies can not be distinguished. Consequently, two persons can be counted and the advertising value for the advertisement is determined more accurately.

Figs. 4A-4D finally show some embodiments for which for the collection of the number information from the counting system 5. In Figure 4A it is shown schematically that the counting system 5 is connected to a network 20 through which the number information can be obtained to a computer 21.

In Figure 4B a counting system 5 is shown schematically with a display 22 from which the number information can be read. In Figure 4C a counting system 5 is shown schematically with a display port 23.

In Figure 4D a camera 3 is shown schematically with a built-in counting system 5 which can be read out in one or more of the ways shown in Figs. 4A-4C.

Figure 5 shows a billboard - camara configuration with passing people. A billboard 31 is located in a street with e.g. houses 32. People 33 are walking through the street, passing frontside of the billboard 31 (which in the figure is seen at its backside); the people who are passing the billboard at a certain moment — and thus are invisible from the backside - are drawn with grey lines. One or two video or repetitive/recurring image cameras 34 make a stream of images of the persons which pass the area (detection area) in front of the billboard. For the sake of simplicity of the illustration of figures 5 and 6, the detection area, covered by the cameras 34 and represented by the camera images, will be considered to coincide with the dimensions of the billboard: of all persons 33' who are passing the billboard (and who may read or see the billboard's content during their pass) thus images will be be taken as long as they are passing the billboard or, more precisely, the detection area covered by the cameras 34.

Figure 6a shows a number of such images 35. It can be seen in the subsequently taken images that the persons 33' are moving from the right to the left or vice versa.

According to a preferred embodiment of the invention of all images 35(a,b,c,d), taken by the camera(s) 34 only a small section (either oriented vertically as illustrated in figure 6a, or oriented horizontally) of each complete image 35 is used for counting the number of passers 33'. By means of hardware or software processing in or outside the cameras 34, only a vertical image strip 36, having a fixed horizontal position, is used for processing.

Figure 6b shows the sections derived from the images 35a...d mutually seperated. Figure 6c shows those sections in a more compacted view, illustrating that the number of persons 31'a, 31'b and 31'c, which were passing the "detection strip" 36 during the relevant observation time represneted by the images 35a...d, can be detected from the collapsed representation shown in figure 6c.

In practice the method as illustrated in figure 6, viz. reducing each image to a relative small section, to assemble those sections, to investigate the assembled section for clusters and to count those clusters results in a rather good score related to the real number of passing people. For that reason applying the method — executed by system 38 - as outlined in the foregoing and illustrated in figures 5 and 6 makes it possible to waive manual counting e.g. by students etc. in order to determinate the reach of advertisements like the advertisement on billboard 31 in figure 5.

Claims

1. Method for counting the number of passers-by passing an object (1,31) with the help of an optical detection system (2), which determines a detection area (6,35), comprising the steps of:

a) detection of one or more objects (4,33) within the detection area by the detection system which is arranged to record images of at least part of the detection area and converting those images into a representation code;

b) counting of he number of detected objects in the representation code.

2. The method according to claim 1, for which the detection system is adjusted in such a way that the detection area is determined where the object (1,31) is visible.

3. Method according to claim 1 or 2, for which the detection system is adjusted in such a way that detection of objects takes place by at least one camera under an angle (a) with the vertical (V).

4. Method according one or more of the preceding claims, for which preceding to step b) at least one image interpretation step (12) is carried out at the representation code for the distinction of objects (4).

5. Method according to claim 4, for which the image interpretation step (12) makes use of beforehand defined object models (M).

6. Method according to one or more of the preceding claims, for which the representation code is not saved.

7. Method according to one or more of the preceding claims, for which the detection system (2) is connected to a network (20) and the number information is obtained through the network.

8. Method according to one or more of the preceding claims, comprising the steps of

- converting each image (35a...d) of the detection area into a representation code which is representative for only a relative small section (36a... d) of the whole detection area;

- assembling those small sections and investigating the resulting section assembly for clusters (37a...c);

- processing the number of clusters, considered as being representative for the said number of passers-by.

9. Method according one or more of the preceding claims, said object being a commercially or otherwise attractive object and the counted number of passers-by being a measure for the object's commercial or other attractiveness.

10. Method according one or more of the preceding claims, said object being an advertisement object and the counted number of passers-by being a measure for the object's advertising value or effectiveness.

11. System (2) arranged for counting the number of passers-by passing an object (1,31) comprising at least one camera (3,34) positioned under an angle (a) with a vertical (V) for the detection of objects (4,33) within a detection area (6,35) of the camera and a counting system (5,38), arranged for the counting of the number of detected objects.

12. System according to claim 11, for which the system comprises at least one first camera with a first detection area and a second camera with a second detection area, for which the first detection area and the second detection area overlap each other at least partially, further comprising a data processing unit (7,38) for the processing of the representation codes of the first and second camera for the counting of the objects.

13. System (2) according to claim 11 or 12, for which at least the counting system (5) is connected to a network for the collection of the number information.

14. System according to one or more of the preceding claims 11 to 13 inclusive, comprising means (38) for converting each image (35a...d) of the detection area into a representation code which is representative for only a relative small section (36a...d) of the whole detection area, as well as means (38) for assembling those small sections and for investigating the resulting section assembly for clusters (37a... c), and, moreover, means (38) for processing the number of clusters, considered as being representative for the said number of passers-by.

15. System according one or more of the preceding claims 11 tol4 inclusive, said object being a commercially or otherwise attractive object and the counted number of passers-by being a measure for the object's commercial or other attractiveness.

16. System according one or more of the preceding claims 11 tolδ inclusive, said object being an advertisement object and the counted number of passers-by being a measure for the object's advertising value or effectiveness.