WO1998052309A1

WO1998052309A1 - Method and device for measuring the attention of living organisms in a plurality of living organisms

Info

Publication number: WO1998052309A1
Application number: PCT/EP1998/002744
Authority: WO
Inventors: Frank Schaeffel; Stefan Weiss
Original assignee: Eberhard-Karls-Universität Tübingen
Priority date: 1997-05-09
Filing date: 1998-05-11
Publication date: 1998-11-19
Also published as: DE19719695A1

Abstract

The invention relates to a method and device for detecting the attention of living organisms, specially human beings, within a plurality of analog living organisms. To this end, light is emitted towards the group. The light reflected by the living organism is detected in the form of an image (40). The image (40) is analyzed in such a way that the light that is reflected by the fundus oculi of the living organisms is filtered and recognized as an image point (53). The image points (53) are counted.

Description

Method and device for measuring the

Attention of living beings within a set of living beings

The invention relates to a method for measuring the attention of living beings within a set of living beings, with the features:

a) sending out a light to the crowd of living beings;

b) Detecting the light reflected by the living being in the form of an image. The invention further relates to a device for measuring the alertness of living beings within a set of living beings with:

a) means for emitting a light on the number of living beings;

b) means for detecting the light reflected by the living being in the form of an image.

It is known e.g. in the advertising industry, to record which attention e.g. creates an advertisement, such as an advertising poster, within a crowd that can see the advertisement by, for example, walking past an advertising poster.

This is usually done by observing the crowd and observing how many people within the crowd are looking at the advertisement. For this purpose, one can also set up a video camera and view or record the image captured by the camera at a different location and finally it is also possible to photograph the crowd in question and to subsequently evaluate the photos taken accordingly.

However, these known methods and devices, which essentially consist either of personal observation or photography / videography, are very complex to carry out and / or evaluate.

On the other hand, there is a need for a way to attract the attention of living things, especially humans, to be able to measure reliably. This is interesting not only for the advertising industry, but also for television stations, for example, who want to determine how much viewers watch a certain television program for how long. To determine these so-called ratings, measuring devices are conventionally used which are set up in certain households, these households being intended to represent a representative cross section through the population.

The people living in the household are now instructed to enter, on a keypad of the device, by pressing a button, which person is watching which program on television within which time period.

It is known that this acquisition of ratings is associated with some uncertainties. Not only is it uncertain whether the people living in the household in question actually operate the device in question reliably, it is also unreliable whether a person in the household who has signaled their capacity as a viewer of a particular program by pressing the button assigned to them, then this The shipment is actually being followed closely or is actually engaged in another activity, for example the newspaper is reading.

There is therefore a need to be able to precisely capture the attention of members of this household.

In a method of the type mentioned at the outset, this object is achieved according to the invention by the following features: c) analyzing the image in such a way that light reflected from the background of the living being is filtered out and recognized as a pixel; and

d) counting the pixels.

In the case of a device of the type mentioned in the introduction, the object on which the invention is based is achieved by:

c) means for analyzing the image in such a way that light reflected from the background of the living being is filtered out and recognized as a pixel; and

d) means for counting the pixels.

The object underlying the invention is completely achieved in this way.

By detecting the light reflected from the background of the eyes, it is ensured first of all that only those persons whose eyes are open and who look with the open eye in a predetermined direction are detected. In this way, all persons who have either closed their eyes or are looking in a direction other than the one of interest can be reliably eliminated as non-attentive.

In a preferred development of the method according to the invention, infrared light is emitted as light.

This measure has the advantage that it is possible to work with a light intensity sufficient for the measurement without this becomes annoying or even harmful for the observed living being, since infrared light is in the invisible spectrum of the light and its amplitude can be adjusted to medically acceptable values.

In a corresponding manner, it is expedient to detect the reflected light through an infrared filter.

This measure has the advantage that the ambient light can be filtered out to a large extent, the filter effect naturally succeeding the better, the fewer infrared components the surrounding light has.

In a further preferred embodiment of the method according to the invention, the light is emitted in a pulsed manner.

This measure has the advantage that it is possible to work with a relatively low mean value of the light intensity, a correspondingly high level being adjustable during the pulse duration.

The pulse duration is preferably of the order of a few ms, in particular of the order of 5 ms.

A method is particularly preferred in which the image is captured as a video image. This measure has the advantage that the image is in a usual signal form, so that subsequent image processing is particularly simple.

In a preferred development of this exemplary embodiment, a plurality of video images are recorded in succession at a predetermined repetition frequency. This measure has the advantage that, on the one hand, long-term observation of living beings is possible, but on the other hand, greater reliability can be achieved by separately evaluating the individual video images.

The repetition frequency is preferably 25 Hz, but can also be 30 Hz, for example.

In the context of this exemplary embodiment, it is further preferred if different light sources are used to record temporally successive video images.

This measure has the advantage that the set or group of life beings can be illuminated from different angles by means of the different light sources, so that the measurements become even more reliable.

It is particularly preferred to read successive video images alternately into at least two different memories. In particular, it is preferred to analyze the image stored in the one memory while the image intended for the at least second memory is being recorded.

This measure has the advantage that it is possible to work with a relatively high repetition frequency of the video images, but on the other hand twice as long time is available for evaluating one image in each case, compared to a procedure in which the images would be processed strictly in series. In particular when using several parallel memories, the possibility of a tremor-free display on a video monitor, in particular on an LCD screen, is opened up.

This measure has the advantage that the observed group of living beings can also be operated by optically viewing a video image, the use of an LCD screen also opening up the possibility of carrying out the method with portable devices.

In a preferred embodiment of the method according to the invention, the number of pixels is represented alphanumerically.

This measure has the advantage that an immediate evaluation of the measured values is possible.

The number of pixels can preferably be superimposed on the video image.

This has the advantage that the measurement is immediately assigned to the video image and e.g. can be documented by printing out the video image with the displayed numbers.

In order to suppress artifacts which are caused, for example, by reflection of the measuring light on well-reflecting surfaces (jewelry, watch glasses and the like) when carrying out the method according to the invention, several preferred method variants are provided:

In a first variant, the pixels are only taken into account if they occur in pairs. This variant takes into account the fact that with careful observation, both eyes are directed in the same direction, so that the pixels must necessarily appear in pairs.

In another variant, the pixels are only taken into account if they are spaced apart by a predetermined distance.

This measure takes into account the fact that e.g. in humans the pupil distance is largely always the same and is around 65 mm.

Artifacts can also be suppressed by only taking into account pixels that occur in pairs and whose connecting line runs essentially horizontally.

This variant takes into account the fact that an attentively observing person usually keeps his head upright.

A further suppression of artifacts is possible by taking into account only those pixels that resemble a circular shape to a predetermined extent.

This variant takes advantage of the fact that the pupil is circular and the light reflected from the back of the eye consequently appears as a circular point. In addition, this variant can be refined by taking into account only those pixels whose diameter lies in a predetermined diameter range. This diameter range can For example, be characterized by a certain number of pixels in the recorded video image.

In another variant of the suppression of artifacts, the image points which are similar to a circular shape are only taken into account if their brightness differs by a predetermined amount from the average brightness of the surrounding background of the image.

This method variant can be used above all if strong infrared light is used to carry out the measurement, the images of which stand out clearly from the surrounding background.

Finally, suppression of artifacts is advantageously also possible in that the image points similar to a circular shape are only taken into account if the image point is surrounded by a further, essentially circular area.

This method variant takes advantage of the fact that the pupil of an eye is surrounded by the likewise circular iris, which can also be recognized in the video image if the measurement is correspondingly precise.

It has already been mentioned that the invention can be used particularly preferably in the advertising industry, so that preferred embodiments of the device according to the invention are preferably designed to be portable.

Finally, a device according to the invention can preferably be operatively connected to a detection device arranged on a television set for determining the inputs switching quotas are in operative connection, or the device is attached to an advertising object.

Further advantages result from the description of the attached drawing.

It goes without saying that the features mentioned above and those yet to be explained below can be used not only in the combination indicated in each case, but also in other combinations or on their own without departing from the scope of the present invention.

Exemplary embodiments of the invention are shown in the drawing and are explained in more detail in the following description. Show it:

Figure 1 shows a highly schematic beam path to explain the recording technology underlying the invention.

2 shows an extremely schematic diagram of a video camera as can be used in the context of the present invention;

3 and 4

Representations of the head of a person who is judged as inattentive (Fig. 3) in one case and attentive (Fig. 4) in the other case;

5 shows an image of a group of people recorded by means of the method according to the invention, some of whom are attentive in the sense of the invention and some who are not alert; 6 shows an extremely schematic illustration to explain a possibility of suppressing artifacts;

7 shows a further illustration, likewise to illustrate possibilities for suppressing artifacts;

8 shows an extremely schematic block diagram of an exemplary embodiment of a device according to the invention; and

9 is an extremely schematic perspective view of an application of the present invention.

1 and 2, 10 denotes a camera, in particular a video camera. The camera 10 comprises a housing 11 and a tube 12 placed at the front. A conventional lens 13 is located at the front end of the tube 12. An infrared filter 14 is preferably arranged in front of the lens 13.

The lens 13 is surrounded by a ring 15, which comprises a multiplicity of light-emitting diodes, in particular infrared light-emitting diodes 16. An image recording element, in particular a CCD element 17 with a schematically indicated connection element 18, is also shown within the housing 11 of the camera 10.

A conventional CCD video camera can be used as the camera 10, for example, as it is sold under the type designation DMK 2002 by the company DBS Digitale Bildverarbeitung, Bremen. Such a camera has good sensitivity in the near infrared (700 to 900 without infrared filter) mm). The lens 13 can be a conventional wide-angle lens of 28 mm focal length in order to be able to monitor a reasonably large field of view.

Commercially available high-power infrared light-emitting diodes, such as are used, for example, are available from Conrad Electronic under the type designation 185809-99. The light emitting diodes 16 can be pulsed synchronously via a corresponding circuit from the serial interface of a computer used for evaluation, the pulse duration preferably corresponding to 5 ms lighting duration per video image. The light energy is then sufficient to carry out the desired measurements at a distance of 5 to 10 m. The visible light is blocked out by the filter 14, which is transparent to infrared between 700 and 900 nm.

The camera 10 is operated such that it can be used with a portable computer housing including a motherboard, video card and LCD display. The video images are preferably displayed at a refresh rate of 25 Hz. In this way, the weight of the overall arrangement can be kept at about 6 kg. The CCD element 17 is selected so that the pixel resolution of the video image is approximately 795 (H) x 596 (V) pixels.

In the right half of FIG. 1, 20 denotes an eye, in particular a human eye. The eye 20 has a lens 21 and an eye fundus 22, which is essentially formed by the retina.

The light 25 emitted by the infrared light-emitting diodes 16 passes through the lens 21 onto the fundus 22 and is diffusely reflected there, as indicated at 26. The light 27 reflected by the lens 21 now passes through the lens 13 of the camera 10 and reaches the CCD element 17.

3 and 4 show how this measurement affects a person 30a or 30b who is "inattentive" in the sense of the present invention and "attentive" in the other case.

3 shows the case of the inattentive person 30a. The person 30a looks to the left in FIG. 3, as indicated by a letter L and an arrow 32. The pupils 31a are therefore directed to the right in the view of FIG. 3. Thus, no emitted light 25 reaches the fundus 22 because the optical axis in the viewing direction according to FIG. 3 differs from the optical axis 28 according to FIG. 1. The image of the eyes of the person 30a recognizable in the video image is therefore inconspicuous.

However, if the person 30b now looks into the camera 10, as shown in FIG. 4, the case shown in FIG. 1 based on a beam path arises. The light 27 reflected by the fundus 22 now reaches the camera 10 and its CCD element 17 in its entirety, so that the pupils 31b are clearly recognizable as light spots.

This effect is known from the usual photography, in which people with a flash device are shown with clearly red eyes because the flash light falls into the open eye and is reflected by the well-perfused retina in the back of the eye. This effect will in poor lighting conditions even more so that the pupil is wide open in these lighting conditions.

In Fig. 5 the case is now shown that not only a single person but rather a group of people is measured simultaneously with the technique explained.

A total of nine people can be seen schematically in FIG. 5, some of them looking in the direction of the camera (attentively) or looking in another direction (inattentive).

5 shows a total of five persons 41, 42, 43, 44, 45, who are attentive in this sense, and four persons 46, 47, 48, 49, who are inattentive in this sense. In addition, there are three artifacts 50, 51, 52 to recognize.

As can be seen from FIG. 5, the eyes (or respectively the iris) of the attentive persons 41 to 45 are depicted as clear image points 53, while this is not the case for the non-attentive persons 46 to 49.

In addition, the video scene 40 shows a first artifact 50 in the person 49 because the infrared light was reflected on a hair clip. In person 47, an artifact 51 is shown by reflection on a belt buckle, while a third artifact 52 is of unknown origin.

In order to be able to reliably suppress these artifacts 50 to 52, various procedures are conceivable.

6 shows, for example, that two pixels 53, which correspond to the pair of pupils of two eyes, are at least approximately in a predetermined distance must be arranged, namely the usual pupil distance of people d _A , which is about 65 mm.

In addition, the connecting line between the two pixels 53 should run approximately horizontally, since attentive people normally hold their heads upright. This is indicated in Fig. 6 with an angle α, which should be approximately 0 ° to the horizontal.

7 denotes further possibilities for suppressing artifacts.

As has already been stated above, the CCD element 17 of the camera 10 preferably has a resolution of 795 x 596 pixels in the horizontal or vertical. The pixels are designated 55 in FIG. 7.

To an artifact, i.e. To be able to reliably detect a non-circular pixel is e.g. a minimum pupil diameter of 4 to 5 pixels, i.e. Picture elements, recommended. With 5 pixels pupil diameter and an average pupil diameter of 5 mm, 1 pixel in the video image corresponds to approximately 1 mm. The field of view of the video scene 40 can therefore be approximately 8 m in size.

The video scene 40 can now be searched for circular pixels which have a diameter of at least 4 pixels. You can also check whether their brightness is higher than the average background brightness.

Finally, it can also be evaluated whether the pixel 53 recognized as the pupil 31 is surrounded by another circle , namely the iris 56 of the eye, as shown in FIG. 7. Of course, the iris 56 does not have to be shown in its full extent, since it can be partially covered by the eyelids above and below.

In the block diagram shown in FIG. 8, the camera 10 with its already explained elements 11 to 18 can be seen at the top.

A first signal line 58 leads from the camera 10 to a memory unit 60. The memory unit 60 comprises a first memory 61 and a second memory 62. A second signal line 63 and a third signal line 64 lead from the first memory 61 and the second memory 62, respectively a control unit 65. The control unit 65 has a fourth control line 66 on the output side, which also serves as a clock line. The control unit 65 preferably comprises an alphanumeric display 68.

A fifth signal line 67 leads from the control unit 65 to a display unit 70. The display unit 70 has an LCD screen 71.

The video image recorded by the CCD element 17 is alternatively fed to the first memory 61 via the first signal line 58 or to the second memory 62 with the subsequent video image. If, for example, a first video image has been read into the first memory 61, this is immediately transferred to the control unit 65 via the second signal line 63, while the subsequent image is read into the second memory 62. In the control unit 65, the image read from the first memory 61 is evaluated in the sense already mentioned by the video scene 40 (FIG. 5) of the Image is searched for corresponding pixels 53, while the artifacts 50 to 52 are suppressed. The result of this counting of the pixels is then shown on the display 68. It is understood that the number of pixels is divided by two in order to deduce the number of attentive persons from the number of pixels.

At the same time, the video scene 40 is displayed on the LCD screen 71 of the display unit 70.

While the video image read in the second memory 62 is being evaluated, the video image that follows in turn is read into the first memory 61, etc.

In order to coordinate and synchronize these operations, the fourth control line 66 serving as the clock line is provided. A clock signal sent out by the control unit 65 not only controls the alternating activation of the memories 61 and 62, it also supplies the CCD element 17 with the necessary synchronization pulses and finally also ignites the light-emitting diodes 16 which are generated during the recording of each video image be switched on for 5 ms, for example.

The necessary image processing algorithms, which are preferably written in the programming language Borland C ++ for Windows 95, run in the control unit 65 to find the pixels 53. This allows detection and counting of the pixels 53 in the video scene 40 in real time, ie with the refresh rate of, for example, 25 Hz. The number of pixels 53 detected together with the time the recording is written to a file and can be displayed as a graphic at any time.

Finally, FIG. 9 shows a preferred application of the invention.

A conventional television set 80 can be seen, on which a detection device 85, which is also known per se, is located. The device 85 is used to record ratings. It is provided with a keypad 86, with which the members of a predetermined household "register" or "deregister" for a particular television program. In this way, it is recorded which members of the family watch which television programs, at what times and for how long.

This arrangement, which is known per se, is supplemented according to the invention by the camera 10, which is located on the detection device 85 or is integrated therein. The remaining evaluation electronics, which can also be located within the arrangement according to FIG. 9 or at a distance from it, are not shown for the sake of simplicity.

If the recording device 85 is combined with a camera 10 and the evaluation arrangement according to the present application, it is not only possible to determine in this way whether the number of people indicated in the keypad 86 is actually gathered in front of the television set, but rather also by recording and evaluating a video scene 40 similar to FIG. 5, determine how many of the people present watch the television program in question at all or indulge in another occupation or even fell asleep.

Claims

claims

1. A method for measuring the attention of living beings within a set of living beings with the features:

a) emitting a light (25) on the amount of living beings;

b) capturing the light reflected by the living being in the form of an image (40);

characterized by the other features:

c) analyzing the image (40) in such a way that light (27) reflected from the background of the eye (22) of the living being is filtered out and recognized as the image point (53); and

d) counting the pixels (53).

2. The method according to claim 1, characterized in that infrared light is emitted as light (25).

3. The method according to claim 2, characterized in that the reflected light is detected by an infrared filter (14).

4. The method according to one or more of claims 1 to 3, characterized in that the light (25) is emitted in a pulsed manner.

5. The method according to claim 4, characterized in that the light is pulsed with a pulse duration of a few ms.

6. The method according to claim 5, characterized in that the pulse duration is about 5 ms.

7. The method according to one or more of claims 1 to 6, characterized in that the image (40) is detected as a video image.

8. The method according to claim 7, characterized in that a plurality of video images are recorded in succession at a predetermined repetition frequency.

9. The method according to claim 8, characterized in that the repetition frequency is approximately 25 Hz.

10. The method according to claim 7 or 8, characterized in that different light sources are used to record temporally successive video images.

11. The method according to claim 10, characterized in that the different light sources are positioned at different locations.

12. The method according to one or more of claims 8 to 11, characterized in that successive video images are read alternately in at least two different memories (61, 62).

13. The method according to claim 12, characterized in that the image stored in the one memory (61) is analyzed, while the image intended for the at least second memory (62) is recorded.

14. The method according to one or more of claims 7 to

13, characterized in that the video image is displayed on an LCD screen (71).

15. The method according to one or more of claims 1 to

14, characterized in that the number of pixels (53) is displayed alphanumerically (68).

16. The method according to claim 15 and one or more of claims 7 to 14, characterized in that the number is superimposed on the video image.

17. The method according to one or more of claims 1 to 16, characterized in that to suppress artifacts, the pixels (53) are only taken into account if they occur in pairs.

18. The method according to claim 17, characterized in that the paired pixels (53) are only taken into account if they are spaced apart by a predetermined distance (dA).

19. The method according to claim 17 or 18, characterized in that the paired pixels (53) are only taken into account if their connecting line (1) is substantially horizontal (α).

20. The method according to one or more of claims 1 to 19, characterized in that to suppress artifacts, the pixels (53) are only taken into account if they resemble a circular shape to a predetermined extent.

21. The method according to claim 20, characterized in that the pixels similar to a circular shape (53) are only taken into account if their diameter is in a predetermined diameter range.

22. The method according to claim 20 or 21, characterized in that the pixels similar to a circular shape (53) are only taken into account if their brightness stands out by a predetermined amount from the average brightness of the surrounding background of the image (40).

23. The method according to one or more of claims 20 to 22, characterized in that the image points (53) which are similar to a circular shape are only taken into account if the image point is surrounded by a further, essentially circular region.

24. Device for measuring the attention of living beings within a set of living beings with:

a) means for emitting a light (25) on the amount of living beings;

b) means for detecting the light reflected by the living beings in the form of an image (40), marked by:

c) means for analyzing the image (40) in such a way that light (27) reflected from the background of the eye (22) of the living being is filtered out and recognized as the image point (53); and

d) means for number of pixels (53).

25. The device according to claim 24, characterized in that it is designed to be portable.

26. The apparatus of claim 24 or 25, characterized in that it is in operative connection with a on a television (80) arranged detection device (85) for determining the ratings.

27. The apparatus according to claim 24 or 25, characterized in that it is attached to an advertising object.