AU7833898A - Voice recognition unit for audience measurement system - Google Patents

Description

" d 'vauo PCT/US98/12034 VOICE RECOGNITION UNIT FOR AUDIENCE MEASUREMENT SYSTEM Technical Field of the Invention The present invention relates generally to an audience measurement system that identifies members of an audience of a program to which a receiver, such as a television, radio, or computer, is tuned, and/or that identifies a program to which a receiver, such as a television, radio, or computer, is tuned. Background of the Invention Measuring audiences of programs has long been of interest to program suppliers and advertisers because audience measurements provide the data from which the effectiveness of programs and advertisements may be eval uated. A variety of well known methods have been em ployed in order to provide an estimate of the total audi ence to a program, to a portion of a program, and/or to a commercial. In addition to the total audience, these methods provide additional estimates of demographically significant audience segments (e.g., the number of men and/or women between certain ages who watched a selected - 1 - " ' 1UUoo PCT/US98/12034 portion of a given program). These methods involve manu ally and/or automatically measuring the viewing and/or listening habits of audience members of statistically selected households and then estimating the total audi 5 ence based on these measurements. Audience members of statistically selected households are usually referred to as panelists. The measurement of the viewing and/or listening habits of an audience generally requires three separate 0 determinations: (1) a determination of the channels or stations to which the monitored receivers are tuned with in the statistically selected households; (2) a determi nation of the programs which were available on the tuned channels at the times during which the receivers were 5 tuned to these channels or stations; and, (3) a determi nation of the household members and visitors who were present in the area of the receivers during the time that the receivers were tuned to selected programs. All three determinations can affect the accu 0 racy of audience measurements. Of the three determina tions discussed above, determination (3) perhaps most - 2 - VV YY/ food PCT/US98/12034 affects the accuracy of audience measurements because the members of an audience of a receiver are usually required to cooperate in the identification process. When members are required to participate in the identification pro 5 cess, they frequently make errors, or they simply forget to identify either themselves or the other members of the audience. For example, one known system which allows audience members to identify themselves in order to indi cate their presence is a "People Meter." The "People 0 Meter" may be a remote control and/or a plurality of push buttons activated by the audience members when they enter or exit the monitored area. However, audience members frequently forget to press any button at all, or they sometimes make errors in pressing the right button. 5 In an attempt to overcome this problem, several measurement systems have been developed. For example, in U.S. Patent No. 3,056,135, Currey et al. describe a sys tem which utilizes strategically placed switches for counting the number of persons entering, leaving, and re 0 maining within a monitored area, and a photographic re corder for periodically recording the composition of the - 3 - W 7 ood PCT/US98/12034 audience. This approach requires that the photographic record be viewed by an operator, which both invades the privacy of the audience members, and imposes an unaccept able cost on the measurement operation. 5 These problems led to a variety of suggestions for passive, non-obtrusive methods of counting (but not identifying) viewers, and of tracking their movement about the monitored area. For example, Kiewit, et al., in U.S. Patent No. 4,644,509, describe an ultrasonic .0 sonar system which counts and tracks members of the audi ence. However, the described system does not identify the members of the audience and, therefore, requires the members of the audience to manually identify themselves. Other systems, such as that described by Kiewit 5 in U.S. Patent 4,930,011, rely upon electronically active tags which are worn or carried by members of an audience and which uniquely identify corresponding ones of these audience members. These systems respond to the tags whenever the audience members wearing the tags are within 0 the monitored area of a receiver. However, audience - 4 - 7 /If UUo PCT/US98/12034 members often forget to wear their tags, in which case such audience members are not identified. Lu, in U.S. Patent No. 4,858,000, and in U.S. Patent No. 5,031,228, discloses an image recognition 5 system for identifying the members of an audience of a receiver. The system uses a video camera to acquire images of the faces of the audience members, and a com puter subsystem to recognize the faces in these images by comparing these faces to reference faces stored in a data 0 base. The system also locates and tracks viewers, and includes infrared illuminators that provide a control lable level of imperceptible illumination of the moni tored area for the video camera. However, the system is generally expensive and difficult to install. 5 The system disclosed by Lu et al., in U.S. Patent No. 5,550,928 also uses image recognition to iden tify audience members. This system incorporates algo rithms for finding bodies and then heads of members of an audience. Once the head is found, a face finding algo 0 rithm extracts facial image representations that are provided to two face recognition routines, an Eigenface - 5 - SYY/z /oon PCTIUS98/12034 recognition routine and a template matching routine. This system can identify audience members who are moving about the monitored area and/or who are not facing the cameras. However, it is still complex and expensive. 5 Determinations (1) and (2) discussed above also can affect the accuracy of audience measurements because one of these determinations alone is normally not suffi cient to identify the programs to which receivers are tuned. That is, a determination of the channels to which 0 a receiver in a statistically selected household is tuned does not lead to a determination of the tuned programs without also knowing what programs were available on the tuned channels. By the same token, a determination of the programs that were available on the tuned channels .5 does not lead to a determination of the tuned programs without knowing the channels to which the receiver in a statistically selected household is tuned. Thus, both determinations are typically required in order to deter mine the programs tuned by the audience members. 0 Acquisition of information about available programs and entering this information into a database for later - 6 - S%7//ooo3 PCT/US98/12034 matching with the tuned channels leads to acquisition and data entry errors that adversely affect the accuracy of audience measurements. Thomas et al., in U.S. Patent No. 5,481,294, 5 addresses these problems with an audience measurement system that detects and records ancillary codes which are embedded in, and identify, the programs to which receiv ers are tuned. Thus, only one determination needs to be made if an ancillary code is present in the viewed pro 0 grams. If ancillary codes are not available in the tuned programs, program signatures are extracted from the pro grams and are compared to reference signatures previously extracted. The metering apparatus disclosed in this patent 5 may include non-intrusive sensors, such as microphones near the receiver, that detect the ancillary codes in the audio output of the monitored receivers. If microphones are used to detect the ancillary codes, second micro phones may be provided to pick up background noise. The 0 outputs of the ancillary code sensing microphones and the noise sensing microphones are processed in order to sub - 7 - S Y/oon PCT/US98/12034 tract the background noise from the signals provided by the ancillary code sensing microphones. This audience measurement system may also in clude a "People Meter" as described above for additional 5 backup. Turner et al., in U.S. Patent No. 4,907,079, disclose a system for monitoring audiences of receivers. The members of the audience provide control commands (such as channel selections) to a microprocessor through 0 an infrared unit which includes a keyboard. Information collected by the microprocessor may be stored and trans mitted to a central computer over conventional phone lines. A motion detector coupled to the microprocessor detects when an audience member has entered or exited the 5 monitored area. A character generator coupled to the microprocessor allows communication with the audience members via alphanumeric messages displayed, for example, on a television screen. Also, a speech output unit cou pled to the microprocessor generates speech signals to 0 communicate to the audience members. - 8 - "uY YY oon PCTIUS98/12034 The Turner et al. system also includes a speech input unit coupled to the microprocessor to allow audi ence members to audibly identify themselves. Audible speech from the audience members is detectable by a mi 5 crophone, is amplified, is digitized, and is provided to the microprocessor. The microprocessor performs a power density spectrum analysis on the signal to derive voice signatures of the particular audience members. These signatures are compared to reference signatures, which 0 were previously stored during initialization of the sys tem, in order to identify viewers. The microprocessor may also normalize the digital voice data and remove small amplitude samples in order to minimize the effects of background noise. When the system disclosed in this 5 Turner et al. patent is activated, an initial display message is placed on the viewing screen which requests the audience to speak their first and last names in order to provide the reference signatures. Voice recognition by itself does not, however, permit a viable audience 0 measurement system. Moreover, the disclosed signal pro - 9 - " . I IUUo PCT/US98/12034 cessing that minimizes the effects of background noise may not be adequate under many conditions. The present invention is directed to a system which identifies members of an audience of a receiver and 5 which solves one or more of the problems noted above. Summary of the Invention In accordance with a first aspect of the pres ent invention, a system for identifying a member of an audience of a receiver comprises first and second micro 0 phones, a noise cancellation unit, and a voice recogni tion unit. The first and second microphones are arranged so that one of the first and second microphones picks up relatively more of a voice signal from the member of the audience and the other of the first and second micro 5 phones picks up relatively more of a sound signal from the receiver. The noise cancellation unit is coupled to the first and second microphones, and the noise cancella tion unit is arranged to substantially eliminate the sound signal from the voice signal. The voice recogni 0 tion unit is coupled to the noise cancellation unit, and - 10 o YY/Z/oo PCTIUS98/12034 the voice recognition unit is arranged to identify the voice signal from the member of the audience. In accordance with another aspect of the pres ent invention, a system comprises a microphone, a voice 5 recognition unit, and a program sensor. The system iden tifies a member of an audience of a program to which a receiver is tuned. The microphone is located to receive a voice signal from the member. The voice recognition unit is coupled to the microphone and is arranged to 0 identify the member from the voice signal. The program sensor is arranged to identify the program. In accordance with yet another aspect of the present invention, a system comprises a passive people meter, a microphone, a voice recognition unit, and a con 5 troller. The system identifies a member of an audience of a program to which a receiver is tuned. The passive people meter is arranged to provide a first identifica tion of the member by passively identifying the member. The microphone is located to receive a voice signal from 0 the member. The voice recognition unit is coupled to the microphone and is arranged to provide a second identifi - 11 - ' /uooa PCT/US98/12034 cation of the member by identifying the member from the voice signal. The controller is arranged to select at least one of the first and second identifications in order to identify the member. 5 In accordance with still another aspect of the present invention, an audience recognition unit comprises a housing, a microphone supported by the housing, and a voice recognition unit supported by the housing and cou pled to the microphone. The voice recognition unit is 0 arranged to recognize a member of an audience from an identifier spoken by the member. Accordingly, the audi ence recognition unit identifies the member of the audi ence of a program to which a receiver is tuned. - 12 - VY YY// oo PCTIUS98/12034 Brief Description of the Drawings These and other features and advantages of the present invention will become more apparent from a de tailed consideration of the invention when taken in con 5 junction with the drawings in which: Figure 1 illustrates the system of the present invention for use in identifying members of an audience of a program to which a television receiver is tuned; Figure 2 is a block diagram of an embodiment of LO the system which is in accordance with the present inven tion and which includes an audience recognition unit (ARU); Figure 3 is a more detailed diagram of the audience recognition unit shown in Figure 2; L5 Figure 4 is a diagram of a noise cancellation arrangement for use in the audience recognition unit shown in Figure 2; Figure 5 illustrates a hand held version of the audience recognition unit shown in Figure 3; ?0 Figure 6 illustrates a table top version of the audience recognition unit shown in Figure 3; - 13 - VVW YY// oo5 PCT/US98/12034 Figure 7 is a flow chart depicting a program which may be implemented by the audience recognition unit shown in Figure 3 during a training period; and, Figure 8 is a flow chart depicting a program 5 which may be implemented by the audience recognition unit shown in Figure 3 during voice recognition of audience members. Detailed Description An audience measurement system 10, which deter ) mines the members of an audience to a program to which a monitored receiver is tuned, is shown in Figure 1. The audience measurement system 10 includes an audience recognition unit 12, which is shown in Figure 1 as a hand held unit but which alternatively may be any other type S of unit, such as a table top unit. The audience recog nition unit 12 permits the members of the audience to be identified either passively or actively, and may or may not interact with a receiver 14 during the identification process. - 14 - "%Y ZI0 'oPCT/US98/12034 A program sensor 16 cooperates with the re ceiver 14 in order to determine the programs (and/or channels) to which the receiver 14 is tuned. The program sensor 16 may be any known sensor which determines chan 5 nels (and/or programs). For example, the program sensor 16 may sense ancillary codes which are embedded in the program signals to which the receiver 14 is tuned and which specifically identify the tuned programs. In addi tion, or alternatively, the program sensor 16 may include 0 a signature extractor which extracts characterizing sig natures from the program signals to which the receiver 14 is tuned so that the signatures may be compared to refer ence signatures in order to identify the programs (and/or channels) to which the receiver 14 is tuned. In addi 5 tion, or alternatively, the program sensor 16 may be an apparatus which determines the channels to which the receiver 14 is tuned by monitoring the tuner of the re ceiver 14. The audience measurement unit 12 and the pro 0 gram sensor 16 communicate with a base unit 18 which may be either a site unit or a home unit. As is known in the - 15 y % 77.ood PCTIUS98/12034 art, a site unit may be provided at each audience area within a household, and all site units within the house hold may communicate to a home unit over the electrical power lines in the walls of the household. The home unit may then communicate by way of the public telephone sys tem with a central unit which gathers tuning and audience information from other households so that the information can be combined into reports useful to program suppliers, advertisers, and the like. In households having only one viewing area, the base unit 18 may be arranged to both receive data from the audience measurement unit 12 and the program sensor 16, and may communicate this data by way of the public telephone system to the central unit. Thus, as shown in Figure 2, the audience recognition unit 12 provides its recognition data to the base unit 18 by way of a communication link 20 which, for example, may be an infrared or RF communication link. The program sensor 16 provides its program identification data to the base unit 18 by way of a communication link 21 which, for example, may be a hard wire, but which may alternatively be any other communication link, such as an - 16 - VV -Y /ZOO PCT/US98/12034 infrared or RF communication link. The base unit 18 collects the audience recognition data from the audience recognition unit 12 and the program identification data from the program sensor 16, and may communicate this data over a public telephone system 22 to a central site 24 which accumulates similar data from other households. As indicated in Figure 2, the audience measure ment system 10 may also include a passive people meter 26. For example, the passive people meter 26 may be of ) the type which passively identifies audience members in a monitored area in accordance with the teachings of U.S. Patent No. 5,550,928 described above. The passive people meter 26 provides its people identification data to the base unit 18 by way of a communication link 28 which, for example, may be a hard wire, but which may alternatively be any other communication link, such as an infrared or RF communication link. Accordingly, if the audience measurement system 10 includes both the audience recognition unit 12 and the ) passive people meter 26, the passive people meter 26 may be used to passively identify audience members in the - 17 - VY~ % Y/ I foo PCT/US98/12034 event that the audience recognition unit 12 is unable to do so. Alternatively, the audience recognition unit 12 may be used to actively or passively identify audience members in the event that the passive people meter 26 is unable to passively identify audience members. As a still further alternative, the outputs of both the audi ence recognition unit 12 and the passive people meter 26 may be applied simultaneously to a soft decision logic that identifies audience members based upon both of these ) outputs. In any of these alternatives, confidence is in creased that audience members can be accurately recog nized. The audience recognition unit 12 is shown in more detail in Figure 3. The audience recognition unit 5 12 includes first and second microphones 30 and 32. The first and second microphones 30 and 32 are positioned in the audience recognition unit 12 so that one of the first and second microphones 30 and 32 tends to pick up rela tively more of the voice signals from the audience mem ) bers, and the other of the first and second microphones 30 and 32 tends to pick up relatively more of the audio - 18 - SYYI. / oo PCT/US98/12034 output from the monitored receiver. For example, the first and second microphones 30 and 32 may be separated so that, in most orientations of the audience recognition unit 12, one of the first and second microphones 30 and 32 is closer to the audience members and the other of the first and second microphones 30 and 32 is closer to the monitored receiver. A preferred separation between the first and second microphones 30 and 32 is 6.72 inches, although any other suitable separation may be used. In ) this embodiment of the present invention, the first and second microphones 30 and 32 may be pointed in opposite directions. Alternatively, one or both of the first and second microphones 30 and 32 may be located remotely from the audience recognition unit 12 so that one of the first 5and second microphones 30 and 32 is closer to the audi ence members and the other of the first and second micro phones 30 and 32 is closer to the monitored receiver. The analog output of the first microphone 30 is amplified by an amplifier 34, and this amplified analog ) output is converted to a digital signal by an analog to digital converter 36. Similarly, the analog output of - 19 - 7Y/A/oo PCT/US98/12034 the second microphone 32 is amplified by an amplifier 38, and this amplified analog output is converted to a digi tal signal by an analog to digital converter 40. Even though the first and second microphones 30 and 32 can be positioned or directed so that one of the first and second microphones 30 and 32 tends to pick up relatively more of the voice signals from the audience members and the other of the first and second microphones 30 and 32 tends to pick up relatively more of the audio ) output from the monitored receiver, the first and second microphones 30 and 32 pick up both the voice signals from the audience members and the audio output from the re ceiver 14. Also, both the first and second microphones 30 and 32 pick up background noise other than the audio 5 output from the receiver 14. In order to reduce or elim inate the effects of background noise and the audio out put from the receiver 14, the outputs from the analog to digital converters 36 and 40 are provided to a noise cancellation arrangement 42. The noise cancellation 3 arrangement 42 processes the outputs from the analog to digital converters 36 and 40 so as to cancel the audio - 20 - YYI" A /oo PCT/US98/12034 output of the receiver 14 as well as background noise from the voice signals provided by the audience members, so that these voice signals may be more easily identi fied. The noise cancellation arrangement 42 may use signal subtraction in order to subtract background noise and the audio output of the receiver 14 from the voice signals provided by the audience members. For example, if the first microphone 30 is directed toward the audi ) ence members, and if the second microphone 32 is directed toward the receiver 14, the signal from the first micro phone 30 may be designated SP (p meaning people), and the signal from the second microphone 32 may be designated as Sr (r meaning reference). In general, the signal SP may 5be given according to the following equation: Sp = aVp + bVr (1) where Sp is the signal from the first microphone 30, VP is the contribution to the signal from the first microphone 30 due to the voice signals of the members of the audi - 21 - V - /Zoo PCT/US98/12034 ence, Vr is the contribution to the signal from the first microphone 30 due to the audio output of the receiver 14 and other background noise, a and b are parameters dis cussed below, and a is greater than b. Similarly, the signal Sr may be given according to the following equation: Sr = cVp + dVr (2) where Sr is the signal from the second microphone 32, c and d are parameters discussed below, and c is less than ) d. Equations (1) and (2) may be subtracted with a weighting parameter e according to the following equa tion: D = eS - S = (ae - c)-V + (be - d) -V (3) p r P r where D is the weighted difference between equations (1) 5and (2), and e is a weighting parameter discussed below. If a, b, c, and d are fixed parameters deter mined by the environment and the features of the first - 22 - ". ' ',o PCTIUS98/12034 and second microphones 30 and 32, the weighting parameter e can be selected so that (be - d) = 0, or so that e = d/b > 1. Accordingly, if (be - d) = 0, the difference D will consist only of the signal VP. In this case, the 5 signal Vp, from equation (3) is given by the following equation: D V (4) p ae - c Because a, e, and c are constants in a given household, these constants may be treated as a single constant k ac ) cording to the following equation: 1 k = (5) ae - c - 23 n u / roo PCT/US98/12034 so that the signal Vp is given by following equation: vp = kD. (6) The weighting parameter e may be determined adaptively during periods when the receiver 14 is on but 5no member of the audience is talking. During such a period, the audience recognition unit 12 can automati cally find the weighting parameter e so that D = 0. This weighting parameter e is then inserted into equations (5) and (6) in order to determine VP, and Vp is processed by a ) voice recognition unit 44 as discussed below. This noise cancellation approach assumes that the first microphone 30 is directed to, or is closer to, the audience than is the second microphone 32, and that the second microphone 32 is directed to, or is closer to, the receiver 14 than is the first microphone 30. For example, if the audience recognition unit 12 is a hand held unit, the first microphone 30 and the second micro phone 32 may be mounted on the audience recognition unit 12 so that they are separated and so that the user is - 24 - ..." UUo PCT/US98/12034 directed to orient the audience recognition unit 12 with the first microphone 30 directed toward the audience and the second microphone 32 directed toward the receiver 14. Similarly, if the audience recognition unit 12 is a table top unit, the first microphone 30 and the second micro phone 32 again may be mounted on the audience recognition unit 12 so that they are separated and so that the user is directed to orient the audience recognition unit 12 with the first microphone 30 directed toward the audience and the second microphone 32 directed toward the receiver 14. Alternatively, whether the audience recognition unit 12 is a hand held unit or a table top unit, the first and second microphones 30 and 32 may be connected to the audience recognition unit 12 by lines or other communica tion media which permit the first microphone 30 to be mounted in close proximity to the audience and the second microphone 32 to be mounted in close proximity to the receiver 14. Instead of canceling noise as discussed above, the noise cancellation arrangement 42 may use phase cancellation. It is noted that sound travels at approxi - 25 - W%7, Y1i/oon PCT/US98/12034 mately 1,120 ft per second in air. Sounds not originat ing equidistant from both of the first and second micro phones 30 and 32, which are preferably separated by 6.72 inches, reach these microphones at different times and are canceled or are at least partially canceled. Based on the actual frequencies of the sounds, and the spacing between the first and second microphones 30 and 32, dif ferent phase angles result. The noise cancellation ar rangement 42 implementing phase cancellation uses these different phase angles in order to cancel the audio out put from the receiver 14 and other background noise from voice signals provided by the members of the audience. In this embodiment of the present invention, the first and second microphones 30 and 32 are preferably direc tional microphones mounted on the same side of the audi ence recognition unit 12. The audience recognition unit 12 may have any orientation with respect to the audience and the receiver 14 as long as the first and second mi crophones 30 and 32 and not pointed directly at the re ceiver 14. - 26 - " '/o000o PCTIUS98/12034 In this embodiment of the present invention, the noise cancellation arrangement 42 may be of the form shown in Figure 4. According to this embodiment, the noise cancellation arrangement 42 includes an amplifier 5 60, which amplifies the signal from the first microphone 30, and an amplifier 62, which amplifies the signal from the second microphone 32. Because the amplifiers 60 and 62 may be essentially the same, only one of the amplifi ers is shown in detail in Figure 4. The outputs of the D amplifiers 60 and 62 are summed by a summer 64 so that background noise is substantially eliminated. The output from the summer 64 is detected and converted to a DC level by a detector 66 and is supplied to a timer 68. If the output of the summer 64 is above a 5 predetermined threshold, the timer 68 is triggered and provides an output to a switch 70 in order to connect the output of the summer 64 to the voice recognition unit 44. The voice recognition unit 44 provides a hold signal through a terminal 72 in order to hold the timer 68 in ) its triggered state until a predetermined time after the voice recognition unit 44 last recognizes a voice. - 27 - TV Y/Zoo PCT/US98/12034 The voice recognition unit 44 may be an RSC164 voice recognition processor provided by Sensory Inc. The voice recognition unit 12 compares the output from the noise cancellation arrangement 42 to prestored reference voice signals (identifiers) stored in a memory. These prestored reference voice signals (identifiers) may be spoken identifier words or voice features from a speaker 's frequency spectrum. When matches are found, members of the audience are identified, and the voice recognition ) unit 44 provides a suitable identification record to a controller 46 for each match. The controller 46 stores the identification record in a memory 48 for later transmission to the base unit 18 by way of a communica tion arrangement 50. The communication arrangement 50 5may use the communication link 20 as previously de scribed. The controller 46 may also be coupled to a clock 52 so that the times that the members of the audi ence are identified may also be stored in the memory 48. A prompt unit 54 may also be coupled to the ) controller 46. The controller 46 controls the prompt unit 54 in order to prompt audience members to either - 28 - ". "7', uo PCT/US98/12034 provide their corresponding voice signals to the first and second microphones 30 and 32, or use a keypad 56 in order to manually identify themselves. The prompt unit 54 may be a voice synthesizer and/or a graphic or video display (such as an LCD or LED) of the audience recog nition unit 12 arranged to provide audible and/or visible prompts to the members of the audience. Alternatively, the prompt unit 54 may be the speakers and/or the screen of the receiver 14 arranged to provide audible and/or visible prompts to the members of the audience in re sponse to interaction with the audience recognition unit 12. The audience recognition unit 12 may be implemented as a hand held unit 90 which is illustrated in Figure 5. The hand held unit 90 includes a housing 92 that houses the elements of the audience recognition unit 12 which are shown in Figure 3. The housing 92 supports an LED display 94 which may perform the functions of the prompt unit 54. As noted above, however, the prompt unit 54 alternatively may be a voice synthesizer, in which case the housing 92 would incorporate an output speaker - 29 - " "''o PCT/US98/12034 to project audible voice prompts from the voice synthe sizer to the members of the audience. The LED display 94 (and/or the voice synthesizer) may perform functions other than prompting. The housing 92 also houses an IR transmitter 96 for establishing the communication link 20 shown in Figure 2. Alternatively, the communication link 20 may be established by an ultrasonic or RF link, a hard wire, or the like. The housing 92 further supports a keypad 98, which may be the keypad 56 shown in Figure 3, 3 for allowing audience members to enter data. For exam ple, predetermined ones of the keys in the keypad 98 may be assigned to members and visitors of the audience so that they may identify themselves manually if they cannot be recognized through their voice signals. The housing 5 92 may contain a rechargeable battery which is arranged to provide power to the audience recognition unit 12 and which may be charged overnight or when the audience rec ognition unit 12 is not in use. The hand held unit 90 also has a power on/off button 95 in order to turn the ) hand held unit 90 on and off, and an LED 97 in order to - 30 - VV YY//oo PCT/US98/12034 indicate that the hand held unit 90 is ready to identify audience members and visitors. Alternatively, the audience recognition unit 12 may be implemented as a table top unit 100 which is 5 illustrated in Figure 6. The table top unit 100 includes a housing 102 that houses the elements of the audience recognition unit 12 which are shown in Figure 3. The housing 102 may support an LED display (not shown) in order to implement the functions of the prompt unit 54. ) As noted above, however, the prompt unit 54 alternatively may be a voice synthesizer, in which case the housing 102 would incorporate an output speaker to project audible voice prompts from the voice synthesizer to the members of the audience. The LED display or the voice synthe 5 sizer may perform functions other than prompting. The housing 102 may also house an IR or RF transmitter (not shown) for establishing the communication link 20 shown in Figure 2. Alternatively, the communication link 20 may be established by an ultrasonic link, a hard wire, or ) the like. The housing 102 supports a keypad 104, which may be the keypad 56 shown in Figure 3, for allowing - 31 - " , ' '1& o 1PCT/US98/12034 audience members to enter data. For example, predeter mined ones of the keys in the keypad 104 may be assigned to members of the audience so that they may identify themselves manually if they cannot be recognized through 5 their voice signals. The table top unit 100 may contain a rechargeable battery which is arranged to provide power to the audience recognition unit 12 and which may be charged overnight or when not in use. Alternatively, the table top unit 100 may have a power cord for providing ) power to the audience recognition unit 12 from an elec trical outlet. Also, the table top unit 100 may have a power on indicator light 106 and a voice detected indica tor light 108. During a training mode, the controller 46 of 5the audience recognition unit 12 may implement a program 120 depicted in Figure 7 by way of a flow chart. The program 120 is executed during the training mode in order to store identifier words or voice features associated with the members of the audience. During training, an ) audience member should assume a position at a specified distance from the audience recognition unit 12. This - 32 - " '7Ioon PCT/US98/12034 audience member initiates training by pressing an appropriate key or button on the keypad 56 of the audi ence recognition unit 12. When the training mode is initiated so that the Program 120 is executed, the audience recognition unit 12, at a block 122, determines whether a predetermined amount of time has passed since the last audience member activity. If the audience recognition unit 12 determines that there has been no audience member activity for the ) predetermined amount of time, the program 120 assumes that the training mode has ended and the program 120 exits. If the predetermined amount of time has not pass ed since the last audience member activity, the audience recognition unit 12, at a block 124, continues to wait 5for a wakeup command to be spoken. The wakeup command should be a unique word or word combination not normally present in the audio of the receiver 14. The audience member should speak the wakeup command in a normal voice to the first and second microphones 30 and 32. When the ) audience recognition unit 12 detects the wakeup command, the audience recognition unit 12, at a block 126, prompts - 33 v7/v M oo PCTIUS98/12034 the audience member to say the identifier word which the viewer has chosen to identify himself or herself. Such an identifier word may be the name of the viewer. The audience recognition unit 12 then deter mines at a block 128 whether the audience member has said the identifier. If not, the audience recognition unit 12, at the block 126, again prompts the audience member to say the identifier word. If the audience recognition unit 12 determines at the block 128 that the audience ) member has said the identifier word, the audience recog nition unit 12, at a block 130, suitably processes the identifier word and stores the identifier word in the memory 48. Flow returns to the block 122 where the training mode is terminated or another audience member 5 enters a corresponding identifier word. During an identification mode, the controller 46 of the audience recognition unit 12 may implement a program 140 depicted in Figure 8 by way of a flow chart. The program 140 is executed during the identification ) mode in order to identify members of the audience. The identification mode may be initiated by pressing an - 34 - -- 7I>IUoo PCT/US98/12034 appropriate key or button on the keypad 56 of the audi ence recognition unit 12. When the identification mode is initiated so that the program 140 is executed, the audience recogni tion unit 12, at a block 142, determines whether an audi ence member desires to end the identification mode by pressing an appropriate key or button on the keypad 56 of the audience recognition unit 12. If the audience recog nition unit 12 determines that the audience member de ) sires to end the identification mode, the program 140 exits. If the audience recognition unit 12 determines that the audience member does not desire to end the iden tification mode, the audience recognition unit 12, at a block 144, waits for a wakeup command to be spoken. The Audience member should speak the wakeup command in a normal voice to the first and second microphones 30 and 32. When the audience recognition unit 12 detects the wakeup command, the audience recognition unit 12, at a block 146, prompts the audience member to say his or her ) identifier word. - 35 - VuJ YIx toon PCTUS98/12034 The audience recognition unit 12 then deter mines at a block 148 whether the audience member has said the identifier word. If not, the audience recognition unit 12, at the block 146, continues to prompt the audi 5 ence member to say the identifier word. If the audience recognition unit 12 determines at the block 148 that the audience member has said the identifier word, the audi ence recognition unit 12, at a block 150, instructs the voice recognition unit 44 to compare the identifier word 0 to the vocabulary of identifier words stored in the mem ory 48. If the audience recognition unit 12 determines at a block 152 that the voice recognition unit 44 detects a match between the spoken identifier word and one of the 5 identifier words stored in the memory 48, the audience recognition unit 12, at a block 154, stores the spoken identifier word and the time of the match as an identification record in the memory 48 for immediate or later communication to the base unit 18. (Alternatively, 0 the audience recognition unit 12, at the block 154, may communicate the spoken identifier word and time directly - 36 - SYuY/Z/oo PCTIUS98/12034 to the base unit 18.) If the audience recognition unit 12 determines at the block 152 that the voice recognition unit 44 cannot match the spoken identifier word to one of the identifier words stored in the memory 48, the audi 5 ence recognition unit 12, at a block 156, prompts the audience member to try again or to use another form of identification. For example, the prompt 140 may prompt the audience member to speak his or her identifier word a predetermined number of times before prompting the audi o ence member to use another form of identification (such as entering the identifier word through use of the keypad 56). Alternatively, or additionally, if the controller 46 determines that the voice recognition unit 44 cannot detect a match at the block 152, the controller 46 at the 5 block 156 may send a message to the base unit 18 indi cating that identification should be made from the pas sive people meter 26. After the audience recognition unit 12, at the block 154, stores the spoken identifier word and the time of the match as an identification re 0 cord in the memory 48, or after the audience recognition unit 12, at the block 156, prompts the audience member to - 37 - ".. ''o PCT/US98/12034 try again, or to use another form of identification, or to send a suitable message to the base unit 18 to seek an identification from the passive people meter 26, or the like, flow returns to the block 142. 5 The advantage of using a wakeup command during normal voice recognition is that the comparison process is simplified. That is, during normal audience recogni tion, each spoken identifier word is compared to every reference identifier word stored in the memory 48. How 0 ever, the wakeup command only needs to be compared to one reference in the memory 48. The block 126 of Figure 7 and the block 146 of Figure 8 may be arranged to also prompt each audience member to speak a cue in addition to the member's identi 5 fier word. The advantage of coupling a cue to an identi fier word is to minimize the possibility that the audi ence recognition unit 12 may incorrectly identify a per son as an audience member when that person's identifier word is contained in the audio output of the receiver 14, 0 but that person is not in the audience. For example, the identifier word may be a name, and the cue may be a word - 38 - " /'/ ooo PCTIUS98/12034 common to all audience members, such as the word "enter," or the word "exit," or the like. Thus, a member of the audience, when entering (or leaving) the monitored area, speaks his or her name, as the identifier word, coupled with the cue. For example, the member may say "John enter" or "Mary enter." Alternatively, the cue may be unique to each audience member. Either arrangement reduces the likelihood that a word audibly emitted by the receiver 14 will be mistaken as an identifier word spoken ) by a member of the audience. Certain modifications of the present invention have been discussed above. Other modifications will occur to those practicing in the art of the present invention. For example, the invention has been described 5 above in connection with measuring the audiences of tele vision programs. However, the present invention is use ful also in measuring the audiences of radio programs and programs of other media. Also, the program sensor 16 is described above ) as sensing the program and/or channel to which the re ceiver 14 is tuned. Instead, the audience recognition - 39 - >ur ooa PCTIUS98/12034 unit 12 may be arranged to accept manual entry of program identifications by the audience members through use of the keypad 56. Moreover, the functions of a remote control may 5 be incorporated into the audience recognition unit 12. Furthermore, instead of relying upon two micro phones, such as the first and second microphones 30 and 32, the audience recognition unit 12 may rely upon a single, close range directional microphone. If so, noise 0 cancellation may not be necessary. In addition, the use of a wakeup command may be eliminated. Also, as described above, the passive people meter 26 may be of the type which passively identifies 5 audience members in a monitored area in accordance with the teachings of U.S. Patent No. 5,550,928 described above. However, the passive people meter 26 may be any known type of passive device that is capable of passively identifying members of an audience. For example, the 0 passive people meter 26 may use an ultrasonic and/or infrared transducer or an array of such transducers in - 40 - 7 D/ /Dueo PCT/US98/12034 order to capture an image of the audience area and may use processing algorithms in order to recognize members in the audience. Moreover, the passive people meter 26, as des 5 cribed above, passively identifies audience members in a monitored area. If the passive people meter 26 cannot passively identify the members in the audience, it can be arranged to count the members of the audience and to prompt manual identification of the members until the 0 number of manually identified members equals the count. Alternatively, if the passive people meter 26 cannot passively identify every member in the audience, it can be arranged to count the members of the audience and to prompt manual identification of those members which it 5 cannot passively identify. As a still further alterna tive, a counter may be used instead of the passive people meter 26. The counter can comprise an ultrasonic and/or infrared transducer or an array of such transducers or other device or devices which can detect people in the 0 audience and can count the number of detected people. Then, the audience measurement system 10 could be ar - 41 uVVVf / /oon PCT/US98/12034 ranged to prompt manual identification until the number of people manually identified equals the count. Furthermore, the audience measurement unit 12, the program sensor 16, and the passive people meter 26 5 are shown in Figure 2 as being separate elements. In stead, the audience measurement unit 12 may include the program sensor 16 and/or the passive people meter 26. Accordingly, the description of the present invention is to be construed as illustrative only and is .0 for the purpose of teaching those skilled in the art the best mode of carrying out the invention. The details may be varied substantially without departing from the spirit of the invention, and the exclusive use of all modifica tions which are within the scope of the appended claims is reserved. - 42 -

Claims (11)

17. The system of claim 1 wherein the voice signal comprises an identifier and a cue of the member of the audience.

18. The system of claim 1 further comprising a passive people meter arranged to passively identify the member of the audience.

19. The system of claim 1 further comprising a communication unit arranged to communicate the identity of the member and/or program identity to a remote point.

20. The system of claim 19 wherein the communication unit is arranged to communicate the iden tity of the member to the remote point by way of infrared signals.

21. The system of claim 1 wherein the first and second microphones, the noise cancellation unit, and - 47 - Yo J/0oo5 PCT/US98/12034 3 the voice recognition unit are housed in a hand held housing. 1 22. The system of claim 1 wherein the first 2 and second microphones, the noise cancellation unit, and 3 the voice recognition unit are housed in a table top housing.

23. The system of claim 1 wherein the first and second microphones are directionally oriented in substantially opposite sensing directions.

24. The system of claim 1 wherein the first microphone is mounted closer to the audience than is the second microphone, and wherein the second microphone is mounted closer to the receiver than is the first micro phone. 1 25. The system of claim 1 further comprising a 2 counter, a display, and a manual entry device, wherein 3 the voice recognition unit is arranged to supply prompts - 48 - " 1 ' UUo PCT/US98/12034 4 to the display in order to prompt manual identification 5 of members of the audience through use of the manual 6 entry device until the members so identified equal a count supplied by the counter. 1 26. A system for identifying a member of an 2 audience of a program to which a receiver is tuned, the 3 system comprising: 4 a microphone located to receive a voice signal 5 from the member; 6 a voice recognition unit coupled to the micro 7 phone and arranged to identify the member from the voice 8 signal; and, 9 a sensor arranged to identify the program or channel. 1 27. The system of claim 26 further comprising 2 a memory arranged to store reference identification 3 information corresponding to the member, and wherein the 4 voice recognition unit is arranged to identify the member - 49 - V" 7~IUOo PCT/US98/12034 5 based upon a comparison of the voice signal and the ref erence identification information. L 28. The system of claim 27 wherein the sensor is arranged to identify the program based upon a feature of the program. L 29. The system of claim 28 wherein the feature is an identification code embedded in the program. L 30. The system of claim 28 wherein the feature 2 is an uniquely identifiable signature extracted from the program. L 31. The system of claim 26 wherein the re ceiver is a television. L 32. The system of claim 26 wherein the re ceiver is a radio. - 50 - VV / /oo PCT/US98/12034 L 33. The system of claim 26 wherein the re ceiver is a multimedia reception device. L 34. The system of claim 26 further comprising a keypad arranged to accept an identification input from the member. L 35. The system of claim 34 wherein the identification input is an identification of the member. 1 36. The system of claim 34 wherein the 2 identification input is a program or channel identifica tion. 1 37. The system of claim 26 further comprising 2 a prompting device arranged to prompt the member to say the voice signal. 1 38. The system of claim 37 further comprising 2 a passive people meter arranged to passively identify the member. - 51 - " f 'o PCT/US98/12034 1 39. The system of claim 26 wherein the voice signal comprises an identifier of the member. 1 40. The system of claim 26 wherein the voice signal comprises an identifier and a cue for the member. 1 41. The system of claim 26 further comprising 2 a passive people meter arranged to passively identify the member. 1 42. The system of claim 26 further comprising 2 a communication unit arranged to communicate the identity of the member to a remote point. 1 43. The system of claim 42 wherein the 2 communication unit is arranged to communicate the iden 3 tity of the member to the remote point by way of infrared signals. - 52 - S 7oo1000 PCTIUS98/12034 1 44. The system of claim 26 wherein the micro 2 phone and the voice recognition unit are housed in a hand held housing. L 45. The system of claim 26 wherein the micro 2 phone and the voice recognition unit are housed in a table top housing. L 46. The system of claim 26 further comprising 2 a counter, a display, and a manual entry device, wherein 3 the voice recognition unit is arranged to supply prompts I to the display in order to prompt manual identification D of members of the audience through use of the manual entry device until the members so identified equal a count supplied by the counter. - 53 - YVn >>ooo PCT/US98/12034 1 47. A system for identifying a member of an 2 audience of a program to which a receiver is tuned, the 3 system comprising: 4 a passive people meter arranged to provide a 5 first identification of the member by passively identify 6 ing the member; 7 a microphone located to receive a voice signal 8 from the member; 9 a voice recognition unit coupled to the micro 0 phone and arranged to provide a second identification of 1 the member by identifying the member from the voice sig 2 nal; and, 3 a controller arranged to select at least one of 4 the first and second identifications in order to identify the member. 1 48. The system of claim 47 wherein the con 2 troller includes a prompting unit arranged to prompt the member to say the voice signal. - 54 - " ''o PCT/US98/12034 L 49. The system of claim 48 further comprising a sensor arranged to identify the program or channel. L 50. The system of claim 47 wherein the con 2 troller includes a prompting unit arranged to prompt the 3 member to say the voice signal if the passive people meter is unable to provide the first identification. L 51. The system of claim 47 wherein the con 2 troller is arranged to select the first identification if 3 the voice recognition unit is unable to provide the sec ond identification. L 52. The system of claim 47 wherein the voice signal comprises an identifier. L 53. The system of claim 47 wherein the voice signal comprises an identifier and a cue. L 54. The system of claim 47 further comprising a sensor arranged to identify the program or channel. - 55 - Vo notoo PCT/US98/12034 L 55. The system of claim 47 further comprising 2a communication unit arranged to communicate the identity of the member to a remote point. L 56. The system of claim 55 wherein the communication unit is arranged to communicate the iden 3tity of the member to the remote point by way of infrared signals. L 57. The system of claim 47 wherein the micro 2phone and the voice recognition unit are housed in a hand held housing. L 58. The system of claim 47 wherein the micro ? phone and the voice recognition unit are housed in a table top housing. L 59. The system of claim 47 further comprising 2 a counter, a display, and a manual entry device, wherein 3the controller is arranged to supply prompts to the dis 1play in order to prompt manual identification of members - 56 - "y /'ooo PCTIUS98/12034 5 of the audience through use of the manual entry device until the members so identified equal a count supplied by the counter. L 60. An audience recognition unit that identi 2 fies a member of an audience of a program to which a 3 receiver is tuned, the audience recognition unit compris I ing: 5 a housing; D a microphone supported by the housing; and, 7 a voice recognition unit supported by the hous 3 ing and coupled to the microphone, the voice recognition unit being arranged to recognize the member from an iden tifier spoken by the member. L 61. The audience recognition unit of claim 60 further comprising a prompt unit supported by the hous ing, wherein the prompt unit is arranged to prompt the member to take an action. - 57 - S 1oo00 PCT/US98/12034 1 62. The audience recognition unit of claim 61 2 further comprising a keypad supported by the housing, 3 wherein the keypad is arranged to permit the member to enter data. 1 63. The audience recognition unit of claim 62 2 further comprising a communication unit supported by the 3 housing, wherein the communication unit is arranged to communicate data to a base unit. 1 64. The audience recognition unit of claim 63 2 further comprising a clock and a memory supported by the 3 housing, wherein the memory stores an identity of the 4 member based upon the spoken identifier, and wherein the 5 memory stores a time from the clock when the member is identified. 1 65. The audience recognition unit of claim 64 wherein the identifier includes a word and a cue. - 58 - nu'- 7 oooD PCT/US98/12034

66. The audience recognition unit of claim 60 further comprising a keypad supported by the housing, wherein the keypad is arranged to permit the member to enter data.

67. The audience recognition unit of claim 66 further comprising a communication unit supported by the housing, wherein the communication unit is arranged to communicate data to a base unit.

68. The audience recognition unit of claim 60 further comprising a communication unit supported by the housing, wherein the communication unit is arranged to communicate data to a base unit.

69. The audience recognition unit of claim 60 further comprising a clock and a memory supported by the housing, wherein the memory stores an identity of the member based upon the spoken identifier, and wherein the memory stores a time from the clock when the member is identified. - 59 - " "' 'wo PCTUS98/12034 L 70. The audience recognition unit of claim 60 wherein the identifier includes a word and a cue. L 71. The audience recognition unit of claim 60 wherein the audience recognition unit is a hand held unit. L 72. The audience recognition unit of claim 60 wherein the audience recognition unit is a table top unit. - 60 -