JPH0935067A - Method and device for discriminating television audience - Google Patents

Abstract

PROBLEM TO BE SOLVED: To enable effective measurement to be started even unless a dictionary of some individual is prepared and to generate the ungenerated dictionary by gathering a face image of the individual during the measurement. SOLUTION: The television audience begins to be measured while the dictionary for image recognition as to at least one of all individuals to be discriminated is not generated yet. An audience discriminating means 105 specifies the individual corresponding to a face image decided to be a guest by an image recognizing means 103 according to the statement from the individual which is inputted through a registration box 700 having a button corresponding to the specific individual one to one and a face image gathering means 108 stores the guest face image in a reference face image group storage means 110-1, etc., when the guest face image is a face image of the individual whose dictionary is not generated yet. A dictionary generating means 11 generates the dictionary for the said individual from the stored face image and the dictionary is used for the image recognition of the image recognizing means 103.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a television viewer identification method and apparatus for identifying an individual who is watching a television receiver by face image recognition in each surveyed household for a television audience rating survey.

[0002]

2. Description of the Related Art A television audience rating is an individual who takes into account the household audience rating that simply indicates the television viewing status installed in the surveyed household and information about who is actually watching the surveyed household. There is an audience rating, and recently, the demand for the latter individual audience rating has been increasing year by year.

The methods for measuring the personal audience rating are roughly classified into active type and passive type. The active type is a method in which the surveyee operates the pushbuttons etc. assigned to him / her when starting and ending the viewing.
Since the survey subject will be forced to perform an unnecessary act, the burden on the survey subject tends to be heavy in the case of a long-term survey.

On the other hand, the passive type is a system for automatically judging the person watching the television, and the burden on the person to be surveyed is remarkably reduced as compared with the active type. In particular, the person watching TV is imaged with a video camera,
The method of identifying the viewer by performing image recognition processing on the image data, especially the face image, has been actively developed and researched because the personal information being viewed is automatically obtained every short time.

For example, in Japanese Patent Application No. 4-341350 filed by the applicant of the present application, a binary image is obtained by performing nOFm value conversion on a multi-gradation face image obtained by imaging a monitoring area. By generating and calculating the characteristic data from the binarized image, and performing the discriminant analysis of the calculated characteristic data based on the reference data of the discriminant analysis held in the dictionary, the individual who is watching is identified. A device has been proposed.
In addition, Japanese Patent Application No. 6-3 for the applicant's own application
No. 3031 discloses a television viewer identifying apparatus for identifying an individual who is watching by image recognition technology and for a face image for which an individual cannot be identified by image recognition, identifying the individual from a tracking result of the viewing place. Is proposed.
Further, in Japanese Unexamined Patent Publication No. 7-95625, two different types of template matching score and alien face confirmation score are obtained by comparing a face image of each individual in an image obtained by imaging a monitoring area with a reference file. After giving a score indicating the degree of similarity, the scores are fused and the individual being viewed is identified by using the matching of body shapes, the tracking result of the viewer, the progress data, etc. as an aid for viewer identification. Such a device has been proposed.

[0006]

By the way, in the identification of a television viewer by this kind of face image recognition, it is determined by a dictionary (also referred to as a reference file etc.) which person the face image in the captured image belongs to. It is called). Therefore, it is necessary to prepare a predetermined dictionary for each individual before starting the measurement, and for that purpose, a face image of each individual serving as a reference is required. However, because the family composition of the surveyed household and the work of each individual vary widely, it is not always possible to obtain facial images of all the households by the time you want to start the survey, which is why There is a problem that the start of is delayed. It may be possible to entrust the surveyed households themselves with the task of registering their own facial images and the task of creating a dictionary, but this would be a burden to the surveyed households because it requires a relatively complicated procedure.

Therefore, an object of the present invention is to enable the measurement to be effectively started even when some personal dictionaries are not prepared, and to create an uncreated dictionary during the measurement. It is to be able to automatically collect the reference face images of.

[0008]

SUMMARY OF THE INVENTION In order to achieve the above object, the present invention provides a feature of an individual face image obtained by periodically imaging a viewer present in a surveillance area in front of a television receiver. A television viewer who identifies from the characteristics of the face image of a predetermined individual stored in the dictionary which one of the predetermined individuals the individual who is watching the television receiver by image recognition processing. In the identification method, the measurement of the television viewer is started in a state where the dictionary is not created for at least one individual among all the individuals to be identified, and the image recognition processing is performed to determine the The guest face image determined to be none of the persons is tried to be identified based on the declaration from the individual input via the registration box, and the guest face image is not yet created in the dictionary among the predetermined individuals. Individual If it can be identified to be a face image so that to save the face image as a reference face image of the person. When a certain number of reference face images of an individual for which a dictionary has not yet been created are saved, a dictionary for that individual is created using the saved reference face images and is used for the image recognition processing.

Further, the television viewer identifying apparatus of the present invention comprises:
In order to achieve the above-mentioned object, an image pickup device that periodically picks up a surveillance area in front of a television receiver, and a face image extraction that extracts a personal face image from the images picked up by this image pickup device Means, a dictionary storage means for storing a dictionary created based on a predetermined face image group for each individual, and a created dictionary for each face image extracted by the face image extraction means. Attempt to identify an individual by using the image recognition unit that assigns the individual number to the face image that can be identified and processes the face image that cannot be identified as a guest face image, and views from individuals who have not created a dictionary. A registration box that accepts reports during medium and non-viewing, a viewer identification unit that identifies the individual who is viewing based on the recognition result of the image recognition unit and the declaration from the individual input via the registration box, The image When the face image processed as the guest face image by the identification means can be identified as the face image of the predetermined individual by the viewer identification means, the dictionary of the identified individual has not been created and When the number of reference face images for creating the dictionary has not reached a certain number, the guest face image specified as the individual is used as the reference face image of the individual, and the reference face image group storage unit corresponding to the individual The face image collection means to be registered in and a personal dictionary for which a dictionary has not yet been created, creates a personal dictionary based on a fixed number of reference face images registered in the reference face image group storage means, and stores the dictionary. The dictionary creating means to be stored in the means and the identification of the viewer identifying means until the dictionary storage means stores all the dictionaries of all the individuals to be identified as the identification result of the individual being viewed. Select fruit, after all the dictionaries stored and a discrimination result selection means for selecting an identification result of the image recognition means.

[0010]

In the television viewer identification method of the present invention, the measurement of the television viewer is started in a state where the dictionary for at least one individual among all the individuals to be identified has not been created. Therefore, when an individual for whom a dictionary has not been created is watching, the individual is not determined as a predetermined individual by the image recognition processing, and is processed as a guest face image. In the present invention, the guest face image determined to be none of the predetermined individuals by the image recognition process is attempted to be identified based on the declaration from the individual input via the registration box. Then, when the guest face image can be identified as the face image of the individual for which the dictionary has not been created among the predetermined individuals, the face image is saved as the reference face image of the individual, and a certain number of images are saved. Then, a personal dictionary is created with the stored reference face image, and the dictionary is provided for image recognition processing.

In the television viewer identifying apparatus of the present invention which implements such a method, the image pickup device periodically picks up the surveillance area in front of the television receiver, and the face image extracting means obtains the image. The face image of the individual is extracted from the captured images, and the image recognition unit attempts to identify the individual for each extracted face image using the dictionary that has been created. A face image that is given a number and cannot be identified is processed as a guest face image. Further, the registration box accepts viewing and non-viewing declarations from individuals who have not created a dictionary. Then, the viewer identifying means identifies the individual who is watching based on the recognition result of the image recognizing means and the declaration from the individual input via the registration box, and the identifying result selecting means identifies all the objects to be identified. The identification result of the viewer identifying means is selected until the personal dictionary of (1) is stored in the dictionary storing means.

In parallel with the above operation, when the viewer identifying means identifies the face image processed as the guest face image by the image recognizing means as a face image of a predetermined individual, the face image collecting means. However, if the identified personal dictionary is not yet created and the number of reference face images for creating the individual dictionary has not yet reached a certain number, the guest face images identified as those of the individual are The reference face image of an individual is registered in the reference face image group storage unit corresponding to the individual. Then, the dictionary creation means creates a personal dictionary for an individual for whom a dictionary has not been created yet, based on a fixed number of reference face images registered in the reference face image group storage means. As a result, the image recognition means uses all the dictionaries including the newly created dictionary to proceed the recognition process of the viewing individual by the image recognition, and the identification result selection means,
The identification result of the image recognition means is adopted as the identification result of the individual who is viewing.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, embodiments of the present invention will be described in detail with reference to the drawings.

Referring to FIG. 1, one embodiment of the television viewer identification apparatus of the present invention is a processing apparatus 100 and an imager 20.
0, a channel sensor 300, a terminal controller 400, an input device 500, a video tape recorder 600, a registration box 700, and a remote controller 800. Further, the processing device 100
Is a face image extraction means 101 and an extracted face image storage means 10
2, image recognition means 103, and recognition result storage means 104
, Viewer identification means 105, and identification result selection means 106
A viewing state recording means 107 and a face image collecting means 108
And a dictionary 10 corresponding to each person to be identified one to one
Dictionary storing means 109 for storing 9-1 to 109-n,
Similarly, it is provided with reference face image group storage means 110-1 to 110-n corresponding to each individual to be identified, a dictionary creation means 111, a registration reference means 112, and an initial face image registration means 113. ing. The processing device 100 can be realized by a personal computer or the like incorporating an image processing board and predetermined software.

The registration box 700 is a means for accepting viewing and non-viewing declarations from an individual who has not created a dictionary, and has a one-to-one correspondence with an individual to be identified on its front panel.
, Push-buttons 701-1 to 701-n and a dictionary creation completion lamp 702 are provided, and are installed on, for example, a television receiver. The family number is written on the push button.

FIG. 2 shows a configuration example of the registration box 700. The control unit 703 controls the push buttons 701-1 to 701-n.
Push button 701-
Latches 706-1 to 1-701 corresponding to 1 to 701-n
The contents of 706-n are alternately inverted to "1" and "0". The contents of these latches 706-1 to 706-n are
The signal is similarly inverted by the signal from the remote controller 800. That is, every time a button (not shown) corresponding to each person provided on the remote controller 800 is pushed, the remote control receiver 704 receives a button signal transmitted as an infrared signal, for example. ,
Of the latches 706-1 to 706-n, the content of the latch corresponding to the received button signal is inverted. The contents of these latches 706-1 to 706-n are input to one of the input terminals of the changeover switches 707-1 to 707-n, and also sent to the selector 708.

In addition, the push buttons 701-1 to 701n have one
Another pair of latches 705-1 to 705 corresponding to one to one
n. These latches 705-1 to 705-n are
This is for holding the recognition result of the image recognition means 103. The control unit 703 periodically reads the recognition result output from the image recognition unit 103 from the recognition result storage unit 104, and "1" is being viewed in the latch corresponding to the individual recognized as being viewed. "0" is set to the latch corresponding to the person who is not recognized as. These 705-
The contents of 1-705-n are the changeover switches 707-1-
It is input to the other input terminal of 707-n.

Changeover switches 707-1 to 707-n
Is a switch that selects either one of the two sets of latches and guides its contents to the illumination drive circuits 709-1 to 709-n that drive the illuminators 710-1 to 710-n. The illumination drive circuits 709-1 to 709-n turn on the illuminators 710-1 to 710-n during the period when the output of the changeover switches 709-1 to 709-n is "1", and during the period of "0". , Turn off the light. Illuminator 710-1 to 710-n
Is for turning on or off the pushbuttons 701-1 to 701-n.

Latches 705-1 to 705-change switches 707-1 to 707-n corresponding to individual pushbuttons 701-1 to 701-n for which a dictionary is created at the start of measurement.
Switch to the n side and push button 701-
Changeover switch 707-1 corresponding to 1-701-n
..- 707-n are switched to the latches 706-1 to 706-n side, the pushbuttons of the personalized dictionary are turned on and off according to the recognition result of the image recognition means 103, and the dictionary is turned on. The uncreated individual pushbutton will be turned on and off according to the individual's declaration. Also, the selector 7
Latch 7 corresponding to a personal push button for which a dictionary has not been created in 08
By setting in advance to select the output of 06-1 to 706-n, it is possible to take out the reporting information of viewing and non-viewing from an individual who has not created a dictionary. This report information is transmitted to the viewer identifying means 105.

The face image collecting means 10 will be described later.
Upon receipt of the dictionary creation completion notification from the controller 8, the control unit 703 sets the content of the latch 711 to “1” and sets the lamp drive circuit 71.
By turning on the dictionary creation completion lamp 702 in step 2, the fact that the personal dictionary that has not been created at the start of measurement is created during the measurement is displayed outside.

The operation of this embodiment thus constructed will be described below.

(1) Creation of initial dictionary When starting actual measurement, a dictionary for each individual is created based on the face image of a predetermined individual to be identified. When visiting the surveyed households, if all are at home, it is possible to obtain the facial image of each individual by shooting all the individuals on the spot, but this is not the only case. Therefore, in this embodiment, the household to be surveyed is given an 8 mm video camera (or a digital still camera may be used in advance) to have the individual's face photographed in advance, and the video tape is played back to create a dictionary. The method of initial creation is adopted. However, face images of some individuals may not be obtained even by such a method due to circumstances, and therefore, in the initial creation of the dictionary, only the dictionary corresponding to the individual who has obtained the face images is created.

First, a video tape on which a face image of an individual is recorded is set on the video tape recorder 600, one face image is reproduced for each individual, and a family number is input from an input device 500 such as a keyboard. Instruct the registration to the initial face image registration means 113 of the processing device 100. The initial face image registration means 113 includes a plurality of reference face image group storage means 110.
-1 to 110-n, one unused reference face image group storage means is secured, and after registering the family number input from the input device 500, the face of the individual reproduced by the video tape recorder 600. The images are stored in the reference face image group storage means. In the case of the present embodiment, the reference face image group storage means 1
Since 25 reference face images are stored in each of 10-1 to 110-n, the above operation is repeated,
Register 25 face images of each individual. Also,
For individuals whose face images could not be obtained, input device 5
Specify the fact from 00 and enter only the family number. At this time, the initial face image registration means 113 secures one unused reference face image group storage means and registers the family number input from the input device 500 therein.

Next, when the dictionary creation means 111 is instructed from the input device 500 to specify the family number of an individual who has finished registration of a predetermined number of reference face images, the dictionary creation means 111 causes the reference face image group storage means 110. A dictionary for each individual used by the image recognition means 103 is created based on the individual reference face image stored in the reference face image group storage means having the designated family number among −1 to 110-n, It is stored in the dictionary storage means 109. The contents of the created dictionary differ depending on the recognition method by the image recognition means 103. In the case of the present embodiment, an individual is identified (determined) by a discriminant analysis method similar to that in Japanese Patent Application No. 4-341350 and Japanese Patent Application No. 6-33031 according to the applicant's own application. The features of the face image for each category (each individual) are registered. The created dictionary also includes the family number.

Next, of the changeover switches 707-1 to 707-n of the registration box 700 shown in FIG. 2, the changeover switch corresponding to the individual who created the dictionary is the latch 7
The switch is switched to the 05-1 side, and the changeover switch corresponding to the person for whom the dictionary is not created is switched to the latches 706-1 to 706-n side. Further, the selector 708 is set so that the outputs of the latches 706-1 to 706-n corresponding to individuals for which a dictionary has not been created are selected.

After the above processing is completed, the input device 50
From 0, the personal family number of the dictionary not created is processed 100
The face image collecting means 108 is notified and the measurement is started. The face image collection means 108 internally stores the personal family number for which a dictionary has not been created, and instructs the identification result selection means 106 to select the identification result of the viewer identification means 105.

In the above embodiment, the initial dictionary was created using the face images of the individuals recorded in advance. However, the image of the face of each individual was taken by the image pickup device 200, and the taken images were taken. An initial dictionary can be created by registering the inside face images in the reference face image group storage means.

(2) Measurement of viewing condition in a state where part of dictionary is not created The image pickup device 200 periodically picks up an image of the monitoring area in front of the television receiver (not shown) from the television side with a wide-angle lens, and picks up the image. The obtained image is transmitted to the face image extraction means 101 of the processing device 100. The image pickup device 200 includes, for example, an infrared video camera and an A / D converter that converts an analog video signal output from the infrared video camera into a digital video signal.

Face image extraction means 101 of the processing apparatus 100
Cuts out a face image of an individual from the images picked up by the image pickup device 200 and stores it in the extracted face image storage means 102.

The image recognizing means 103 inputs the face images stored in the extracted face image storing means 102, and features of each face image and a predetermined face registered in the dictionary stored in the dictionary storing means 109. Based on the characteristics of the face image of the individual, it is recognized which of the predetermined individuals the individual who is watching the television receiver is, and the recognition result is stored in the recognition result storage means 10.
4 is output. In the case of this embodiment, each recognition result includes
The face image itself subjected to the recognition process, the family number assigned to the individual when the individual can be identified, and the guest number indicating a guest when the individual cannot be identified are included.

The contents of the recognition result storage means 104 are referred to by the registration box 700, the viewer identification means 105, the identification result selection means 106 and the face image collection means 108.

In the registration box 700, as described with reference to FIG. 2, of the pushbuttons 701-1 to 701-n, the pushbutton corresponding to the dictionary-created individual is output to the recognition result storage means 104. The on / off state is controlled according to the recognition result. Further, the lighting and extinguishing states of the other pushbuttons are controlled according to the operation of the pushbuttons and the report from the individual by the remote controller 800.

On the other hand, the viewer identifying means 105 identifies the individual who is watching based on the recognition result by the image recognition in the recognition result storage means 104 and the declaration from the individual input via the registration box 700. The viewer identification means 1 is shown in FIG.
An example of the processing of 05 is shown. The viewer identifying unit 105 first determines whether or not the recognition result storage unit 104 includes the recognition result processed as a guest (S1), 1
If none exists (NO in S1), the current recognition result output to the recognition result storage means 104 is output to the identification result selection means 106 as an identification result (S2).

On the other hand, when the recognition result processed as a guest is included (YES in S1), the number of recognition results processed as guests, that is, the number of guests and the fact that the user is watching through the registration box 700 is declared. It is determined whether or not the number of people is equal (S3). When the number of guests is equal to the number of viewers according to the declaration (YES in S3), the identification result in which the guest number of the recognition result determined to be a guest among the recognition results in the recognition result storage unit 104 is replaced with the family number of the reporter Is output to the identification result selection means 106 (S4).
At this time, if the number of guests is one (Y in S5)
ES), the face image recognized as the guest is identified as the face image of the filer, the face image collection unit 108 is notified of the identified family number, and the face image collection is instructed (S6). ).

When the number of guests is different from the number of viewers according to the declaration (NO in S3), when the number of guests is larger than the number of viewers according to the declaration (YES in S7), some individuals may omit the declaration. Of the recognition results determined to be guests in the recognition result storage means 104, the identification result in which the guest number of the recognition result for the number of viewers by the declaration is replaced with the family number of the filer is used as the identification result selecting means 106. Output (S8). On the other hand, when the number of guests is smaller than the number of viewers reported (NO in S7), it is judged that some individuals have not watched but forgot to report that they are not watching, whichever Since it is not possible to determine whether or not the person has made a declaration, the recognition result selection means 10 uses the recognition result itself output to the recognition result storage means 104 as the recognition result.
It outputs to 6 (S2).

Since the identification result selection means 106 is instructed by the face image collection means 108 to select the viewer identification means 105 side, the identification result output from the viewer identification means 105 is selected and the viewing situation is selected. Output to the recording means 107.

The viewing situation recording means 107 inputs the identification result output from the identification result selecting means 106, and creates viewing personal information indicating which individual or guest is watching at that time. In addition, a viewing situation record is created and accumulated by adding the on / off state of the television receiver measured by the well-known channel sensor 300 and the channel information during channel selection.

The terminal controller 400 sends the viewing situation record accumulated in the viewing situation recording means 107 to a center (not shown) through a telephone line or the like.

(3) Collection of Face Images and Creation of Dictionary During Measurement of Viewing Situation with Partially Uncreated Dictionary FIG. 4 and FIG. 5 are flowcharts showing an example of the processing of the face image collecting means 108. As described above, when the face image collection unit 108 is notified of a family number for which a dictionary has not been created after the dictionary has been created by the input device 500, the face image collection unit 108 stores it internally (S11), and sends it to the identification result selection unit 106. An instruction is issued to select the viewer identification means 105 side (S1).
2). After that, the face image collection unit 108 waits for a face image collection instruction from the viewer identification unit 105 (S1).
3).

When a face image collection instruction is received from the viewer identifying means 105 (YES in S13), the family number (viewer identifying means 105 notified by the individual through the registration box 700 is the only guest notified at the same time. It is checked whether the determined family number) is included in the currently stored dictionary-uncreated family number (S14). If not included,
Since the face images of the individual with the family number have already been collected, the current collection instruction is ignored and the process returns to step S13 to wait for the next collection instruction.

When the notified family number is one of the family numbers for which a dictionary has not been created (YES in S14), it is determined that the family number is stored in the recognition result storage means 104 and is an individual. Enter the guest face image (S1
5) Check whether the guest face image is appropriate as a reference face image. First, a head shape check for checking the degree of inclination of the head of the guest face image and a face inside area check for checking the degree of the face facing sideways are performed (S16, S18). This is because a face image whose head is tilted too much is not suitable for the reference face image, and a face image whose face is turned sideways is not suitable for the reference face image, and therefore such a face image is excluded. In addition, the head shape check is generally a human face (head).
Has an oval shape, it is performed by comparing the inclination angle of the long axis of the oval shape with a predetermined angle α. Also, if the face faces the front, the members (eyes, nose, mouth, etc.) in the face internal region are substantially line-symmetric with respect to the long axis of the egg shape, and the check is performed based on that. As a result of the above checks, if it is confirmed that the degree of inclination of the head exceeds the allowable range or the orientation of the face exceeds the allowable range (NO in S17, S1
(NO in 9), and returns to step S13 to wait for the next collection instruction.

On the other hand, when the tilt angle of the head and the orientation of the face are within the permissible range (YES in S19), the registration referring means 11
The similarity is checked by performing template matching between the face image already collected in the reference face image group storage means corresponding to the family number of this time and the guest face image of this time via 2 (S).
20). If there are many face images that have already been collected, the average degree of similarity with any number of face images may be averaged, or the average degree of similarity with all face images may be examined. May be. Then, if the similarity between the guest face image this time and the face image already collected is not within the allowable range (NO in S21), the process returns to step S13. this is,
This is to prevent the facial image of another person from being mixed in due to an error in reporting.

As a result of the above investigation, if the similarity of the guest image of this time is within the allowable range with respect to the already collected face images (YES in S21), the face image of this time is a standard corresponding to the family number. It is registered in the face image group storage means through the registration reference means 112 (S22). In the case of the first face image to be collected, no face image has already been collected, so the similarity degree is not checked in step S20, and
At 21, the image is considered to be within the allowable range and registered in the reference face image group storage means.

Next, it is determined whether or not the number of collected sheets reaches a predetermined number, that is, 25 sheets in the present example (S23), and if not, the process returns to step S13. On the other hand, if the predetermined number has been collected (YES in S23), the current family number is removed from the family numbers in the dictionary that have not been created (S24). Then, it is determined whether or not there is no family number for which a dictionary has not been created (S25).
O), and returns to step S13 to wait for the next collection instruction.

On the other hand, when the collection of the face images this time eliminates the family numbers for which no dictionary has been created, that is, when a predetermined number of reference face images have been collected for all individuals for whom no dictionary has been created (S
If YES in 25), the process proceeds to dictionary update control.

In this embodiment, the dictionary is updated at the timing when the television receiver is turned off. Therefore, the face image collecting means 108 has a channel sensor 30.
When the ON / OFF signal of the television receiver is input from 0, and the face image collection means 108 detects the timing when the television receiver is turned off (YES in S26), the dictionary creation means 111 creates a dictionary. (S2
7).

The dictionary creating means 111 is the face image collecting means 1
When the dictionary is instructed from 08, the reference face image group storage means 110-1 to 110-n in which the family number is registered
Based on the above, all dictionaries 109-1 to 109-n are recreated by the same method as the initial dictionary creation and stored in the dictionary storage means 109, and all the previously stored dictionaries are erased. Note that all dictionaries, including already created dictionaries, are created in the case of dictionaries used in the discriminant analysis method. When using dictionaries by other methods, uncreated dictionaries are used. Only need to create.

When the dictionary creating means 111 creates a dictionary for all individuals to be identified, the face image collecting means 108 selects the image recognition means 103 side with respect to the identification result selecting means 109. Instruct (S28). Further, in order to notify the outside that the creation of all personal dictionaries has been completed, the registration box 700 is notified and the dictionary creation completion lamp 702 is turned on (S29). With this, the face image collecting means 108 has finished its role, and the processing is ended.

By the operation of the face image collecting means 108 as described above, the dictionary of the individual who has not been created at the time of starting the measurement is automatically created based on the face image of the individual which is automatically collected during the measurement. It was done.

(4) Measurement of viewing situation after completion of dictionary When all personal dictionaries 109-1 to 109-n have been created, the fact is notified to the outside by turning on the dictionary creation completion lamp 702. In this state, the registration box 700 is no longer needed, and the registration box 700 is removed by an individual in the household or a person in charge of the research company. The viewer identification means 105 stops its function when the registration box 700 is removed.

Even after the dictionary is completed, the image pickup device 200 periodically picks up the monitoring area, and the face image extraction means 101 cuts out a face image of an individual from the picked-up images and stores it in the extracted face image storage means 102. To do. Image recognition means 103
The person inputs this extracted face image and tries to recognize an individual from the features of each face image and the features of the face image of a predetermined individual registered in the dictionary. In this case, the dictionary storage means 1
Since the dictionaries 109-1 to 109-n of all individuals are stored in 09, the image recognizing means 103 uses all of these dictionaries and the individual who is watching the television receiver out of the predetermined individuals. The recognition result is output to the recognition result storage means 104. Since the identification result selection unit 106 is instructed by the face image collection unit 108 to select the image recognition unit 103 side, the recognition result output from the image recognition unit 103 to the recognition result storage unit 104 is being viewed. Adopted as a result of individual identification,
It is output to the viewing situation recording means 107.

The viewing condition recording means 107 inputs the identification result output from the identification result selecting means 106 to create personal information during viewing, and the on / off state of the television receiver from the channel sensor 300 and the selection. Creates and stores viewing status records by adding channel information of stations. The terminal controller 400 uses the viewing status recording means 107.
The viewing situation record accumulated in the above is sent to a center (not shown) through a telephone line or the like.

The embodiments of the present invention have been described above.
The present invention is not limited to the above embodiments, and various other additions and changes are possible. For example, in the above-described embodiment, when the reference face images for creating the uncreated dictionary are collected, the process proceeds to the step of automatically creating the dictionary. However, the dictionary based on the collected reference face image group is created from the input device 500. It can also be done at the time of the instruction. Further, as shown in Japanese Patent Application No. Hei 6-33031, which is the application of the present applicant, a television capable of comprehensively identifying an individual who is watching by combining image recognition technology and tracking technology of the individual who is watching It is also applicable to a viewer identification device. At that time, regardless of whether the TV receiver is on or off, the individual is tracked all day long, and even if the facial image cannot be confirmed as a predetermined individual by image recognition, it is judged as a predetermined individual from the past tracking results. If possible, the identification accuracy can be increased by identifying the person with the predetermined person.

[0054]

As described above, according to the present invention, the following effects can be obtained.

In order to identify the viewer based on the viewer identification based on the image recognition using the dictionary, the declaration from the individual entered through the registration box when the guest is determined is used as an aid for the viewer identification. , It is possible to start the measurement even when some personal dictionaries are not prepared.

If the guest face image can be identified as a face image of an individual for whom a dictionary has not yet been created, that face image is saved as a reference face image for that individual. The reference face image can be automatically collected during the measurement.

When a certain number of reference face images of an individual for which a dictionary has not yet been created are saved, a dictionary for that individual is created using the saved reference face images for image recognition processing. Viewers can be identified without making a declaration.

[Brief description of drawings]

FIG. 1 is a block diagram of an embodiment of a television viewer identification device of the present invention.

FIG. 2 is a diagram showing a configuration example of a registration box.

FIG. 3 is a flowchart showing a processing example of a viewer identifying unit.

FIG. 4 is a flowchart showing a part of a processing example of a face image collecting unit.

FIG. 5 is a flowchart showing the remaining part of the processing example of the face image collecting means.

[Explanation of symbols]

 100 ... Processing device 101 ... Face image extraction means 102 ... Extracted face image storage means 103 ... Image recognition means 104 ... Recognition result storage means 105 ... Viewer identification means 106 ... Identification result selection means 107 ... Viewing situation recording means 108 ... Face image Collection means 109 ... Dictionary storage means 109-1 to 109-n ... Dictionary 110-1 to 110-n ... Reference face image group storage means 111 ... Dictionary creation means 112 ... Registration reference means 113 ... Initial face image registration means 200 ... Imaging Container 300 ... Channel sensor 400 ... Terminal controller 500 ... Input device 600 ... Video tape recorder 700 ... Registration box 701-1 to 701-n ... Push button 702 ... Dictionary creation completion lamp 800 ... Remote controller

Claims (3)

[Claims] 1. From the characteristics of a face image of an individual obtained by periodically capturing an image of a viewer present in a surveillance area in front of a television receiver and the characteristics of a face image of a predetermined individual stored in a dictionary. In the television viewer identification method for identifying which of the predetermined individuals the individual who is viewing the television receiver in the image recognition process, at least one of all individuals to be identified The guest face image, which has been measured by the television viewer in a state where the dictionary for an individual has not been created and which is determined to be none of the predetermined individuals in the image recognition processing, is registered in the registration box. If it is possible to identify the guest face image based on the declaration from the individual input via the personal computer, and if the guest face image can be identified as the face image of an individual for which a dictionary has not been created yet, the face image. That Television viewer identification method is characterized in that so as to save it as a human reference face image. 2. When a certain number of individual reference face images for which no dictionary has been created have been saved, a personal dictionary is created using the saved reference face images for use in the image recognition processing. The television viewer identification method according to claim 1, wherein: 3. A television viewer identification device for identifying an individual who is watching a television receiver, comprising: an imager for periodically imaging a surveillance area in front of the television receiver; and an imager captured by the imager. Face image extracting means for extracting a face image of an individual from the obtained images, dictionary storing means for storing a dictionary created based on a predetermined reference face image group for each individual, and the face image extracting means. For each face image extracted in step 1, try to identify the individual using the created dictionary, assign the individual number to the face image that can be identified, and guest face image for the face image that cannot be identified. An image recognition unit that processes as a registration box, a registration box that receives a viewing and non-viewing report from an individual who has not created a dictionary, a recognition result of the image recognition unit, and a declaration from an individual input via the registration box Based on the above, the viewer identifying means for identifying the individual being viewed, and the face image processed as the guest face image by the image recognizing means could be identified by the viewer identifying means as the face image of the predetermined individual. In this case, if the identified personal dictionary is not yet created and the number of reference face images for creating the individual dictionary has not yet reached a certain number, a guest face image identified as the individual's dictionary is displayed. A face image collection unit that is registered in the reference face image group storage unit corresponding to the individual as the reference face image of the individual, and a fixed number of the face image collection units registered in the reference face image group storage unit for an individual for which a dictionary has not been created yet. The dictionary creating means for creating the personal dictionary based on the reference face image and storing the dictionary in the dictionary storage means, and the dictionary of all the individuals to be identified as the identification result of the individual being viewed And a discrimination result selecting means for selecting the discrimination result of the viewer discriminating means and selecting the discrimination result of the image recognizing means after all the dictionaries are stored. A television viewer identification device characterized by: