JP5103287B2 - ADVERTISEMENT EFFECT MEASUREMENT DEVICE, ADVERTISEMENT EFFECT MEASUREMENT METHOD, ADVERTISEMENT EFFECT MEASUREMENT PROGRAM, AND RECORDING MEDIUM CONTAINING THE PROGRAM - Google Patents

Description

The present invention relates to an advertisement effect measuring apparatus, an advertisement effect measuring method, an advertisement effect measuring program, and a recording medium on which the program is recorded, which can objectively measure the effect of advertisement media using a camera and a human face detection / posture estimation technique. Is.

  In recent years, advertising media called “digital signage”, which transmits images using electric display devices outdoors or in stores, has begun to spread rapidly. This new advertising form is considered to be more cost-effective than conventional advertising media because advertisers can send and update the latest information in real time by cooperating with the network. Along with this, a technique for objectively measuring advertising effectiveness has been strongly demanded.

As a method for objectively measuring the advertising effect, a method described in Patent Document 1 is known. This is to install a camera on the advertising media side, extract the person reflected in the camera based on the contour information, etc., evaluate the extracted number as the degree of attention to the advertising media, and total the evaluation values is there. By combining this with a face detection technique (for example, Non-Patent Document 1), it is possible to measure a more accurate degree of attention. For example, a method described in Non-Patent Document 2 has been proposed as a method for estimating the posture of a face with high accuracy.
Japanese Patent No. 3757584 Yuji Mita, Toshimitsu Kaneko, Osamu Hori, “Probabilistic Incremental Sign Correlation Suitable for Image Matching of Individual Differences”, IEICE Transactions D-II, Vol. J88-D-II, No. 8, pp.1614-1623, 2005. Shingo Ando, Yoshinori Kusachi, Akira Suzuki, Kenichi Arakawa, “Pose Estimation Method for 3D Objects Using Support Vector Regression”, IEICE Transactions D, Vol.J89-D No.8, pp.1840-1847 , 2006.

  However, generally known face detection methods are not only easily affected by face orientation and lighting fluctuations, but also easily react to patterns similar to faces, so if they are temporarily detected as a face from the background On the contrary, there may be a case where the face cannot be detected temporarily (hereinafter referred to as detection omission). Therefore, noise is mixed into the evaluation summary value, resulting in an inaccurate count result. Specifically, in overdetection, the number of people is counted more than actual. In addition, in the detection failure, the number of people is counted less than the actual number, or the same person is mistakenly counted more than two (if you lose your face temporarily in different frames, (Because the same person is judged as another person before and after). These are problems that cannot be solved simply by introducing a person tracking technique or the like. This is because tracking does not include an operation of correcting the detection result (for example, deleting it later).

  The present invention has been made in view of the above-mentioned problems, and its purpose is to measure the advertising effect, which can calculate the number of attentions with high accuracy and measure it as an advertising effect even if face detection is incomplete. An apparatus, an advertisement effect measuring method, an advertisement effect measuring program, and a recording medium recording the program are provided.

  In order to achieve the above object, in the present invention, the time when a person is gazing at the advertising media is measured, and as a result of the aggregation, the number of persons gazing at an arbitrary time or more within a predetermined period (for example, one day) Is calculated. An example of the total result is shown in FIG. This excludes things that only saw advertising media for an instant (considering overdetection is also included in this), so not only can you calculate highly reliable figures, but people who have seen advertising media for a long time By extracting a part of the video that is only visible and being visually confirmed by humans, it is possible to analyze the advertising effect in more detail.

  Further, the face detection state is defined as three states of (1) provisional detection, (2) positive detection, and (3) gaze. Then, using the previous detection result in time, each detected face is assigned to one of these three states, and the state transition of each person is recorded and aggregated, so that the more accurate number of attention can be obtained. Calculate and measure as advertising effectiveness. Whether or not a person is paying attention to the advertising media can be determined by performing face posture estimation after detecting the face and examining whether the face is oriented in a specific range.

In other words, the advertising effect measuring device according to claim 1 shoots with a camera installed in the vicinity of the advertising media, counts the number of persons who can be regarded as paying attention to the advertising media, and objectively measures the advertising effectiveness. An advertising effect measuring device, wherein the image input means for inputting an image captured by the camera as digital data, and a face detection means for detecting an area with a face from the input image input to the image input means, Based on the face area detected by the face detecting means, a face posture estimating means for estimating a parameter value representing a face posture, a detection position, a size, a face similarity passed from the face detecting means, a face pose parameters passed from the face pose estimation means, detecting the position of the previous frame that is passed from the previous detection status storage means, the size, based on the face state, like the face of the current frame of the detected face A state judging means for judging, based on the detected ID and gaze time passed from the face condition of the current frame prior to detection condition storing means passed from said state discriminating means, the detection ID and watching time information of the detected face Detection ID / gaze time giving means to be given, face detection time, detection position and size passed from the face detection means, face state passed from the state discrimination means, and passed from the detection ID / gaze time giving means Based on pre-detection state storage means for temporarily storing each information of detection ID and gaze time, face detection time passed from the face detection means, detection ID passed from the detection ID / gaze time giving means, and gaze time An attention log storage means for picking up each information of detection time, detection ID, and gaze time, which is a detection result, and accumulating as an attention log, and the attention log storage means Based on the interest who logs passed includes a target's tally means for counting the attention persons within a specified time period from the target's log, and a result output means for outputting a result of the aggregation in the interest toll collecting unit, The state determination means includes a first condition that a difference between a size in the current frame and a size in the previous frame is within a determined range, and a detection position in the current frame and a detection position in the previous frame. With respect to the second condition that the distance is within the determined range and the third condition that the face posture parameter in the current frame is within the specified range, the detected face in the current frame is preceded in descending order of face similarity. If the first condition or the second condition is not satisfied, the face state of the current frame is determined as a temporary detection result, and the first condition and the second condition are satisfied. Must meet the requirements If so, the face state of the current frame is determined as the positive detection result, and if any of the three conditions is satisfied, the face state of the current frame is determined as the positive detection result and the gaze state, and the detection ID / gaze time giving means is When the face state in the current frame is the positive detection result and the gaze state, the time difference between the previous frame and the current frame is added to the gaze time .

The advertisement effect measurement apparatus according to claim 2, Oite to claim 1, wherein the target's tally means, and the target's to be counted, a person continues to watch the advertisement specified time or more defined And counting.

Further, the advertisement effect measuring method according to claim 3 is an objective measurement of the advertisement effect by counting the number of persons who are photographed with a camera installed in the vicinity of the advertisement medium and counting the advertisement medium as being noticed. An advertisement effect measuring method, wherein an image input unit inputs an image captured by the camera as digital data, and a face detection unit detects a face from the input image input by the image input step. A face detecting step for detecting a certain area; a face posture estimating means for estimating a parameter value representing the face posture based on the face area detected by the face detecting step; and a state determining means. Are detected position, size, and face similarity passed from the face detection means, face posture parameters passed from the face posture estimation means, and pre-detection state storage means Detecting position of a frame, the size, based on the face state, and a state determination step of determining the face status of the current frame of the detected face, the detection ID · gaze time providing means, is passed from the state discriminating means Based on the face state of the current frame and the detection ID and the gaze time passed from the previous detection state storage means, a detection ID / gaze time provision step for providing the detected face detection ID and gaze time information, and the pre-detection state storage Each information of the face detection time, detection position and size passed from the face detection means, the face state passed from the state determination means, the detection ID passed from the detection ID / gaze time giving means, and the gaze time The pre-detection state storage step for temporarily storing the information and the attention log storage means are received from the face detection time passed from the face detection means and the detection ID / gaze time provision means. An attention log storage step of picking up each information of detection time, detection ID, and attention time as detection results based on the detection ID and the attention time, and accumulating it as an attention log; Based on the attention log delivered from the log storage means, the number-of-interests counting step for counting the number of attentions within the specified period from the attention log, and the result output means, the results aggregated by the number-of-interests counting step A result output step for outputting, wherein the state determination step includes a first condition that a difference between a size in the current frame and a size in the previous frame is within a predetermined range, and a detection position in the current frame. And the third condition that the distance between the detection position in the previous frame is within the determined range and the third condition that the face posture parameter in the current frame is within the specified range The detected face in the current frame is associated with the detected face of the previous frame in descending order of face similarity, and if the first condition or the second condition is not satisfied, the face state of the current frame is determined as the provisional detection result If the first condition and the second condition are satisfied but the third condition is not satisfied, the face state of the current frame is determined as a positive detection result. If any of the three conditions is satisfied, the face state of the current frame is determined. The positive detection result and the gaze state are determined, and the detection ID / gaze time adding step adds the time difference between the previous frame and the current frame to the gaze time when the face state in the current frame is the positive detection result and the gaze state. , it is characterized in that.

The advertisement effect measuring method according to claim 4, Oite to claim 3, wherein the target's tally step and a target person to be counted, a person continues to watch the advertisement specified time or more defined And counting.

An advertisement effect measurement program according to claim 5 is an advertisement effect measurement program that causes a computer to execute each step according to claim 3 or 4 .

A recording medium according to claim 6 is a computer-readable recording medium in which an advertisement effect measurement program for causing a computer to execute the steps according to claim 3 or 4 is recorded.

  According to the present invention, the time during which a person is gazing at an advertising medium is measured, and the number of persons who have been gazing for an arbitrary time or more is measured as a totaling result. It is possible to realize accurate advertisement effect measurement even if face detection is incomplete by defining three states of positive detection and (3) gaze and recording and counting the state transitions.

  In addition, the number of people increases due to over-detection, the number of people decreases due to over-detection, and the number of people decreases due to over-detection. Similarly, one person is determined to be another person at another time due to over-detection. As a result, the actual number of persons is not increased, so that highly accurate measurement can be performed.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings, but the present invention is not limited to the following embodiments. FIG. 2 is a block diagram showing the configuration of the advertisement effect measuring apparatus according to the embodiment of the present invention.

  In FIG. 2, the advertisement effect measuring apparatus according to the present embodiment includes an image input unit 11 as an image input unit, a face detection unit 12 as a face detection unit, a face posture estimation unit 13 as a face posture estimation unit, A state determination unit 14 as a state determination unit, a detection ID / gaze time provision unit 15 as a detection ID / gaze time provision unit, a pre-detection state storage unit 16 as a pre-detection state storage unit, and an attention log storage unit As the attention log storage unit 17 as the above, a number-of-interests counting unit 18 as the number-of-interests counting unit, and a result output unit 19 as the result output unit, the function of each unit is achieved by, for example, a computer.

  FIG. 3 shows an embodiment of the advertisement effect measuring method of the present invention, and is a flowchart of processing performed by the advertisement effect measuring apparatus of FIG. The entire advertisement effect measuring method of this embodiment is roughly divided into two processes. That is, the process of leaving the information of the attention reflected on the camera as a log (hereinafter referred to as the attention log) (each step in FIG. 3A), and the process of counting and outputting the attention log (FIG. 3B Each step)).

  Hereinafter, this embodiment will be described in detail with reference to FIGS.

  The image input unit 11 transmits image information input by the camera to the face detection unit 12 (step S11). Assume that the camera is installed in the vicinity (for example, the upper part) of the advertisement media posting position. In addition to digital signage, the advertisement media includes ordinary poster advertisements, general home TVs, and the like.

  The face detection unit 12 detects a region with a face from each image transmitted from the image input unit (step S12). This process can be detected at high speed using a statistical method such as stochastic incremental code correlation disclosed in Non-Patent Document 1, for example. It should be noted that the similarity indicating the face-likeness used for detection here is used in the state determination unit 14 later.

  The face posture estimation unit 13 estimates the face posture with high accuracy by using the detected brightness pattern of the face area as an input (step S13). Specifically, a parameter estimation method combining principal component analysis and support vector regression disclosed in Non-Patent Document 2 can be applied. Here, there are two types of parameters representing the posture of the face, the yaw angle and the pitch angle.

  The state determination unit 14 refers to the detected face position, size, and state in the previous frame from the previous detection state storage unit 16 described later, and determines the state of each detected face in the current frame (step S14). Here, the detected face states are the following three types.

State 1: Temporary detection result State 2: Positive detection result State 3: Positive detection result and gaze state First, each detected face of the current frame is sorted by similarity indicating the likelihood of a face. Then, the detection faces in the previous frame are associated with the detected faces in descending order of similarity. Associating is performed on a one-to-one basis, and is associated with a condition that satisfies both of the following two conditions.

Condition 1: When the size of the current frame detection face is s, the size s _pre of the previous frame detection face is applied to the following equation (1).

Condition 2: When the center position of the current frame detection face is (x, y), the center position (x _pre , y _pre ) of the previous frame detection face is applied to the following equation (2).

Here, _kmin , _kmax , _lmin , _lmax , _mmin , _mmax are constants given in advance. If there are two or more applicable items, the one with the shortest distance between the center positions is adopted. Then, after all the detected faces in the current frame are associated, their states are determined based on FIG.

  In FIG. 4, when the detection result of the previous frame is in state 1, whether or not there is a detection result associated with the current frame (satisfying the two conditions of the expressions (1) and (2)) exists. If it does not exist, the detection result of the current frame is set as no detection.

  When the detection result of the previous frame is in state 2 and state 3, whether or not there is a detection result associated with the current frame (satisfying the two conditions of the expressions (1) and (2)) exists. If it does not exist, the current frame detection result is set to state 1 (in this case, the detection position and size of the previous frame are inherited and reflected in the current frame detection result).

  If there is an associated detection result in step S14a and step S14b, it is determined in step S14c whether the posture is within the specified range. If it is not within the specified range, the detection result of the current frame is set to state 2. If it is within the specified range, state 3 is set.

  When the detection result of the previous frame is not detected, it is determined whether or not the current frame remains uncorrelated with any result of the previous frame (step S14d). The result is state 1.

  The posture in step S14c within the specified range indicates a case where the posture parameters yaw and pitch of the current frame detection face satisfy the following expression (3).

Note that y _min , y _max , p _min , and p _max are constants given in advance. By appropriately setting these values, a certain degree of freedom can be given to the relative positional relationship between the camera and the advertising media. For example, it is possible to cope with a state where the image is taken from the side of the slant. FIG. 5 shows an example of the result of state determination.

  The detection ID / gaze time providing unit 15 obtains the detection ID information and the gaze time information in the current frame based on the detection ID information and the gaze time information of the detection face of the previous frame associated with each detected face state. Grant (step S15). A specific method will be described with reference to FIG.

  In FIG. 5, when it is in state 1 and there is no detected face corresponding to the previous frame (ID: 12), a new detection ID (ID: 14) is issued and assigned, and the gaze time is set to 0 second. To do. If the detected face corresponding to the previous frame is in the state 1 (ID: 11), the detection ID and the gaze time information are inherited from the corresponding previous frame detected face as they are.

  In the case of state 2 (ID: 10), the detection ID and gaze time information are directly inherited from the corresponding previous frame detection face.

  In the case of state 3 (ID: 13), the detection ID is inherited from the previous frame as it is, and the gaze time is (gaze time information of the corresponding previous frame detection face) + (elapsed time between the previous frame and the current frame). And In FIG. 5, the elapsed time between the previous frame and the current frame is 1 sec.

  The pre-detection state storage unit 16 records all face detection times, positions, sizes, states, detection IDs, and gaze times in the current frame in the memory or HDD (step S16). At that time, the previously recorded data is deleted.

  The attention log storage unit 17 records and stores the detection time, detection ID, and gaze time of all detected faces in the current frame as a attention log in the memory or HDD (step S17). At that time, the previously recorded data will not be erased and will be added.

  The processes in steps S11 to S17 are repeatedly executed until the camera input is completed.

  Next, the number-of-interests counting unit 18 inputs the attention log, the target period (specified by the user), and the gaze time threshold (specified by the user) recorded in the attention-log storage unit 17, and the number of attentions Are counted (step S18).

  Specifically, first, an attention log within the target period is extracted. Although the target period is specified by the user, there may be a case where the target advertisement is limited to a part of the time zone in digital signage, TVCM, or the like. In that case, the time schedule information of the target advertisement is input and further narrowed down by the time zone in which the advertisement is displayed.

  Next, the detection face information (detection time, detection ID, gaze time) with different detection IDs is picked up one by one while referring to the detection time from the latest detection time. By this operation, only one face detection information with the same detection ID is left with the maximum gaze time information. Further, the case where the gaze time information is smaller than the gaze time threshold is excluded. The number of detected faces finally remaining is counted as the number of attention. A plurality of gaze time threshold values may be specified. In that case, a plurality of attention numbers are calculated as shown in FIG. Further, the number of attentions may be expressed by the number of attentions per unit time (for example, the number of attentions per day is totaled for one month). Further, when a plurality of gaze time threshold values are provided, the number of viewers may be weighted and added to be used as an overall evaluation score.

  The result output unit 19 outputs the counting result obtained by the number-of-interests counting unit 18 (step S19).

  Further, the present invention supplies a storage medium (recording medium) that records software program codes for realizing the functions of the above-described embodiments to a system or apparatus, and the CPU (MPU) of the system or apparatus stores the storage medium. This can also be realized by reading and executing the stored program code. In that case, the program code itself read from the storage medium realizes the functions of the above-described embodiment, and the storage medium storing the program code, for example, CD-ROM, DVD-ROM, CD-R, CD-RW, MO, HDD, etc. constitute the present invention.

Explanatory drawing which shows an example of the total result of the number of attention in the example embodiments of the present invention. The block diagram which shows the example of embodiment of the advertising effectiveness measuring apparatus of this invention. The flowchart which shows the example of embodiment of the advertisement effect measuring method of this invention. Explanatory drawing which shows the state discrimination | determination process in the embodiment of this invention. Explanatory drawing which shows an example of the process of detection ID and gaze time provision in the embodiment of this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 ... Image input part, 12 ... Face detection part, 13 ... Face posture estimation part, 14 ... State discrimination | determination part, 15 ... Detection ID and gaze time provision part, 16 ... Pre-detection state storage part, 17 ... Attention person log storage part , 18 ... Number-of-interests counting section, 19 ... Result output section.

Claims (6)

An advertising effect measurement device that objectively measures the advertising effect by counting the number of persons who can be regarded as paying attention to the advertising media, photographed with a camera installed in the vicinity of the advertising media,
Image input means for inputting an image photographed by the camera as digital data;
Face detection means for detecting a region with a face from the input image input to the image input means;
A face posture estimation means for estimating a parameter value representing the posture of the face based on the face area detected by the face detection means;
Detection position, size, face similarity passed from the face detection means, face posture parameters passed from the face posture estimation means, detection position, size, face in the previous frame passed from the previous detection state storage means State determining means for determining the face state of the current frame of the detected face based on the state;
Based on the face state of the current frame passed from the state discriminating means, the detection ID and the gaze time passed from the previous detection state storage means, a detection ID and gaze time provision for giving detected face detection ID and gaze time information Means,
Each information of the face detection time, detection position, size passed from the face detection means, the face state passed from the state determination means, the detection ID passed from the detection ID / gaze time giving means, and the gaze time is temporarily stored. Pre-detection state storage means for storing in
Based on the face detection time passed from the face detection means, the detection ID passed from the detection ID / gaze time giving means, and the gaze time, information on the detection time, detection ID, and gaze time as detection results is picked up and noticed Attention log storage means for accumulating as a person log,
On the basis of the attention log delivered from the attention log storage means, the number-of-interest counting means for counting the attention within a specified period from the attention log;
A result output means for outputting the result totaled by the number-of-interests counting means ,
The state determination means includes a first condition that a difference between a size in the current frame and a size in the previous frame is within a determined range, and a detection position in the current frame and a detection position in the previous frame. With respect to the second condition that the distance is within the determined range and the third condition that the face posture parameter in the current frame is within the specified range, the detected face in the current frame is preceded in descending order of face similarity. If the first condition or the second condition is not satisfied, the face state of the current frame is determined as a temporary detection result, and the first condition and the second condition are satisfied. If the condition is not satisfied, the face state of the current frame is determined as a positive detection result, and if any of the three conditions is satisfied, the face state of the current frame is determined as a positive detection result and a gaze state.
The detection ID / gaze time giving means adds a time difference between the previous frame and the current frame to the gaze time when the face state in the current frame is the positive detection result and the gaze state.
An advertising effect measuring device characterized by that. The advertisement effect measuring device according to claim 1, wherein the number-of-interests counting means counts the attentions to be counted as a person who has continued to watch the advertisement for a specified time or more. . An advertising effect measurement method for objectively measuring the advertising effect by counting the number of people who can be regarded as paying attention to the advertising media, taken with a camera installed in the vicinity of the advertising media,
An image input step in which an image input means inputs an image captured by the camera as digital data;
A face detecting unit for detecting a region having a face from the input image input by the image input step;
A face posture estimation step for estimating a parameter value representing the posture of the face based on the face region detected by the face detection step;
The detection position, size, and face similarity passed from the face detection means, the face posture parameter passed from the face posture estimation means, and the detection position in the previous frame passed from the previous detection state storage means. A state determining step for determining the face state of the current face of the detected face based on the size and the face state;
The detection ID and gaze time information of the detected face is detected based on the face state of the current frame passed from the state discriminating means and the detection ID and gaze time passed from the previous detection state storage means. A detection ID / gaze time granting step,
The pre-detection state storage means has a face detection time, detection position and size passed from the face detection means, a face state passed from the state determination means, a detection ID passed from the detection ID / gaze time giving means, and a gaze A pre-detection state storage step for temporarily storing each piece of time information;
Based on the face detection time passed from the face detection means, the detection ID passed from the detection ID / gaze time giving means, and the gaze time, the attention log storage means detects the detection time, detection ID, and gaze time as detection results. The attention log storage step of picking up each information of and accumulating as the attention log,
The number-of-interests counting means counts the number-of-interests counting step for counting the number of attentions within a specified period from the attention-log based on the attention-logs delivered from the attention-log storage means;
A result output means comprising a result output step for outputting the results totalized by the number-of-interests counting step;
The state determination step includes: a first condition that a difference between a size in the current frame and a size in the previous frame is within a determined range; and a detection position in the current frame and a detection position in the previous frame. With respect to the second condition that the distance is within the determined range and the third condition that the face posture parameter in the current frame is within the specified range, the detected face in the current frame is preceded in descending order of face similarity. If the first condition or the second condition is not satisfied, the face state of the current frame is determined as a temporary detection result, and the first condition and the second condition are satisfied. If the condition is not satisfied, the face state of the current frame is determined as a positive detection result, and if any of the three conditions is satisfied, the face state of the current frame is determined as a positive detection result and a gaze state.
The detection ID / gaze time giving step adds a time difference between the previous frame and the current frame to the gaze time when the face state in the current frame is the positive detection result and the gaze state.
A method for measuring the effectiveness of an advertisement. 4. The advertisement effect measuring method according to claim 3, wherein the number-of-interests counting step performs counting by defining a person to be counted as a person who has continued to watch the advertisement for a specified time or longer. . An advertisement effect measurement program for causing a computer to execute each step according to claim 3 or 4 . A computer-readable recording medium on which an advertisement effect measurement program for causing a computer to execute each step according to claim 3 or 4 is recorded.

Images