JP5489197B2 - Electronic advertisement apparatus / method and program - Google Patents

Description

  The present invention relates to an apparatus for extracting a face image having specific conditions from an image of a person, an electronic advertisement apparatus including the apparatus, and the like.

  It is known that a device called an electronic advertising device or digital signage is installed in a store or the like so that a customer visiting the store or the like can view an advertisement video. The electronic advertising device includes an image display device such as a liquid crystal display device, for example, and displays the advertising video recorded in the electronic advertising device or the advertising video distributed from a higher-level distribution device. It is a device that displays.

  In Patent Document 1, an electronic advertisement apparatus is provided with a camera, and an image of a human face part (hereinafter referred to as “face image”) for viewing the screen of the electronic advertisement apparatus is detected from an image captured by the camera. It has been proposed that the human gender, age, etc. are estimated and an advertisement video corresponding to the gender, age, etc. is displayed to enhance the advertising effect of the electronic advertising device.

  In Patent Document 2, a camera is provided in the vicinity of a display that is installed on a street and performs advertising, images a passerby who passes through the peripheral area of the display, and a composite video including the passer's video is displayed on the display. It has been proposed to display.

  In Patent Document 3, an image capturing unit is incorporated in a display, a person who views an advertisement video displayed on the display is imaged, attributes of the person such as gender or age group are determined, and the advertisement video It has been proposed to calculate the audience rating for each person attribute.

  Patent Document 4 discloses a method and apparatus for detecting a human head from an image acquired by a video acquisition means, and a method for acquiring a front face image from a partial image of the head detected by the method and apparatus. An apparatus is disclosed.

  Patent Document 5 discloses a face orientation estimating apparatus and method for estimating the orientation of a face in an image.

JP-A-2005-251170 JP 2006-251300 A JP 2006-254274 A JP 2000-222576 A JP 2004-094491 A

  The invention disclosed in Patent Documents 1 and 3 captures a person who views an advertisement video and the like, estimates the attribute of the person, displays an advertisement video according to the attribute, or displays the estimated attribute. It is used as a document for evaluating the effects of advertising videos. Further, the device disclosed in Patent Document 2 displays a captured human image on a display to raise the effect of advertisement by calling attention of a human viewing an advertisement video or the like. In other words, the inventions disclosed in Patent Documents 1 to 3 enhance the advertising effect of the electronic advertising device by directly or indirectly using an image obtained by capturing a person viewing an advertising video or the like.

  Now, extracting only “a human face image that has a special interest in advertisement video” is considered to further enhance the effects of the inventions disclosed in Patent Documents 1 to 3.

  On the other hand, in general, electronic advertising devices need to be seen by many people. Therefore, digital signage is installed in a place where there are many passers-by. Therefore, the field of view of the imaging means provided in the electronic advertising device includes many people other than “persons who were there by chance”, that is, “humans who have a special interest in advertising images”. it is conceivable that.

  However, the invention disclosed in Patent Documents 1 to 3 excludes “an accidental human face image” from an image captured by an image capturing unit incorporated in an electronic advertisement apparatus, It lacks the means to extract only “human face images that have a special interest in”. Therefore, if such a means is realized and provided in the electronic advertising device, the advertising effect of the electronic advertising device can be further enhanced.

The present invention has been made in view of such a background, and excludes a “person who was there by chance” from an image of a person in front of the image display device, thereby providing an “image display device. Provided is an electronic advertisement apparatus / method and program for extracting only a face image of a person who has a special interest in an image, estimating an attribute of the person, and displaying an advertisement video corresponding to the attribute Is.

In order to solve the above-mentioned problems, an electronic advertisement apparatus according to the present invention includes an image display apparatus that displays an advertisement video, and an imaging apparatus that captures an image of a person viewing the advertisement video displayed on the image display apparatus. In the apparatus, an extraction range cutout unit that cuts out a predetermined range of an image from the original image captured by the imaging device, and a front face image extraction unit that extracts a front-facing face image from the image cut out by the extraction range cutout unit A face image selection unit that selects a face image closest to the image display device from the face images extracted by the front face image extraction unit; and the face image selected from the face image selected by the proximity face image selection unit. A person attribute estimating means for estimating an attribute of the person related to the image, and a display for displaying on the image display device an advertisement video corresponding to the attribute of the person estimated by the person attribute estimating means. And o control means, the predetermined range, the user is characterized by comprising an extraction range setting unit for setting arbitrarily.

The electronic advertising method according to the present invention is an electronic advertising method by an electronic advertising device comprising an image display device for displaying an advertising video and an imaging device for imaging a person viewing the advertising video displayed on the image display device. , the range of the image to be cut out from the original image the imaging device imaged, an extraction range setting step for the user to arbitrarily set the image of the range in which the imaging device is set by the extraction range setting step from the original image captured An extraction range cutout step to cut out, a front face image extraction step to extract a face image facing the front from the image cut out in the extraction range cutout step, and the image from the face image extracted in the front face image extraction step A proximity face image selection step for selecting a face image closest to the display device, and a face image selected in the proximity face image selection step A person attribute estimating step for estimating the attribute of the person related to the face image, and a video control step for causing the image display device to display an advertisement video corresponding to the attribute of the person estimated in the person attribute estimating step. It is characterized by having.

The program according to the present invention is installed in a computer that controls an electronic advertisement apparatus including an image display apparatus that displays an advertisement video and an imaging apparatus that images a person viewing the advertisement video displayed on the image display apparatus. Then, the computer uses an extraction range cutout unit that cuts out a predetermined range of images from the original image captured by the imaging device, and a front face image extraction that extracts a front-facing face image from the image cut out by the extraction range cutout unit A face image selecting unit that selects a face image closest to the image display device from the face images extracted by the front face image extracting unit, and the face image selected from the face image selected by the close face image selecting unit. A person attribute estimating means for estimating the attribute of the person related to the image, and an advertisement video corresponding to the attribute of the person estimated by the person attribute estimating means Video control means for displaying on the display device, the predetermined range, the user is equal to or to function as, the extraction range setting unit for setting arbitrarily.

  According to the present invention, the face image of the person who is in a specific range of the original image (for example, near the center of the field of view of the image pickup means) and faces the image pickup means is extracted. It is possible to accurately extract a face image of a person who is particularly interested in. Further, if such an electronic advertisement apparatus is provided with such a face image extraction device, a face image of a person who has a special interest in an advertisement video or the like provided by the electronic advertisement device is extracted, and information obtained from the face image is extracted from the electronic advertisement device. It can be used as feedback for marketing information.

  Hereinafter, the best mode for carrying out the present invention will be described.

  FIG. 1 is a conceptual configuration diagram of an electronic advertising device including a face image extracting device according to an embodiment of the present invention.

  As shown in FIG. 1, the electronic advertising device 1 includes a display device 2 and an imaging device 3.

  The display device 2 is a liquid crystal display device that is controlled by the video control device 4 and displays an advertisement video or the like based on a video signal supplied from the video control device 4.

  The imaging device 3 is a video camera whose optical axis is arranged parallel to the optical axis of the display device 2 (that is, the person 5 facing the display device 2 is also parallel to the imaging device 3). Yes, it is possible to take an image of the person 5 viewing the advertisement video displayed on the display device 2.

  The image picked up by the image pickup device 3 is sent to the face image extraction device 6. The face image extraction device 6 processes the image, and among the people 5, a person (hereinafter referred to as a “specific person”) estimated to have a special interest in the advertisement video displayed on the display device 2 or the like. A face image is extracted and sent to the person attribute estimation device 7.

  The person attribute estimation device 7 estimates the attributes (gender and age) of the specific person based on the face image of the specific person sent from the face image extraction device 6. The gender and age of the specific person estimated by the person attribute estimation device 7 are fed back to the video control device 4, and the video control device 4 advertises the video according to the gender and age of the specific person (for example, the specific person). If it is a man, the advertisement image of the product for men is displayed on the display device 2.

  Now, the face image extraction device 6 has a configuration as shown in FIG.

  The face image extracting device 6 is a computer that extracts only the face image of the specific person, that is, the face image of a person who has a special interest in the advertisement video displayed on the display device 2 from the image captured by the image capturing device 3. And includes an extraction range setting unit 61, an extraction range cutout unit 62, a front face image extraction unit 63, and a close-up face image selection unit 64.

  The extraction range setting unit 61 sets and inputs a range for setting the range in order to limit the range for extracting the face image of the specific person to a part of the image 8 captured by the imaging device 3. It is. In this embodiment, as shown in FIG. 3, the center of the image 8 is the coordinate origin (0, 0), the left and right X coordinates are −50 and 50, and the upper and lower Y coordinates are −50 and 50. The system is defined, and the extraction range is set by the coordinates of the center and the length of the side.

  For example, if (0, 0, 50, 50) is input to the extraction range setting unit 61, the coordinate origin (0, 0) as shown at A in FIG. A rectangular area having a width of 50 is set as the extraction range. If (25, 0, 50, 100) is input, the right half of the image 8 is set as the extraction range, as indicated by B in FIG. If (0, 0, 100, 100) is input, the entire area of the image 8 is set as the extraction range.

  In addition, the extraction range may be defined by a circular area instead of a rectangle. In this case, the extraction area can be defined by the coordinates and radius of the center of the circle.

  Alternatively, the extraction range may be defined by a ratio to the area of the image 8. For example, when (25) is input to the extraction range setting unit 61, a partial image having an area ratio of 25% with respect to the entire image 8 centering on the coordinate origin (0, 0) may be set as the extraction range.

  Alternatively, the extraction range setting unit 61 may be omitted, and the extraction range may be fixed to a predetermined range. In any case, the center of the extraction range is generally set to the coordinate origin (0, 0). This is because it can be considered that a person who has a particular interest in the advertisement video or the like displayed on the display device 2 often stands in the center in front of the display device 2, that is, in the center of the field of view of the imaging device 3.

  The extraction range cutout unit 62 cuts out an image of the extraction range set by the extraction range setting unit 61 from the image captured by the imaging device 3. For example, when (0, 0, 50, 50) is set and input to the extraction range setting unit 61, the extraction range cutout unit 62 captures images of five persons as shown in FIG. A partial image 9 as shown in FIG.

  The front face image extraction unit 63 searches the partial image 9 and extracts a face image facing the front. In the partial image 9, three persons are imaged. However, a person who has a special interest in an advertisement video or the like displayed on the display device 2 may have the display device 2 facing the front of the face. It can be considered that there are many.

  Various well-known means can be selected for the extraction of the front-facing face image. For example, a standard front face image may be provided inside the front face image extraction unit 63, and the similarity between the face image extracted from the image and the front face image may be obtained. Alternatively, in the front face image, both human eyes are arranged substantially symmetrically with respect to the center line of the outline of the face (see FIG. 5A). It may be asymmetric (see FIG. 5B). In addition, an allowable range may be set when extracting a face image facing the front. That is, even if the face image is swung to the left or right within a predetermined allowable range around the position facing the front, the face image may be processed as facing the front.

  In order to avoid complications, detailed description of means for extracting a face image facing the front is omitted. As described above, Patent Document 3 discloses a means for detecting a human head from an image and acquiring only a front face image from the detected partial image of the head, and means for estimating the orientation of the face in the image Is disclosed in Patent Document 4. Moreover, since it is described also in various other well-known documents, please refer to it.

  When a plurality of front-facing face images are extracted, it is necessary to select one face image. Therefore, the close-up face image selection unit 64 selects the face image of the person 5 that is closest to the display device 2 from the front-facing face images extracted by the front face image extraction unit 63. This is because it is considered that a person who has a special interest in the advertisement video displayed on the display device 2 is often viewing the advertisement video or the like in the immediate vicinity of the display device 2.

  For estimating the distance between the person 5 and the display device 2, the distance between the left and right eyes on the face image is used. This is because when the person 5 approaches the display device 2 (imaging device 3) as shown in FIG.

  Alternatively, the two imaging devices 3 can be arranged on the left and right sides, so-called stereo photography can be performed, and the distance can be estimated based on the image shift caused by the parallax.

  Further, the face image extraction apparatus 6 executes a face image extraction program as shown in FIG.

  This face image extraction program is executed when the image pickup device 3 of the electronic advertising device 1 picks up an image. First, a predetermined (or set using the extraction range setting unit 61) is extracted from the image picked up by the image pickup device 3. The image of the range is cut out (STEP 1).

  Next, a face image of a person facing the front with respect to the imaging device 3 (= to the display device 2) is extracted from the clipped image (STEP 2). This is because it is presumed that a person who has a special interest in the video is facing the display device 2 (If the person is not facing the front even though the image is cut out, I can't say).

  If there is no face image of the person facing the front (STEP 3; NO), the program is terminated, and if the face image is extracted (STEP 3; YES), the process proceeds to STEP 4.

  Of the face images of the person facing the front, the face image closest to the display device 2 (= closest to the imaging device 3) is selected and output to the person attribute estimation device 7, and the program is terminated. This is because it is estimated that a person who has a special interest in the video displayed on the display device 2 approaches the display device 2. If there is only one face image of the person facing the front, that one is selected.

  Now, the configuration of the video control device 4 will be described. As shown in FIG. 8, the video control device 4 physically includes a schedule database (DB) 41, an advertisement video DB 42, an input / output interface (I / F) 43, an input / output unit 44, and a control unit. 45.

  The schedule DB 41 stores a distribution schedule for distributing (displaying) advertisement video to the display device 2. As shown in FIG. 9, the distribution schedule includes a time (hour, minute, second) and a storage location address (URL (Uniform Resource Locator)) of a display target advertisement video (for example, a moving image with sound).

  As shown in FIG. 10, the advertisement video DB 42 stores video data of an advertisement video to be displayed (for example, a moving image with sound in MPEG format). Each video data is specified by a URL. In the distribution schedule, video data to be distributed is specified by this URL. Each advertisement video has an ID (video ID) and attribute information of the target person.

  The input / output interface 43 is a communication interface that outputs the video data of the advertisement video to the display device 2 and receives the attribute information of the person 5 output by the person attribute estimation device 7.

  The input / output unit 44 includes a keyboard, a mouse, a display device, and the like, and inputs various instructions and data to the control unit 45 and displays an output from the control unit 45.

  The control unit 45 includes a processor and the like, and includes a timer TC and operates according to a control program. Specifically, the control unit 45 reads the advertisement video to be displayed on the display device 2 from the advertisement video DB 42 and supplies it from the input / output interface 43 according to the distribution schedule registered in the schedule DB 41. In addition, the control unit 45 edits and updates the distribution schedule in response to an instruction from the input / output unit 44.

  Further, the control unit 45 changes the distribution schedule based on the attribute information of the person 5 input from the person attribute estimation device 7, reads out the advertisement video corresponding to the attribute information from the advertisement video DB 42, and enters it. It is supplied from the output interface 43.

Next, the operation of the electronic advertisement apparatus 1 having the above configuration will be described.
First, the control unit 45 of the video control device 4 reads out the advertisement video to be displayed from the advertisement video DB 42 according to the current time measured by the timer TC and the schedule registered in the schedule DB 41 and supplies the advertisement video to the display device 2. The display device 2 displays the supplied advertisement video.

  On the other hand, the imaging device 3 captures a front image, acquires a frame image with a frame period of 1/30 seconds, for example, and transmits the frame image to the face image extraction device 6. The face image extraction device 6 executes the above-described processing, and transmits the result (face image) to the person attribute estimation device 7. The person attribute estimation device 7 estimates the age and sex (attribute information) of the person 5 related to the face image based on the face image, and transmits the result to the video control device 4.

  When the video control device 4 receives the attribute information from the person attribute estimation device 7, the control unit 45 performs a process as shown in FIG.

  First, the control unit 45 stops the advertisement video distribution according to the distribution schedule described in the schedule DB 41 (STEP 11).

  Next, the control unit 45 searches the advertisement video DB 42, reads an advertisement video (corresponding video) having attribute information corresponding to the attribute information received from the person attribute estimation device 7 (STEP 12), and distributes it to the display device 2. (STEP 13). When the display of the corresponding video is completed (STEP 14; YES), the distribution of the advertising video according to the distribution schedule is resumed (STEP 15), and the process is terminated.

  As described above, the electronic advertising device 1 includes the face image extracting device 6, and from the images captured by the imaging device 3, a facial image of a person who has a special interest in the advertising video displayed by the electronic advertising device 1. And the age and sex of the person can be fed back to the video control apparatus 4 via the person attribute estimation apparatus 7. Therefore, the effect of the advertisement by the electronic advertisement apparatus 1 can be enhanced.

  The technical scope of the present invention is not limited to this embodiment. It goes without saying that various application modifications can be made within the scope of the technical idea shown in the claims.

  Further, in the description of the present embodiment, for convenience of explanation, each component (for example, the extraction range cutout unit 62) of the face image extraction device 6 is illustrated as including independent hardware. The face image extraction apparatus of the invention is not limited to the one having such a configuration. Of course, it is also possible to configure the face image extraction apparatus with a single computer by realizing each of the above-described components by software. Similarly, it goes without saying that all or part of the components of the electronic advertising device can be realized by software, and the electronic advertising device can be configured by a single computer.

  In the present embodiment, an example is shown in which the person attribute estimation device 7 estimates the sex age of the person 5 based on the face image extracted by the face image extraction device 6 and feeds it back to the video control device 4. However, the electronic advertisement apparatus according to the present invention is not limited to such a configuration. That is, the technical scope of the present invention is not limited by how the face image extracted by the face image extraction device 6 is used. Alternatively, the face image extracted by the face image extraction device 6 is recorded on a medium or used in an external system (that is, a system other than the electronic advertising device 1) using a communication means. To be included in the scope.

It is a notional block diagram of an electronic advertising device provided with a face image extraction device. It is a notional block diagram of a face image extraction device. It is a figure explaining the function of an extraction range setting part. It is a figure explaining the function of an extraction range cutout part. It is a figure explaining the principle of front image extraction. It is a figure explaining the principle of a close face image selection. It is a flowchart which shows the example of a face image extraction program. It is a notional block diagram of a video control apparatus. It is a figure which shows the structural example of schedule information. It is a figure which shows the structural example of advertisement video information. It is a flowchart which shows the example of the change process program of an advertisement image | video.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Electronic advertisement apparatus 2 Display apparatus 3 Imaging apparatus 4 Video control apparatus 5 Person 6 Face image extraction apparatus 7 Person attribute estimation apparatus 8 Image 9 Partial image 41 Schedule database 42 Advertisement video database 43 Input / output interface 44 Input output section 45 Control section 61 Extraction range setting unit 62 Extraction range cutout unit 63 Front face image extraction unit 64 Proximity face image selection unit

Claims (4)

An image display device for displaying advertisement video;
In an electronic advertising device comprising an imaging device for imaging a person viewing an advertising video displayed on the image display device,
Extraction range cutout means for cutting out an image of a predetermined range from the original image captured by the imaging device;
A front face image extracting means for extracting a front face image from the image cut out by the extraction range cutting means;
Proximity face image selection means for selecting a face image closest to the image display device from the face images extracted by the front face image extraction means;
Person attribute estimating means for estimating the attribute of a person related to the face image from the face image selected by the proximity face image selecting means;
Video control means for causing the image display device to display an advertisement video according to the attribute of the person estimated by the person attribute estimation means ;
An electronic advertisement apparatus comprising an extraction range setting means for a user to arbitrarily set the predetermined range . The electronic advertising device according to claim 1, wherein the image cut out by the extraction range cutout unit is a partial image having the center of the original image as the center. An image display device for displaying advertisement video;
In an electronic advertising method by an electronic advertising device comprising an imaging device for imaging a person viewing an advertising video displayed on the image display device,
An extraction range setting step in which the user arbitrarily sets the range of the image cut out from the original image captured by the imaging device;
An extraction range cutout step of cutting out an image of the range set in the extraction range setting step from the original image captured by the imaging device;
A front face image extraction step of extracting a front face image from the image cut out in the extraction range cutout step;
A proximity face image selection step of selecting a face image closest to the image display device from the face images extracted in the front face image extraction step;
A person attribute estimation step for estimating an attribute of a person related to the face image from the face image selected in the proximity face image selection step;
A video control step of causing the image display device to display an advertisement video corresponding to the attribute of the person estimated in the person attribute estimation step;
An electronic advertisement method characterized by comprising: An image display device for displaying advertisement video;
Installed in a computer that controls an electronic advertising device including an imaging device that images a person viewing the advertising video displayed on the image display device;
Extraction range cutout means for cutting out an image of a predetermined range from the original image captured by the imaging device;
A front face image extracting means for extracting a front face image from the image cut out by the extraction range cutout means;
Proximity face image selection means for selecting the face image closest to the image display device from the face images extracted by the front face image extraction means;
A person attribute estimating means for estimating an attribute of a person related to the face image from the face image selected by the proximity face image selecting means;
Video control means for causing the image display device to display an advertisement video according to the attribute of the person estimated by the person attribute estimation means;
Extraction range setting means for the user to arbitrarily set the predetermined range;
Program to function as.

Images