JP6611772B2 - Control method, information processing apparatus, and control program - Google Patents

Description

  The present disclosure relates to a control method, an information processing apparatus, and a control program.

  2. Description of the Related Art In recent years, a digital signage system has been used in which a display device is arranged in a space where a plurality of persons come and go, and an advertisement such as a store, a facility, or a product is displayed for the person who comes and goes in the space.

  Patent Document 1 discloses an advertising system using digital signage that advertises a store to a customer by distributing video and information in a timely manner to a plurality of display terminals arranged in a predetermined area.

JP 2010-210781 A

  The advertising system of patent document 1 can advertise a store to a customer. In the digital signage system, it is required to display an advertisement that matches the needs of a person who can obtain a higher advertising effect among persons browsing the display device.

  The purpose of the control method, the information processing device, and the control program is to provide a system that has a display device that can be viewed by a plurality of people, and a person who can obtain a higher advertising effect among those viewing the display device. It is possible to display an advertisement that matches the needs of the person.

  A control method according to an aspect of the present embodiment is a method for controlling an information processing device that can be connected to an imaging device, a display device, and a storage device, and displays each advertisement for each of a plurality of advertisements on the storage device. A table associated with display conditions for determining whether or not the image is stored, the information processing apparatus acquires an input image captured by the imaging apparatus, detects a person from the input image, and identifies the detected person. Or, it is classified into a plurality of groups, each evaluation score of one or a plurality of groups is calculated, a predetermined number of groups are extracted from the classified groups in descending order of evaluation scores, and extracted from a plurality of advertisements. Displaying an advertisement whose evaluation score of the selected group satisfies the display condition on the display device.

  In the control method, the predetermined number is preferably 1.

  In addition, in the control method, in the group extraction, if there is no advertisement among the plurality of advertisements where the evaluation score of the extracted group satisfies the display condition, the next of the extracted group is selected from the classified groups. It is preferable that a group having a high evaluation score is newly extracted, and in the display, an advertisement satisfying the display condition for the evaluation score of the newly extracted group is displayed on the display device.

  In the control method, the storage device further stores a user table in which registered face information of each user is registered for each of a plurality of users, and the input face of each person detected from the input image in group extraction It is preferable to collate the information with the registered face information of each user registered in the user table, and extract a group from the group including the persons who have been successfully collated.

  In the control method, the storage device stores the schedule information of each user for each of a plurality of users in the user table, and in the group extraction, the schedule information corresponding to the person who has been successfully verified is stored in the storage device for a predetermined time. When a schedule is included, it is preferable to prohibit extraction of a group including the person.

  In the control method, the storage device stores information on advertisements browsed by each user in the past for each of a plurality of users in the user table, and is included in an extracted group of the plurality of advertisements in the display. It is preferable to display on the display device an advertisement that has been browsed in the past by the person to be viewed and the evaluation score of the extracted group satisfies the display condition.

  In the control method, the storage device stores information on stores or facilities used by each user in the past or purchased products for each of a plurality of users in the user table. It is preferable that an advertisement of a store or facility used in the past by a person included in the extracted group or a purchased product and an advertisement in which the extracted group evaluation score satisfies the display condition is displayed on the display device.

  Further, in the control method, it is preferable to determine whether or not a person having the first facial expression is included in the classified group and extract the group based on the determination result.

  In the control method, it is preferable to determine whether or not a person who faces the display device is included in the classified group and extract the group based on the determination result.

  In the control method, the advertisement is a store advertisement, the evaluation score is the number of people in each group, and the display condition is the current vacant seat in the store where the extracted group evaluation score is the target of each advertisement. It is preferable that it is below several.

  In the control method, the storage device stores the number of seats or the current vacancy rate in the store that is the target of each advertisement for each of a plurality of advertisements in the table, and is extracted from the plurality of advertisements in the display. It is preferable to display an advertisement for a store having the highest vacancy rate among advertisements for which the evaluation score of the selected group satisfies the display condition.

  In the control method, it is preferable that the display condition and the evaluation point are based on at least one of the age, sex, physique, or preference of a person included in the group.

  In the control method, the display condition and the evaluation point may be based on at least one of the age variation degree, the gender variation degree, the physique variation degree, or the preference variation degree of the persons included in the group. preferable.

  Further, in the control method, it is preferable to classify the detected persons into one or a plurality of groups only when the number of people imaged within a predetermined time is less than the predetermined number.

  An information processing apparatus according to an aspect of the present embodiment is an information processing apparatus that can be connected to an imaging device, a display device, and a storage device, and displays each advertisement for each of a plurality of advertisements on the storage device. A table associated with a display condition for determining whether or not the image is stored, an acquisition unit that acquires an input image captured by the imaging device, a detection unit that detects a person from the input image, and a detected person A classification unit for classifying into one or a plurality of groups, a calculation unit for calculating each evaluation point of one or a plurality of groups, and a group extraction for extracting a predetermined number of groups from the classified groups in descending order of evaluation points And a display control unit that displays, on the display device, an advertisement in which the evaluation score of the extracted group satisfies the display condition among the plurality of advertisements.

  A control program according to one aspect of the present embodiment is a control program for an information processing device that can be connected to an imaging device, a display device, and a storage device. The storage device stores each advertisement for each of a plurality of advertisements. A table associated with display conditions for determining whether or not to display is stored, the information processing device acquires an input image captured by the imaging device, detects a person from the input image, and detects the detected person Are classified into one or more groups, the respective evaluation points of one or more groups are calculated, a predetermined number of groups are extracted from the classified groups in descending order of evaluation points, Displaying an advertisement on which the evaluation score of the extracted group satisfies the display condition is executed.

  According to the control method, the information processing apparatus, and the control program according to the present embodiment, in a system having a display device that can be browsed by a plurality of persons, a higher advertising effect can be obtained among the people browsing the display device. It is possible to display an advertisement that matches a person's needs according to the person's needs.

It is a schematic diagram for demonstrating the outline | summary of the information provision system 1. FIG. It is a schematic diagram for demonstrating the outline | summary of the information provision system 1. FIG. It is a figure which shows an example of schematic structure of the information provision system. (A), (b) is a figure which shows an example of the data structure of various tables. 2 is a diagram illustrating an example of a schematic configuration of an information processing device 6. FIG. It is a flowchart which shows the example of operation | movement of a display control process. 6 is a diagram showing an example of a display screen 700 displayed on the display device 4. FIG.

  Hereinafter, a control method, an information processing apparatus, and a control program according to an aspect of the present disclosure will be described with reference to the drawings. However, it should be noted that the technical scope of the present disclosure is not limited to the embodiments, and extends to the invention described in the claims and equivalents thereof.

(Outline of information provision system)
Hereinafter, an outline of the information providing system will be described with reference to schematic diagrams shown in FIGS.

  The information providing system includes an imaging device, a display device, and an information processing system. The information processing system includes a storage device and an information processing device. The information processing apparatus can be communicably connected to each of the imaging device, the display device, and the storage device. The display device is arranged in a space where a plurality of persons come and go, such as a concourse such as a hotel, a station or an airport, or a shopping mall, and displays an advertisement of a store, a facility, a product, etc. to the person coming and going in that space It is a digital signage device. The imaging device is arranged in the vicinity of the display device so as to capture a space in which a person who views the advertisement displayed on the display device travels, and generates an input image capturing the space. The storage device is, for example, a database device, and the storage device stores a table in which advertisement contents, display conditions, and the like are associated for each of a plurality of advertisements (FIG. 2). The display condition is a condition for determining whether or not to display each advertisement on the display device.

  The information processing apparatus acquires an input image from the imaging apparatus and detects a person from the acquired input image. Then, the information processing apparatus classifies the detected persons into groups and calculates an evaluation score for each group. The information processing apparatus extracts a predetermined number of groups from the sorted groups in descending order of evaluation score, and extracts an advertisement in which the evaluation score of the extracted group satisfies the display condition. Then, the information processing device displays the extracted advertisement on the display device. For example, the evaluation score is the number of people in each group, and the display condition is that the number of people in the group is equal to or less than the current number of vacant seats in the store. The predetermined number is, for example, 1.

  In the example shown in FIGS. 1 and 2, in the storage device, the advertisement A of the store A having 5 vacant seats, the advertisement B of the store B having 3 vacant seats, and the advertisement C of the store C having 1 vacant seat are stored. Is stored. On the other hand, the person shown in the input image is classified into a group A including four persons, a group B including two persons, and a group C including one person. In this case, the evaluation score of the group A is 4, the evaluation score of the group B is 2, the evaluation score of the group C is 1, and the group A having the highest evaluation score (the largest number of people) is extracted. And the advertisement A of the store A whose number of vacant seats is 5 is extracted from the advertisement memorize | stored in the memory | storage device, and the advertisement A of the store A is displayed on a display apparatus. Note that the advertisement B of the store B having the current number of vacant seats 3 and the advertisement C of the store C having the current number of vacant seats 1 are not extracted and are not displayed on the display device.

  As described above, the information processing apparatus targets a person who constitutes a group having a large number of persons who come and go in the space where the digital signage apparatus is arranged, and is effective for the group (useful for the group). Is displayed on the digital signage device. As a result, the information processing apparatus can display an advertisement that matches the needs of the person with respect to a person who can obtain a higher advertising effect among the persons browsing the digital signage apparatus.

(Configuration of information providing system 1)
FIG. 3 is a diagram illustrating an example of a schematic configuration of the information providing system 1.

  The information providing system 1 includes one or a plurality of display units 2 and an information processing system 3.

  Each display unit 2 includes a display device 4 and an imaging device 5. The information processing system 3 includes an information processing device 6 and a database device 7. The information processing device 6 can be connected to each display device 4, the imaging device 5, and the database device 7 via a network 8 such as an intranet or the Internet.

  The display device 4 is a digital signage device having a display unit such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display device 4 is arranged in a space where a plurality of persons come and go, and displays store advertisements and the like for the people who come and go in the space. The display device 4 further includes a processing unit such as a CPU (Central Processing Unit), a storage unit such as a semiconductor memory, a communication unit, and the like. The processing unit receives display data for displaying the advertisement from the information processing device 6 via the communication unit, stores the display data in the storage unit, and displays the advertisement on the display unit based on the display data.

  The imaging device 5 includes a photoelectric conversion element having sensitivity to visible light such as a CCD (Charge Coupled Device) element or a C-MOS (Complementary Metal Oxide Semiconductor) element, and imaging optics for forming an image on the photoelectric conversion element. This is a camera having a system and an A / D converter that amplifies an electrical signal output from the photoelectric conversion element and performs analog / digital conversion. The imaging device 5 is disposed in the vicinity of the display device 4 belonging to the same display unit 2 and is disposed so as to capture a space in which a person who views an advertisement displayed by the display device 4 comes and goes. The imaging device 5 generates an input image obtained by converting each RGB image obtained by imaging the space into a digital image in which each pixel has a luminance value in the range of 0 to 255. The imaging device 5 further includes a processing unit such as a CPU, a storage unit such as a semiconductor memory, a communication unit, and the like. The processing unit stores the generated input image in the storage unit, and transmits the input image to the information processing device 6 via the communication unit.

  The database device 7 is an example of a storage device, and includes a storage unit including at least one of, for example, a semiconductor memory, a magnetic tape device, a magnetic disk device, or an optical disk device. The storage unit stores an advertisement table, a user table, and the like, which will be described later, as data. The table referred to in the present embodiment is an example of a database format, and may be a format in which data is pooled in the database device 7. The database device 7 further includes a processing unit such as a CPU and a communication unit. The processing unit transmits various information stored in the storage unit to the information processing device 6 via the communication unit in response to a request from the information processing device 6.

(Data structure of various tables)
4A and 4B are diagrams illustrating examples of data structures of various tables. The database device 7 does not store some or all of the various tables described below, and the information processing device 6 or another database device that can be connected to the information processing device 6 can store these various tables. Some or all of them may be stored. Further, each piece of information included in each table may be stored in a format other than the table, for example, may be stored independently.

  FIG. 4A shows an advertisement table. In the advertisement table, for each of a plurality of advertisements registered in the information providing system 1, information such as identification information (advertisement ID), advertisement target identification information (target ID), advertisement contents, and display conditions is displayed. It is stored in association. The advertisement ID is information uniquely assigned to each advertisement. The target ID is information uniquely assigned to each advertising target. The advertisement target is, for example, a store such as a restaurant or a store, a facility such as a hotel or a movie theater, or a product.

  The advertising content is specific content for advertising each advertising target. When the advertising target is a store, for example, store information such as the name, telephone number, address, image, recommended dish or recommended product of each store is stored as the advertisement content. When the advertising target is a facility, facility information such as the name, telephone number, address, image, or recommended point of each facility is stored as the advertisement content. For example, when the advertising target is a product, product information such as the name, image, or recommended point of each product is stored as the advertisement content.

  The display condition is a condition for determining whether or not to display each advertisement. When the advertising target is a restaurant, the display condition is set such that, for example, the number of groups (evaluation points described later) of the target of the advertisement is equal to or less than the current number of seats of each restaurant. In that case, information such as the current number of empty seats and the number of seats of each restaurant is stored in the advertisement table in association with the display conditions. These pieces of information are set at an arbitrary timing from the information processing device 6, a store terminal (not shown) installed in each restaurant, or a portable terminal (not shown) owned by a person concerned in each restaurant.

  The display condition may be set based on at least one of the age, sex, physique, or preference of a person included in the group that is the target of the advertisement. When the advertising target is a store, a facility, or a product, as a display condition, for example, the number or ratio of persons having a preset age, gender, physique, or preference among persons included in a target group of the advertisement (described later) Is set to be equal to or higher than a threshold value. In this case, information such as age, sex, physique or preference setting value and threshold value is stored in the advertisement table in association with the display condition. This set value is not limited to a single value, and may be a plurality of values or a value having a predetermined range.

  Alternatively, the display condition may be set based on at least one of the age variation degree, the gender variation degree, the physique variation degree, or the preference variation degree of the person who displays the advertisement. For example, as the display condition, it is set that the age variation degree, the gender variation degree, the physique variation degree, or the preference variation degree (evaluation point described later) of the person to be displayed as an advertisement is equal to or greater than a threshold value. In that case, information such as a threshold value is stored in the advertisement table in association with the display condition.

  As the display condition, it may be set that any one of the above display conditions or a combination of any two or more thereof is satisfied.

  FIG. 4B shows a user table. In the user table, for each of a plurality of users registered in the information providing system 1, identification information (user ID), name (user name), user information, registered face information, schedule information, browsing information, and usage of each user Information such as information is stored in association with each other. The user ID is information uniquely assigned to each user. The user information is information indicating the age, sex, physique or preference of each user. The registered face information is feature information indicating a registered face image obtained by imaging each user's face registered in advance or a feature point in the registered face image.

  The schedule information indicates each user's schedule and the time at which the schedule is executed. The database device 7 communicates with a mobile terminal (not shown) owned by each user using a communication protocol such as Hypertext Transfer Protocol (HTTP), and is a Web for setting and browsing a schedule. Provide the screen to the mobile device. The schedule information is set at an arbitrary timing from the mobile terminal owned by each user using the Web screen.

  The browsing information indicates the advertisement (ID) browsed by each user in the past and the time when the advertisement was browsed. The database device 7 provides the mobile terminal with a Web screen for displaying the stored advertisement in response to a request from the mobile terminal owned by each user. The browsing information is updated when a web screen for displaying an advertisement is provided on a mobile terminal owned by each user.

  The usage information indicates a store or facility or a purchased product (ID) used by each user in the past, a time when the user uses the store or facility, or a time when a product is purchased. The database device 7 communicates with a terminal installed in each advertisement target store or facility or a store selling each advertisement target product, or a portable terminal (not shown) owned by a person related to each store or facility. Usage information is set from each terminal when each user uses each store or facility or purchases each product.

(Schematic configuration of the information processing apparatus 6)
FIG. 5 is a diagram illustrating an example of a schematic configuration of the information processing apparatus 6.

  The information processing apparatus 6 is a server, a personal computer, or the like, analyzes an input image captured by the imaging apparatus 5, extracts a group that is a target of the advertisement, and is effective for the group (useful for the group). A) The advertisement is displayed on the display device 4. For this purpose, the information processing apparatus 6 includes a communication unit 61, a storage unit 62, and a processing unit 63.

  The communication unit 61 includes a communication interface circuit for connecting the information processing apparatus 6 to the network 8. The communication unit 61 supplies data received from the imaging device 5 or the database device 7 to the processing unit 63. The communication unit 61 transmits the data supplied from the processing unit 63 to the display device 4 or the like.

  The storage unit 62 includes, for example, at least one of a semiconductor memory, a magnetic disk device, and an optical disk device. The storage unit 62 stores a driver program, an operating system program, an application program, data, and the like used for processing by the processing unit 63. The various programs are stored in the storage unit 62 from a computer-readable portable recording medium such as a CD-ROM (Compact Disk Read Only Memory) and a DVD-ROM (Digital Versatile Disk Read Only Memory) using a known setup program. May be installed. In addition, the storage unit 62 temporarily stores temporary data related to predetermined processing.

  The processing unit 63 includes one or a plurality of processors and their peripheral circuits. The processing unit 63 controls the overall operation of the information processing apparatus 6 and is, for example, a CPU. The processing unit 63 controls the operation of the communication unit 61 and the like so that various processes of the information processing device 6 are executed in an appropriate procedure based on a program or the like stored in the storage unit 62. The processing unit 63 executes processing based on programs (operating system program, driver program, application program, etc.) stored in the storage unit 62. The processing unit 63 can execute a plurality of programs (such as application programs) in parallel.

  The processing unit 63 includes an acquisition unit 631, a detection unit 632, a classification unit 633, a calculation unit 634, a group extraction unit 635, an advertisement extraction unit 636, a display control unit 637, and the like. Each of these units included in the processing unit 63 is a functional module implemented by a program executed on the processor included in the processing unit 63. Alternatively, these units included in the processing unit 63 may be implemented in the information processing apparatus 6 as independent integrated circuits, microprocessors, or firmware.

(Example of operation of display control processing by information processing device 6)
FIG. 6 is a flowchart illustrating an example of operation of display control processing by the information processing apparatus 6. This operation flow is executed mainly by the processing unit 63 in cooperation with each element of the information processing apparatus 6 based on a program stored in the storage unit 62 in advance.

  First, the acquisition unit 631 acquires an input image captured by the imaging device 5 from the imaging device 5 via the communication unit 61, and stores the input image in the storage unit 62 (step S101).

  Next, the detection unit 632 detects a person from the acquired input image (step S102). The detection unit 632 extracts, for example, a change area having a temporal change in luminance value in a plurality of input images acquired from the imaging device 5 using, for example, inter-frame difference processing or background difference processing. The detection unit 632 detects a person from the input image by extracting, from the extracted change areas, a change area that seems to be a person from a feature amount such as the size of the change area, as a person area.

  The detection unit 632 may further identify each person by comparing the input face information of each person detected from the input image with the registered face information of each user registered in the user table. The detection unit 632 extracts edge pixels having a luminance value difference equal to or greater than a predetermined value with respect to the extracted pixels from the surrounding pixels. The detection unit 632 detects an elliptical edge distribution having a predetermined size and approximating the contour shape of the head from the extracted edge pixels, and extracts an area surrounded by the edge distribution as a face area. The detecting unit 632 detects an elliptical edge distribution using, for example, a generalized Hough transform.

  The detection unit 632 receives the registered face information of each user registered in the information providing system 1 from the database device 7 via the communication unit 61, and the input face information in the extracted face area and the received registered face information The similarity with each of the above is calculated. The detection unit 632 calculates a characteristic amount specific to a human face in the extracted face area, and normalizes a difference between the characteristic amount calculated from the extracted face area and the characteristic amount based on the received registered face information Is calculated as the similarity. The feature amount is, for example, the position and / or shape of a characteristic part of the face such as the eyes, nose, and mouth. For example, the detection unit 632 extracts an edge pixel from the face area, and if the position and / or pattern of the edge pixel satisfies a predetermined condition for a part such as an eye, nose, or mouth, the position is detected. Calculated as a feature quantity. The detection unit 632 may calculate the extracted face area and the normalized cross-correlation value of the registered face image as the similarity.

  When the highest similarity among the calculated similarities is equal to or greater than a predetermined threshold, the detection unit 632 indicates that the person shown in the extracted face area is a user corresponding to the registered face information for which the similarity is calculated. It is determined that there is a match and the verification is successful.

  Next, the classification unit 633 classifies the persons detected by the detection unit 632 into one or a plurality of groups (step S103). For example, the classification unit 633 has a distance (for example, a distance between centroid positions) between each person region extracted by the detection unit 632 and a size (for example, an area) that is equal to or less than a predetermined distance. It is determined whether there is another person area whose difference is equal to or smaller than a predetermined size. Then, the classification unit 633 classifies the person regions whose distance is equal to or smaller than the predetermined distance and whose size difference is equal to or smaller than the predetermined size into the same group. In addition, when there is another person area whose distance is equal to or smaller than a predetermined distance and whose size difference is equal to or smaller than a predetermined size with respect to the person area classified into a specific group, the classification unit 633 also selects the person area. Classify into that particular group.

  Next, the calculation unit 634 calculates the evaluation score of each group classified by the classification unit 633 (step S104). For example, the calculation unit 634 uses the number of people included in each group as an evaluation score.

  Note that the calculation unit 634 may calculate an evaluation score based on at least one of the age, sex, physique, or preference of a person included in each group. In that case, the calculation unit 634 receives the user information of each user specified by the detection unit 632 from the database device 7 via the communication unit 61, and from the received user information of each user, the person information included in each group. Identify age, gender, physique or preference. Further, the calculation unit 634 receives each information included in the advertisement table from the database device 7 via the communication unit 61, and among the persons included in each group, the age, gender, The number or ratio of persons having a physique or preference setting value is calculated as an evaluation score.

  Note that the calculation unit 634 may specify the age, sex, or physique of a person included in each group using a known image processing technique. In that case, when an image is input, the age, gender, or physique of each person is determined using a classifier that has been pre-learned to output the age, gender, or physique of the person in the input image. Identify. This discriminator is pre-learned using a plurality of images including persons having various ages, genders or physiques, for example, by deep learning, and stored in the storage unit 62 in advance. The calculation unit 634 inputs each person region extracted from the input image to the discriminator, and specifies the age, gender, or physique of each person from the information output from the discriminator.

  Alternatively, the calculation unit 634 may calculate the evaluation score based on at least one of the age variation degree, the gender variation degree, the physique variation degree, or the preference variation degree of the person included in each group. . For example, the calculating unit 634 uses the age variation degree, the gender variation degree, the physique variation degree, or the preference variation degree of each person included in each group as evaluation points. As the degree of variation, the reciprocal of variance or deviation is used so that the smaller the variation is, the larger the value is. Alternatively, as the degree of variation, for example, variance or deviation may be used so as to have a larger value as the variation is larger.

  Next, the group extraction unit 635 extracts a predetermined number of groups in descending order of evaluation score from the groups classified by the classification unit 633 (step S105). The predetermined number is, for example, 1. The predetermined number may be a value of 2 or more.

  Next, the advertisement extraction unit 636 extracts, from the advertisements included in the advertisement table, an advertisement in which the evaluation score of any group extracted by the group extraction unit 635 satisfies the display condition (Step S106). The advertisement extraction unit 636 receives each piece of information included in the advertisement table from the database device 7 via the communication unit 61, and based on the received information, the evaluation score of any group extracted by the group extraction unit 635 Extracts advertisements that satisfy the display condition.

  For example, when the display condition is that the number of people included in each group is equal to or less than the current number of vacant seats of each advertisement target restaurant, the advertisement extracting unit 636 may calculate the number of vacant seats as the evaluation score. The advertisement of the store which is more than the number of people included in is extracted. Thereby, when the person who browsed the advertisement visits the store, the information processing apparatus 6 can display the advertisement of the store that is highly likely to provide the service immediately.

  In that case, the advertisement extracting unit 636 may be an advertisement of a store having the highest current vacancy rate among the advertisements included in the advertisement table, among the advertisements whose group evaluation points extracted by the group extracting unit 635 satisfy the display conditions. May be extracted. The advertisement extraction unit 636 calculates the current vacancy rate from the number of vacant seats and the number of seats in each store. The database device 7 may store the current vacancy rate instead of the number of seats in each store in the advertisement table, and the advertisement extraction unit 636 may acquire the current vacancy rate from the database device 7. As a result, the information processing device 6 can display an advertisement of a store that strongly expects a visitor.

  In addition, when the display condition is that the number or ratio of persons having a predetermined age, gender, physique, or preference among the persons included in each group is a threshold value or more, the advertisement extraction unit 636 corresponds to each advertisement. The advertisement whose number or ratio calculated as an evaluation score with respect to the set value of age, sex, physique, or preference is equal to or more than a threshold corresponding to each advertisement is extracted. In addition, when the display condition is that the variation degree of the age, sex, physique, or preference of the person included in each group is equal to or greater than the threshold, the advertisement extraction unit 636 indicates that the variation degree calculated as the evaluation score is The advertisement which becomes more than the threshold value corresponding to is extracted.

  Next, the processing unit 63 determines whether or not an advertisement satisfying the display condition of the evaluation score of any group extracted by the group extraction unit 635 exists in the advertisement included in the advertisement table (Step S63). S107).

  When the advertisement whose evaluation score satisfies the display condition does not exist, the processing unit 63 shifts the process to step S105 and repeats the processes of steps S105 to S107. In that case, in step S105, the group extraction unit 635 newly extracts a group having the next highest evaluation score after the already extracted group from the groups classified by the classification unit 633. In step S <b> 106, the advertisement extraction unit 636 extracts an advertisement in which the evaluation score of the group newly extracted by the group extraction unit 635 satisfies the display condition from the advertisements included in the advertisement table.

  On the other hand, when there is an advertisement whose evaluation score satisfies the display condition, the display control unit 637 transmits display data for displaying the advertisement extracted by the advertisement extraction unit 636 to the display device 4 via the communication unit 61. The advertisement is displayed on the display device 4 (step S108), and the series of steps is completed.

  The display control unit 637 transmits display data to the display device 4 that belongs to the same display unit 2 as the imaging device 5 that captured the input image. Note that the display control unit 637 may transmit display data to another display device 4. In that case, the information processing device 6 stores in advance the imaging direction of each imaging device 5, the arrangement position of each display device 4, and the like. The display control unit 637 performs tracking processing of the group extracted by the group extraction unit 635 using a known image processing technique, and estimates the traveling direction of the group. The display control unit 637 is arranged on one or a plurality of display devices 4 that are arranged on the estimated traveling direction side with respect to the imaging device 5 that has captured the input image and that are disposed within a predetermined distance from the imaging device 5 that has captured the input image. Send display data.

  FIG. 7 is a diagram illustrating an example of a display screen 700 displayed on the display device 4.

  As shown in FIG. 7, the advertisement content of the advertisement extracted by the advertisement extraction unit 636 is displayed on the display screen 700. For example, when the extracted advertisement is a store advertisement, store information such as a store name, a telephone number, an address, an image, and a recommended dish is displayed.

  Note that when the person included in each group includes a user registered in the information providing system 1, the information processing apparatus 6 may display the user ID of the user on the display screen 700. As a result, the user recognizes that the advertisement is displayed to him and is more likely to be interested in the displayed advertisement. The database device 7 stores an image related to the user (for example, a person in charge of the user in the store) for each advertisement target in the user table, and the information processing device 6 relates to the user on the display screen 700. An image to be displayed may be displayed. This also increases the possibility that the user is interested in the displayed advertisement.

  Further, the information processing apparatus 6 may display a URL (Uniform Resource Locator) or the like of a Web page indicating details of the advertisement on the display screen 700. Thereby, the person who browsed the display screen 700 can access the Web page and acquire detailed information, and the information processing apparatus 6 can improve convenience.

  In step S <b> 105, the group extraction unit 635 may extract a group from a group including a person who has been successfully verified with the user registered in the information providing system 1 by the detection unit 632. Thereby, since the information processing apparatus 6 targets a user who is registered in advance in the information providing system 1 in order to acquire information from the advertisement, it is possible for a user who is highly likely to obtain a higher advertising effect. An advertisement can be displayed.

  In addition, when the schedule information corresponding to the person who has been successfully verified with the user registered in the information providing system 1 includes a schedule within a predetermined time (for example, one hour) from the current time, the group extraction unit 635 displays the person. Extraction of included groups may be prohibited. The group extraction unit 635 receives the collated user schedule information from the database device 7 via the communication unit 61, and determines whether the received schedule information includes a schedule within a predetermined time from the present time. Thereby, the information processing apparatus 6 can suppress displaying an advertisement to a user who has a plan and cannot expect a store visit and has a low advertising effect.

  Further, the group extraction unit 635 may determine whether or not a person having a specific facial expression is included in the classified group, and extract the group based on the determination result. The group extraction unit 635 prohibits extraction of a group including a person having a negative specific facial expression (first facial expression) such as angry or sad. In addition, the group extraction unit 635 may extract a group from a group including a person having a positive specific facial expression (second facial expression) such as laughing or joyful. In that case, when the image is input, the group extraction unit 635 has a certainty that the person having the specific expression is reflected in the input image, or the person in the input image is the specific expression. Determine whether each group contains a person with a specific facial expression using a classifier that has been pre-learned to output the degree of anger (degree of anger, degree of laughter, etc.) To do. This discriminator is pre-learned using a plurality of images including a person having a specific facial expression by, for example, deep learning, and stored in the storage unit 62 in advance. The group extraction unit 635 extracts an area including each group from the input image, inputs the extracted area to the classifier, and determines whether each group includes a person having a specific facial expression based on information output from the classifier. To do.

  Note that the group extraction unit 635 has a statistical value (average value, total value, variance value, etc.) or a maximum value or minimum value of the degree that each person included in the group has a specific facial expression is greater than or equal to a threshold value. Group extraction may be prohibited. Alternatively, the group extraction unit 635 may prohibit extraction of a group in which the number or ratio of persons having a specific facial expression included in the group is greater than or equal to a threshold value. Alternatively, the group extraction unit 635 may extract a group from a group in which the statistical value of the degree that each person included in the group has a specific facial expression, or the maximum value or the minimum value is equal to or greater than a threshold value. . Alternatively, the group extraction unit 635 may extract a group from groups in which the number or ratio of persons having a specific facial expression included in the group is equal to or greater than a threshold value. As a result, the information processing device 6 can suppress the display of the advertisement to a user who is unlikely to browse the advertisement and has a low advertisement effect, or a user who is highly likely to use or purchase the advertisement target. It is possible to display an advertisement for.

  Further, the group extraction unit 635 may determine whether or not the classified group includes a person facing the front (display device 4 side), and may extract the group based on the determination result. The group extraction unit 635 extracts a group from a group including, for example, a person facing the front (display device 4 side). In that case, when an image is input, the group extraction unit 635 uses each classifier that has been pre-learned so as to output the probability that a person facing the front is reflected in the input image. It is determined whether or not the group includes a person facing the front. This discriminator is pre-learned using a plurality of images including a person facing the front by, for example, deep learning, and stored in the storage unit 62 in advance. The group extraction unit 635 extracts an area including each group from the input image, inputs the extracted area to the classifier, and determines whether each group includes a person facing the front from the information output from the classifier. To do. Note that the group extraction unit 635 may extract a group from groups in which the number or ratio of persons facing the front included in the group is equal to or greater than a threshold value. Thereby, the information processing device 6 can display the advertisement to a user who is highly likely to view the advertisement.

  In step S106, the advertisement extraction unit 636 may extract an advertisement from advertisements browsed in the past by a person included in the group extracted by the group extraction unit 635. The advertisement extraction unit 636 receives browsing information of a user who has been successfully verified from the database device 7 via the communication unit 61, and identifies an advertisement that the user has browsed in the past from the received browsing information. In this case, in step S107, the display control unit 637 displays an advertisement browsed in the past by a person included in the group on the display device 4. Thereby, the information processing apparatus 6 can display an advertisement in which the target person is interested.

  In addition, the advertisement extraction unit 636 may extract an advertisement from advertisements of stores or facilities used in the past by a person included in the group extracted by the group extraction unit 635 or purchased products. The advertisement extraction unit 636 receives the usage information of the user who has been successfully verified from the database device 7 via the communication unit 61, and from the received usage information, stores or facilities that the user has used in the past or purchased products. Identify. In that case, in step S107, the display control unit 637 displays on the display device 4 an advertisement of a store or facility used in the past by a person included in the group or a purchased product. Thereby, the information processing apparatus 6 can display an advertisement in which a person to be displayed is interested.

  In step S103, the classification unit 633 may classify the detected persons into one or a plurality of groups only when the number of people captured within a predetermined time is less than the predetermined number. As a result, the information processing apparatus 6 suppresses erroneously grouping unrelated persons and targeting an inappropriate group as an advertisement target when a large number of persons appear in the input image. It becomes possible.

  As described in detail above, the information processing apparatus 6 is a group having a high evaluation score among a group of persons extracted from an input image in the information providing system 1 having the display apparatus 4 (digital signage) that can be viewed by a plurality of persons. An advertisement whose evaluation score satisfies the display condition is displayed. As a result, the information processing apparatus 6 can display an advertisement that matches the needs of the person who can obtain a higher advertising effect among the persons browsing the display apparatus 4.

  The information processing device 6 and the database device 7 may be configured as the same device. In that case, each table stored in the database device 7 is stored in the storage unit 62 of the information processing device 6, and the information processing device 6 reads out each table from the storage unit 62, whereby each information included in each table is stored. To get.

  Further, the information processing device 6 and the display device 4 and / or the imaging device 5 may be configured as the same device. In that case, each part of the information processing device 6 and each part of the display device 4 and / or the imaging device 5 are connected via a bus, and the information processing device 6 is connected to the display device 4 and / or the imaging device 5 via the bus. Send and receive various data.

  It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made thereto without departing from the spirit and scope of the present invention.

DESCRIPTION OF SYMBOLS 4 Display apparatus 5 Imaging apparatus 6 Information processing apparatus 7 Database apparatus 631 Acquisition part 632 Detection part 633 Classification part 634 Calculation part 635 Group extraction part 636 Advertisement extraction part 637 Display control part

Claims (16)

A method for controlling an information processing device connectable to an imaging device, a display device, and a storage device,
The storage device stores, for each of a plurality of advertisements, a table associated with display conditions for determining whether to display each advertisement.
The information processing apparatus is
Obtaining an input image captured by the imaging device;
Detecting a plurality of persons from the input image;
Classifying the detected plurality of persons into a plurality of groups composed of one or more persons;
Calculating the respective evaluation points of the previous group Kifuku number,
Extracting a predetermined number of groups less than the number of the classified groups in descending order of the evaluation score from the classified groups;
Of the plurality of advertisements, the display device displays an advertisement whose evaluation score of the extracted group satisfies the display condition.
A control method for an information processing apparatus.   The control method according to claim 1, wherein the predetermined number is one. In the group extraction, if there is no advertisement among the plurality of advertisements that has an evaluation score of the extracted group that satisfies the display condition, the next of the extracted group from the classified group. Newly extracted groups with high evaluation points,
3. The control method according to claim 1, wherein, in the display, an advertisement in which the evaluation score of the newly extracted group satisfies the display condition among the plurality of advertisements is displayed on the display device. The storage device further stores a user table in which registered face information of each user is registered for each of a plurality of users.
In the extraction of the group, the input face information of each person detected from the input image is collated with the registered face information of each user registered in the user table, and the group including persons who have been successfully collated is included. The control method according to claim 1, wherein groups are extracted. The storage device stores schedule information of each user for each of the plurality of users in the user table,
5. The control method according to claim 4, wherein in the extraction of the group, when the schedule information corresponding to the person who has succeeded in the collation includes a schedule within a predetermined time from the present time, the extraction of the group including the person is prohibited. . In the user table, for each of the plurality of users, information on advertisements browsed in the past by each user is stored in the storage device,
In the display, an advertisement that has been browsed in the past by a person included in the extracted group among the plurality of advertisements and an advertisement satisfying the display condition is displayed on the display device. The control method according to claim 4 or 5. In the user table, for each of the plurality of users, information on stores or facilities used by each user in the past or purchased products is stored in the storage device,
In the display, among the plurality of advertisements, a person included in the extracted group is an advertisement of a store or facility used in the past or a purchased product, and the evaluation score of the extracted group satisfies the display condition. The control method according to claim 4, wherein a satisfying advertisement is displayed on the display device.   In the extraction of the group, it is determined whether or not a person having a specific expression is included in the classified group, and the group is extracted based on the determination result. Control method.   In the group extraction, it is determined whether or not the classified group includes a person facing the display device, and the group is extracted based on the determination result. The control method according to item. The advertisement is a store advertisement,
The evaluation score is the number of people in each group,
The control method according to any one of claims 1 to 9, wherein the display condition is that an evaluation score of the extracted group is equal to or less than a current number of vacant seats in a store that is a target of each advertisement. In the table, for each of the plurality of advertisements, the number of seats in the store that is the target of each advertisement or the current vacancy rate is stored in the storage device,
11. The display according to claim 10, wherein, in the display, an advertisement of a store having the highest vacancy rate is displayed on the display device among advertisements in which the evaluation score of the extracted group satisfies the display condition among the plurality of advertisements. The control method described.   The control method according to any one of claims 1 to 11, wherein the display condition and the evaluation score are based on at least one of age, sex, physique, or preference of a person included in the group.   The display condition and the evaluation point are based on at least one of a degree of variation in age of persons included in the group, a degree of variation in gender, a degree of variation in physique, or a degree of variation in preference. The control method according to any one of 12 above.   The control according to any one of claims 1 to 13, wherein, in the classification, the detected persons are classified into one or a plurality of groups only when the number of people imaged within a predetermined time is less than a predetermined number. Method. An information processing device connectable to an imaging device, a display device, and a storage device,
The storage device stores, for each of a plurality of advertisements, a table associated with display conditions for determining whether to display each advertisement.
An acquisition unit that acquires an input image captured by the imaging device;
A detection unit for detecting a plurality of persons from the input image;
A classification unit for classifying the detected plurality of persons into a plurality of groups including one or more persons;
A calculation unit that calculates respective evaluation points of the previous group Kifuku number,
A group extraction unit that extracts a predetermined number of groups smaller than the number of the classified groups in descending order of the evaluation score from the classified groups;
A display control unit that causes the display device to display an advertisement in which the evaluation score of the extracted group satisfies the display condition among the plurality of advertisements;
An information processing apparatus comprising: A control program for an information processing device connectable to an imaging device, a display device, and a storage device,
The storage device stores, for each of a plurality of advertisements, a table associated with display conditions for determining whether to display each advertisement.
In the information processing apparatus,
Obtaining an input image captured by the imaging device;
Detecting a plurality of persons from the input image;
Classifying the detected plurality of persons into a plurality of groups composed of one or more persons;
Calculating the respective evaluation points of the previous group Kifuku number,
Extracting a predetermined number of groups less than the number of the classified groups in descending order of the evaluation score from the classified groups;
Of the plurality of advertisements, the display device displays an advertisement whose evaluation score of the extracted group satisfies the display condition.
A control program characterized by causing

Images