JP2022067773A - Display control system and display control program - Google Patents

Abstract

To provide a display control system that guides a customer to a non-congested facility without displaying degree of congestion of a facility in a display control system and a display control program.SOLUTION: Each signage installed in each store calculates, on the basis of on selection conditions including gender and age of a customer in front of the signage and a degree of congestion at other signage-installed stores, content selection information (content selection probability of the other stores) for the signage (S4), and displays, on the basis of the content selection information, content (interrupted content) on the other signage-installed stores (other stores) (S6). Thereby, each signage can guide, without displaying congestion level of a signage-installed store (especially information that the store is vacant), the customer to a non-congested and safe store that meets the above selection conditions among other signage-installed stores (other stores).SELECTED DRAWING: Figure 8

Description

The present invention relates to a display control system and a display control program.

Recently, the number of customers visiting commercial complexes has decreased due to the increase in the number of people infected with the so-called new coronavirus (COVID-19). This means that customers are avoiding crowded areas and areas with many non-mask wearers.

There is a system that displays the degree of congestion (availability) of a facility for the purpose of guiding the customer to a facility that is not congested under the situation where the customer avoids a crowded place as described above. As an example of the system for displaying the degree of congestion of the facility as described above, for example, the system described in Patent Document 1 below can be mentioned.

Japanese Patent No. 6153177

However, the facility manager who feels resistance to the method of clearly displaying the information (congestion degree) that the facility is vacant, such as the system shown in Patent Document 1, is uncool. Due to the large number, such systems are not widely used.

The present invention solves the above-mentioned problems, and an object of the present invention is to provide a display control system and a display control program capable of guiding a customer to a non-congested facility without displaying the degree of congestion of the facility. And.

In order to solve the above problems, the display control system according to the first aspect of the present invention is based on a display device having a photographing means installed in each of at least two or more facilities and an image captured by the photographing means. An estimation means that detects a person in front of each of the display devices and estimates the attributes of the detected person, and a person detected by the estimation means that enters the facility where each of the display devices is installed. Photographing of the display device for each of the congestion degree determination means for determining the congestion degree of the facility in which each of the display devices is installed based on the number of people and the display devices installed in the two or more facilities. A selection condition including the attribute of a person in front of the display device estimated by the estimation means from an image captured by the means and a display device other than the display device determined by the congestion degree determination means were installed. Content selection information generation means for generating content selection information, which is information for selecting content to be displayed on the display device based on the degree of congestion of the facility, and display devices installed in the two or more facilities, respectively. However, the display control means for controlling to display the content related to the facility in which the display device other than the display device is installed is provided based on the content selection information.

In this display control system, the display control means is based on the content selection information while each of the display devices installed in the two or more facilities displays the content related to the facility in which the display device is installed. Therefore, it may be controlled to display the content related to the facility in which the display device other than the display device is installed.

In this display control system, the content selection information generation means has a selection condition including an attribute of a person in front of the display device for each of the display devices installed in the two or more facilities, and the other display devices. It is desirable to generate the content selection information based on the degree of congestion of the facility in which the display device is installed, as determined by the degree of congestion determination means, in addition to the degree of congestion of the facility in which the display device is installed.

In this display control system, the selection condition used by the content selection information generation means includes the attribute of the person in front of the display device, the time zone at that time, and the facility where the display device is installed. It is desirable to include at least one of the conditions of the local weather where the display device is located and the location of the facility where the display device is installed.

In this display control system, the content selection information generation means vectorizes and inputs the selection condition, and outputs a neural network that outputs the selection probability of the content related to the facility in which the display device other than the display device is installed. You may be prepared.

In this display control system, when the above-mentioned neural network is provided, for each of the display devices installed in the two or more facilities, from the human face image in the image taken by the photographing means of the display device, the person's face image is used. I was in front of a facility where a display device was installed using the face recognition information generation means that generates face recognition information for the person's face recognition and the face recognition information generated by the face recognition information generation means. In front of a display device estimated by the estimation means from a movement detection means for detecting whether or not a person has moved to a facility in which another display device is installed and an image captured by the photographing means of the display device. The movement detection means detects the selection conditions including the attributes of the person who was in the room and whether or not the person who was in front of the facility where the certain display device was installed moved to the facility where the other display device was installed. It is desirable to further provide a training data set generation means for automatically generating a training data set of the neural network model based on the results.

The display control program according to the second aspect of the present invention is a display control program that controls switching of contents to be displayed on a display device having a photographing means installed in each of at least two or more facilities, and is a display control program that controls a computer. Of the estimation means that detects a person in front of each of the display devices and estimates the attributes of the detected person from the image captured by the photographing means, and the person detected by the estimation means, the display device. A congestion degree determination means for determining the degree of congestion of the facility in which each of the display devices is installed based on the number of people who have entered the facility in which each is installed, and a display device installed in the two or more facilities. For each of the above, the selection condition including the attribute of the person in front of the display device estimated by the estimation means from the image taken by the shooting means of the display device, and the display device other than the display device determined by the congestion degree determination means. A content selection information generation means that generates content selection information, which is information for selecting content to be displayed on the display device, based on the degree of congestion of the facility where the other display device is installed, and the two or more of the above. Each of the display devices installed in the facility functions as a display control means for controlling the display of the content related to the facility in which the display device other than the display device is installed, based on the content selection information.

According to the display control system according to the first aspect of the present invention and the display control program according to the second aspect, each of the display devices installed in two or more facilities has the attribute of the person in front of the display device. Control to display the content related to the facility where the other display device other than the display device is installed, based on the selection condition including the display device and the degree of congestion of the facility where the display device other than the display device is installed. do. As a result, each of the above display devices installs a display device other than the display device without displaying the degree of congestion of the facility where the display device is installed (particularly, information that the facility is vacant). It is possible to guide customers to a non-congested facility that meets the above selection conditions among the facilities that have been opened.

The block block diagram which shows the schematic structure of the display control system of one Embodiment of this invention. The block diagram which shows the schematic hardware configuration of the signage in FIG. The block diagram which shows the hardware configuration of the signage management server in FIG. The functional block configuration diagram of the above signage and the signage management server. The functional block block diagram in the content selection information generation part in FIG. Explanatory diagram of the content list delivered to each signage from the above signage management server. Explanatory drawing of display control processing according to the said content list in the said signage. The flowchart of the display control processing of the interrupt content to each signage performed in the said display control system. Explanatory diagram of customer attribute estimation processing in the above signage. An explanatory diagram of the flow up to interrupt content selection in the above content selection information generation unit. An explanatory diagram of the content display in the above signage. The block diagram of the functional block necessary for the automatic generation and learning of the learning data set in the signage and the signage management server. The time chart of the automatic generation processing of the learning data set of the content selection NN in FIG.

Hereinafter, a display control system and a display control program according to an embodiment embodying the present invention will be described with reference to the drawings. FIG. 1 is a block configuration diagram showing a schematic configuration of a display control system 10 according to the present embodiment. As shown in FIG. 1, the display control system 10 mainly includes signage 1a, 1b, etc., which are tablet terminals for digital signage installed in each store (“facility” in the claim) Sa, Sb, etc. It is composed of these signage 1a, 1b and the like and a signage management server 5 connected via the Internet. In the following description, signage 1 is a general term for signage 1a, 1b and the like, and store S is a general term for stores Sa, Sb and the like. The display control system 10 includes one or more signage 1s (“display device” in the claims) and a wireless LAN router 3 in each store S. The signage 1 includes a built-in camera 2 (“shooting means” in the claims).

The signage 1 is installed in a place where an image can be taken around the entrance / exit such as in front of the store S, and a customer (in front of the signage 1) who visits the store (in front of the signage 1) on the touch panel display 14 (see FIG. 2) (claim). In addition to displaying contents such as advertisements for "people") in the above, the customer reflected in the frame image is detected based on the frame image from the built-in camera 2, and the attribute of the detected customer is estimated.

The above-mentioned signage management server 5 is a server for signage management installed in a management department (head office or the like) of a store S. Although the details will be described later, the signage management server 5 receives customer information such as the above-mentioned customer attributes from each signage 1, and information for selecting the content to be displayed on each signage 1 based on the customer information. The content selection probability (“content selection information” in the claim) is generated. The signage 1 and the signage management server 5 described above correspond to the "computer" in the claims.

Next, with reference to FIG. 2, the hardware configuration of the above-mentioned tablet type signage 1 will be described. In addition to the built-in camera 2 described above, the signage 1 includes a System (System-on-a-Chip) 11, a touch panel display 14, a speaker 15, a memory 16 for storing various data and programs, and a communication unit 17. , A secondary battery 18, and a charging terminal 19. The SoC 11 includes a CPU 12 that controls the entire device and performs various operations, and a GPU 13 that is used for inference processing of various trained DNN (Deep Neural Networks) models.

The program stored in the memory 16 includes the signage side display control program 20. The signage side display control program 20 includes various inference models. The communication unit 17 includes a communication IC and an antenna. The signage 1 is connected to the signage management server 5 on the cloud via the communication unit 17 and the Internet. Further, the secondary battery 18 is a battery such as a lithium ion battery that can be repeatedly used by charging, and stores power from a commercial power source after being converted into DC power by an AC / DC converter. It is supplied to each part of the signage 1.

Next, the hardware configuration of the signage management server 5 will be described with reference to FIG. The signage management server 5 includes a CPU 21 that controls the entire device and performs various operations, a hard disk 22 that stores various data and programs, a RAM (Random Access Memory) 23, a display 24, an operation unit 25, and a communication unit. It has 26 and.

The program stored in the hard disk 22 includes the server-side display control program 27. The server-side display control program 27 and the signage-side display control program 20 correspond to the display control program in the claims.

FIG. 4 shows the functional blocks (during normal operation) of the signage 1 and the signage management server 5 described above. In addition to the above-mentioned built-in camera 2, touch panel display 14, and speaker 15, the signage 1 has a video input unit 31, an estimation unit 32 (“estimation means” in the claims), and a content display control unit 34 as functional blocks. (Part of the "display control means" in the claims). Further, the signage management server 5 includes a content selection information generation unit 37 (“content selection information generation means” in the claim) as a functional block. The content selection information generation unit 37 includes a (learned) content selection NN (Neural Networks) 38 (“neural network” in the claims).

The video input unit 31 is mainly realized by SoC11 (including an I / O chipset (not shown) in FIG. 2 and receives (data) of a video stream input from the built-in camera 2 of each signage 1. And decode it to make a frame image (data). The estimation unit 32 detects a person (customer) in front of the signage 1 from the frame image (photographed image) taken by the built-in camera 2 of each signage 1 and estimates the attribute of the detected customer. The customer attributes estimated by the estimation unit 32 include gender, age, whether or not a mask is worn, and whether or not disinfection (of fingers with alcohol) is performed.

The content selection information generation unit 37 on the signage management server 5 side estimates each signage 1 installed in two or more stores S from the frame image (photographed image) taken by the built-in camera 2 of the signage 1. , Select the content to be displayed on the signage 1 based on the selection conditions including the gender and age of the customer in front of the signage 1 and the degree of congestion of the store S in which the signage 1 other than the signage 1 is installed. Generate content selection information (content selection probability), which is information for signage. For example, the content selection information generation unit 37 estimates the signage 1a installed in the store Sa in FIG. 1 from the frame image (photographed image) taken by the built-in camera 2 of the signage 1a, and the signage 1a is estimated by the estimation unit 32. Store Sb, etc. where other signage 1b, etc. are installed, determined based on the selection conditions including the gender and age of the person in front and the number of customers visiting the store transmitted from other signage 1b, etc. other than the signage 1a (others). Based on the degree of congestion of the store), content selection information which is information for selecting the content to be displayed on the signage 1a is generated.

The content display control unit 34 on the signage 1 side, together with the server side display control unit 47 in FIG. 5 described later, has each signage 1 installed in each store S, and the store S (own store) in which the signage 1 is installed. Regarding the store S (other stores around the own store) in which other signage 1 other than the signage 1 is installed, based on the above content selection information (content selection probability of other stores) while displaying the content related to. Control the display of content. For example, the content display control unit 34 and the server-side display control unit 47 may display the above-mentioned content selection information while the signage 1a in FIG. 1 displays the content related to the store Sa (own store) in which the signage 1a is installed. Based on (the selection probability of the content of another store), the content related to the store Sb or the like (other stores around the own store) in which the signage 1b or the like other than the signage 1a is installed is controlled to be displayed.

Among the functional blocks of the signage 1, the video input unit 31, the estimation unit 32, and the content display control unit 34 are mainly realized by the SoC 11 in FIG. Further, the content selection information generation unit 37 on the signage management server 5 side is mainly realized by the CPU 21 and the communication unit 26 in FIG.

FIG. 5 shows an example of a functional block in the content selection information generation unit 37 of the signage management server 5. In this example, the content selection information generation unit 37 of the signage management server 5 is a signage customer information receiving unit 41, a congestion degree / risk degree determination unit 42 (congestion degree determination means), a content selection probability calculation unit 43, and an interrupt content determination unit. It includes 44, an interrupt content distribution unit 45, and a content list distribution unit 46. Of these functional blocks, the interrupt content determination unit 44, the interrupt content distribution unit 45, and the content list distribution unit 46 constitute the server-side display control unit 47 shown by the broken line. The server-side display control unit 47 functions as the "display control means" in the claims together with the content display control unit 34 on the signage 1 side. Further, the signage management server 5 has a signage customer information file (referred to as “signage customer information F” in FIG. 5 and the following description) 48 and a content information file (FIG. 5 and FIG. 3) on the hard disk 22 (see FIG. 3). In the following description, (referred to as "content information F") 49 is stored. The signage customer information F48 stores customer information such as customer attributes and store visitor detection information transmitted from the signage group (signage 1a, 1b, etc.). The content information F49 stores various contents to be sent to each signage 1 and a content list which is a list of contents related to the own store displayed by the signage 1 of each store S.

In the example shown in FIG. 5, the content selection information generation unit 37 of the signage management server 5 contains the signage customer information receiving unit 41, the congestion / risk determination unit 42, the content selection probability calculation unit 43, and the server side display. An example is shown in the case where the functional blocks of the control unit 47 (interruption content determination unit 44, interrupt content distribution unit 45, and content list distribution unit 46) are provided. Among these functional blocks, the signage customer information receiving unit 41 is shown. The congestion / risk determination unit 42 and the server-side display control unit 47 do not necessarily have to be provided in the content selection information generation unit 37. Therefore, in the claim, the content selection information generation means corresponding to the content selection information generation unit 37, the congestion degree determination means corresponding to the congestion degree / risk degree determination unit 42, the server-side display control unit 47, and the content display control unit. The display control means corresponding to 34 is described as a separate configuration requirement.

For convenience of explanation, among the functional blocks in FIG. 5, the content list distribution unit 46 will be described. When each signage 1 requests the signage management server 5 to download the content list at the time of starting the device and at a predetermined time (for example, 0:00 at midnight), each signage 1 is installed in the content list distribution unit 46. The content list related to the store is read from the content information F49 and distributed to each signage 1.

Next, the above content list will be described. The content list is information indicating the (display) order, display time, and the like of the contents to be played on the signage 1 of each store S. The content selection information generation unit 37 (mainly the content list distribution unit 46) of the signage management server 5 distributes a content list related to the store in which each signage 1 is installed to each signage 1, thereby distributing each signage 1 (contents). The display control unit 34) controls (instructs) to repeatedly display each content related to its own store for a predetermined time.

For example, when the content list 51 of the A restaurant where the signage 1a is installed has the contents shown in FIG. 6A, the signage 1a installed in the A restaurant has the content list 51 as shown in FIG. According to this, after displaying the menu (contents) of the A restaurant for 10 seconds, the process of displaying the in-store image of the A restaurant for 15 seconds is repeated. Further, when the content list 52 of the B movie theater in which the signage 1b is installed has the content shown in FIG. 6B, the signage 1b installed in the B movie theater follows the content list 52 and is in the B movie theater. The process of displaying the cardholder guide for 30 seconds, the process of displaying the introduction image of movie 1 for 60 seconds, and the process of displaying the introduction image of movie 2 for 30 seconds are repeated.

Returning to the description of each functional block of FIG. The above-mentioned signage customer information receiving unit 41 receives customer information such as customer attributes and detection information of customers (customers who have entered the store) transmitted from the signage group (signage 1a, 1b, etc.). , Signage Stored in customer information F48. Of the functional blocks of FIG. 5, the functions of the congestion / risk determination unit 42, the content selection probability calculation unit 43, the interrupt content determination unit 44, and the interrupt content distribution unit 45 will be described in FIG. 8 below. Will be described in.

Next, with reference to the time chart of FIG. 8, a mechanism for displaying interrupt content (content related to another store) to each signage 1 in the display control system 10 will be described. First, as shown in FIG. 9, the SoC11 (estimation unit 32) of each signage 1 (signages 1a, 1b, etc.) takes the frame image (photographed image) input from the built-in camera 2 of the own machine to obtain the signage 1. The customer in front is detected (S1), and the attributes of the detected customer (gender, age, presence / absence of wearing a mask, presence / absence of disinfection (of fingers with alcohol, etc.), etc.) are estimated (S2). Further, the SoC11 (estimation unit 32) tracks the customers detected in S1 and detects the customers (customers visiting the store) who have entered the store in which the signage 1 is installed among the customers in front of each signage 1. do. The SoC11 and the communication unit 17 of each signage 1 store the customer information including the detection information of the visitor and the attribute information of the customer estimated in S2 above at regular time intervals (for example, every 30 seconds) on the signage management server. Send to 5.

The signage management server 5 (signage customer information receiving unit 41) stores the above-mentioned customer information received from each signage 1 in the signage customer information F48. Then, the congestion / risk determination unit 42 (see FIG. 5) of the signage management server 5 determines the number of customers visiting the store obtained from the above customer information (that is, among the customers detected by the estimation unit 32 of each signage 1). Based on the number of customers who entered the store where each signage 1 is installed), the degree of congestion of the store where each signage 1 is installed is determined, and the number of customers and customers who come to the store obtained from the above customer information. Based on the attribute information (whether or not a mask is worn and whether or not disinfection is performed), the degree of danger of the store where each signage 1 is installed is determined (S3).

A specific example of the congestion / risk determination process of the store in S3 is as follows. First, regarding the congestion degree determination process of the store, the administrator of the display control system 10 determines in advance for each store where the signage 1 is installed (when the signage 1 is installed in each store, when the store is remodeled, etc.). In addition, capacity information such as "If the number of customers in the store is 10 or less, it is vacant, if it exceeds 10 people, it will be a little crowded, and if it exceeds 15 people, it will be crowded." , Store (retain) in the hard disk 22 of the signage management server 5. Then, the congestion / risk determination unit 42 of the signage management server 5 determines the number of customers obtained from the above customer information and the number of customers stored in the hard disk 22 for each store in which the signage 1 is installed. The current number of people in the store is calculated from the average staying time in the store, and the degree of congestion of each store is determined from the current number of people in the store and the capacity information of each store.

As a modification of the above-mentioned congestion degree determination process, the SoC11 (estimation unit 32) of the signage 1 detects the entry and exit of customers from the frame image by the built-in camera 2 of the signage 1 installed at the entrance and exit of the store. Information on the entry and exit of customers is transmitted to the signage management server 5, and the CPU 21 (congestion degree / risk determination unit 42) of the signage management server 5 is based on the information on the entry and exit of customers, and the current number of people in each store. May be calculated to determine the degree of congestion in each store based on the current number of people in the store.

In addition, regarding the risk level determination process of the store, the administrator of the display control system 10 determines in advance for each store where the signage 1 is installed (when the signage 1 is installed in each store, when the store is remodeled, etc.). "It is dangerous if there are 3 or more people in the store who do not wear masks", "It is dangerous if there are 3 or more people in the store who have not disinfected", or "3 or more people wear masks in the store". The risk capacity information such as "It is dangerous if there are people who have not done so or if there are 3 or more people who have not disinfected" and the average staying time in the store of the customers are stored in the hard disk 22 of the signage management server 5. (Hold). Then, the congestion / risk determination unit 42 of the signage management server 5 determines the number of customers who do not wear a mask and / or the number of customers who have not been disinfected, obtained from the above customer information for each store where the signage 1 is installed. And, from the average staying time of the customers in the store stored in the hard disk 22, the number of customers who do not wear the current mask and / or the number of customers who have not been disinfected in the store are calculated, and the current number of customers who do not wear the mask is calculated. The risk level of each store is determined from the number of customers and / or the number of customers who have not been disinfected and the risk level capacity information of each store.

When the congestion / risk determination process of the store in S3 is completed, the content selection probability calculation unit 43 (see FIG. 5) of the signage management server 5 is installed in a certain signage 1a (for example, the A restaurant shown in FIG. 6 or the like). Based on the "selection condition" including the gender and age of the customer in front of the signage and the customer information transmitted from the other signage 1b, the store where the signage 1b is installed (for example, FIG. 6) determined in S3 above. Content selection probability (for selecting the interrupted content to be displayed on the signage 1a) for the above signage 1a (“content selection information” in the claim) based on the degree of congestion and the degree of danger of the B movie theater shown in the above. Is calculated (S4).

The above "selection condition" includes the (estimated) gender and age of the customer in front of the signage 1a included in the customer information transmitted from the above signage 1a, as well as the time point (content selection probability). Conditions such as the time zone (at the time of execution of the calculation process), the weather in the region where the store where the signage 1a is installed, the location of the store where the signage 1a is installed, and the like are included. This selection condition is, for example, "13:00 to 15:00 (time zone), (weather) rainy weather, (store location) A district, ((estimated) gender / age) women in their 30s". And so on. Among the above "selection conditions", information such as the time zone at that time, the weather in the region where the store where the signage 1a is installed, and the location of the store where the signage 1a is installed can be obtained by the signage management server 5 (CPU21) using a timer. , Information stored in the hard disk 22, information on the network, and the like.

The content selection probability calculation unit 43 may be implemented using the trained neural network (content selection NN38) shown in FIG. 4, or may be implemented using a simple table. The content selection probability calculation unit 43 uses the above table or the trained neural network (content selection NN38) to select the most from the "selection conditions" including the gender and age of the customer in front of a certain signage 1. Outputs the matching content selection probability (for a certain signage 1).

When a certain "selection condition" is input, the content selection probability calculation unit 43 obtains the learned content selection NN38 (see FIG. 4) or the table using the store guide (food menu, etc.) of the store Sb. 5%, store information of store Sc is 3%, store information of store Sd is 2%, and so on, as the content displayed by signage 1 of a certain store (for example, store Sa) under a certain "selection condition". It is a probability (selection probability) that the content related to another store (for example, stores Sb, Sc, Sd) around a certain store should be selected.

When the content selection probability calculation unit 43 is implemented using AI (content selection NN38) as shown in FIG. 4, the above "selection condition" is embedded in the input vector to the content selection NN38. Will be done. For example, among the "selection conditions", the time zones of 9:00 to 10:00 and 10:00 to 11:00 and 12:00 to 13:00 are converted into codes 1, 2, and 3, respectively. , Embedded (captured) in the input vector. Further, the weather in the region where the store where the signage 1a is installed is also converted into codes such as 1 (fine), 2 (cloudy), and 3 (rain) and embedded in the input vector. As described above, when the learned content selection NN38 inputs a vector in which a coded selection condition is embedded for a certain signage 1 (for example, the signage 1a), another signage other than the certain signage 1 is installed. The selection probability of the content related to other stores (for example, stores Sb, Sc, etc. around the signage 1a) is output.

The content selection probability calculation unit 43 represents the safety of other stores calculated from the selection probability of contents related to the other stores (for example, stores Sb, Sc, etc.) and the degree of congestion / risk of other stores. By multiplying the factor (hereinafter referred to as "safety factor"), the final selection probability of the content related to other stores is calculated and output. The above "safety factor" is, for example, a value in the range of 0 to 1, and the higher the degree of congestion and risk of the other stores (facility), the closer to 0, and the lower the degree of congestion and risk. Approach 1

For example, the content selection probability calculation unit 43 directly obtained by using the content selection NN38 or the table, the selection probability of the content related to the above other stores is 5% for the store guide of the store Sb and 3% for the store guide of the store Sc. When the store guide of the store Sd is 2%, the final content selection probability output by the content selection probability calculation unit 43 is as follows. That is, the content selection probability in this case is 5% for the store guide of the store Sb x the "safety factor" of the store Sb, the store guide of the store Sc is 3% x the "safety factor" of the store Sc, and the store of the store Sd. The guidance will be 2% x store Sd "safety factor". For example, when the "safety factors" of the stores Sb, Sc, and Sd are 0.6, 0, and 1, respectively, the final content selection probability output by the content selection probability calculation unit 43 is the store of the store Sb. The information is 3% (5% x 0.6), the store information of the store Sc is 0% (3% x 0), and the store information of the store Sd is 2% (2% x 1).

Returning to the description of FIG. When the content selection probability calculation process of S4 is completed, the interrupt content determination unit 44 of the signage management server 5 determines (selects) the interrupt content for a certain signage 1 (for example, signage 1a) (S5).

Next, with reference to FIG. 10, the flow up to the selection of the interrupted content in the content selection information generation unit 37 will be described. As described above, the congestion / risk determination unit 42 of the signage management server 5 determines the number of customers and the attribute information of the customers (whether or not a mask is worn and disinfection) obtained from the customer information received from each signage 1. The degree of congestion and the degree of danger of the store where each signage 1 is installed are determined based on the presence or absence of the signage. Then, the content selection probability calculation unit 43 has a "selection condition" including the gender and age of the customer in front of a certain signage 1a (for example, the signage installed in the A restaurant shown in FIG. 6 and the like), and another signage 1b. The store where the signage 1b is installed (another store around the store where a certain signage 1a is installed, for example, the B movie shown in FIG. 6 or the like, determined by the congestion degree / risk degree determination unit 42 based on the customer information transmitted from The final content is obtained by calculating the selection probability of the above-mentioned interrupt content for signage 1a based on the degree of congestion and the degree of danger of the building) and multiplying the selection probability of this interrupt content and the above-mentioned safety coefficient. Get the selection probability.

The interrupt content determination unit 44 generates a random number R, and determines the interrupt content for the signage 1a from the final content selection probability and the random number R. Specifically, for example, as shown in FIG. 10, the final content selection probability output (calculated) by the content selection probability calculation unit 43 is 35% for the introductory video of movie 1 in the B movie theater and B movie. If the introductory video of movie 2 in the hall is 10%, the interrupt content determination unit 44 generates a random number R from 0 to 99, and the generated random number R is 0 <R. When <34, the introductory video of the movie 1 of the B movie theater is selected, and when 35 <R <44, the introductory video of the movie 2 of the B movie theater is selected as the interrupt content for the signage 1a. The interrupt content distribution unit 45 in FIG. 5 transmits the interrupt content selected by the interrupt content determination unit 44 to the signage 1a. The interrupt content determination unit 44 does not select the interrupt content when the generated random number R is 45 <R <99. Therefore, at this time, the interrupt content distribution unit 45 does not transmit the interrupt content to the signage 1a.

In the description of FIG. 10, the example in which only the content related to the other one store (B movie theater) around the store where the signage 1a is installed is displayed as the interrupt content is shown, but the store where the signage 1a is installed is shown. Compared to the A restaurant where the signage 1a is installed, all of the other stores in the vicinity where the signage is installed (for example, the B movie theater where the signage 1b is installed and the C general store where the signage 1c is installed) are installed. If it is safe and the risk is low, the content related to these a plurality of stores may be displayed as the interrupt content.

Returning to the description of FIG. When the interrupt content determination process of S5 is completed and the interrupt content distribution unit 45 transmits the interrupt content to the signage 1a, the signage 1a is in the interval of displaying the content related to the store in which the own machine is installed. The interrupt content, which is the content related to the store where the signage 1b or the like is installed, is displayed (S6).

Next, the display of the content in the signage 1 will be described with reference to FIG. When the signage 1 receives the interrupt content from the signage management server 5 (interrupt content distribution unit 45), the signage 1 interrupts the reproduction of the content (related to the store where the machine is installed) according to the above content list, and (other signage). Display the interrupt content (related to the store where 1 is installed). For example, when the signage 1 installed in the A restaurant receives the introduction video of the movie 1 of the B movie theater as the interrupt content as shown in FIG. 11 from the signage management server 5, the above signage 1 is the content list. The reproduction of the content related to the A restaurant (for example, the menu (content) of the A restaurant) according to the above is interrupted, and the introduction video (interrupted content) of the movie 1 of the B movie theater is displayed on the touch panel display 14.

In addition to the above-mentioned interrupted content (introduction video of movie 1 of B movie theater), the signage 1 installed in the A restaurant is congested from the signage management server 5 in the store (A restaurant) where the machine is installed. / Or the information on the degree of danger may be received, and the received information indicating the degree of congestion and / or the degree of danger of the store where the own machine is installed (own store) may be superimposed and displayed on the interrupt content. At this time, since it is inappropriate to display the information that the inside of the store is dangerous, the signage 1 has a high degree of congestion of the received own store even if the received degree of danger of the own store is high. Similarly, it is better to display that your store (the store where your machine is installed) is crowded.

As shown in FIG. 11, the content display control unit 34 (see FIG. 4) of the signage 1 reproduces the content (related to the store where the machine is installed) according to the above content list when the display of the interrupted content is completed. To resume.

The display control system 10 has a function of automatically generating a learning data set of the content selection NN 38 shown in FIG. This learning data set automatic generation function will be described with reference to FIGS. 12 and 13. FIG. 12 shows the above-mentioned automatic learning data set generation in the signage 1 and the signage management server 5, and functional blocks necessary for learning using the automatically generated learning data set. The signage 1 has a face recognition information generation unit 62 (billing) in addition to the built-in camera 2 shown in FIG. 4, a video input unit 31, and an estimation unit 32 as a functional block required for the learning data set automatic generation function. It is equipped with the "face recognition information generation means") in the section. The face recognition information generation unit 62 generates face recognition information for customer face recognition from the customer's face image in the frame image (photographed image) taken by the built-in camera 2 of each signage 1. Specifically, the face recognition information generation unit 62 detects with the face detection unit 61 (learned DNN (Deep Natural Networks) model for face detection (hereinafter, abbreviated as “face detection model”)) of the estimation unit 32. Face authentication information for customer face recognition is generated from the cutout image (face image) of the customer's face.

On the other hand, the signage management server 5 has a movement detection unit 64 (“movement detection means” in the claim) and a learning data set generation unit 65 (in the claim) as functional blocks necessary for automatic generation of learning data set and learning. It includes a "learning data set generation means") and a learning unit 67. Further, the signage management server 5 includes a learning data set 66 that collects teacher data (training data) of the content selection NN38. The learning data set 66 is stored in the hard disk 22 (see FIG. 3). The movement detection unit 64 uses the face recognition information generated by the face recognition information generation unit 62 to allow a customer who was in front of a store where one signage 1 is installed to a store where another signage 1 is installed. Detects whether or not it has moved. The learning data set generation unit 65 includes selection conditions including the attributes (gender / age) of the customer who was in front of the above-mentioned signage 1 estimated by the estimation unit 32 from the image taken by the built-in camera 2 of the signage 1. The learning data set of the content selection NN38 based on the result of the movement detection unit 64 detecting whether or not the customer who was in front of the above-mentioned store where the signage 1 is installed has moved to another store where the signage 1 is installed. 66 is automatically generated. Further, the learning unit 67 learns the content selection NN 38 by using the learning data set 66 automatically generated by the learning data set generation unit 65.

FIG. 13 is a time chart showing an automatic generation process of the learning data set 66 of the content selection NN38. A specific example of the automatic generation processing of the above-mentioned learning data set 66 will be described with reference to FIG. In this example, the estimation unit 32 on each signage 1 side includes a gender / age estimation model as a learned DNN model for inference processing, in addition to the face detection model used for the face detection unit 61 described above. Further, the face recognition information generation unit 62 includes a vectorization model. On the other hand, on the signage management server 5, the movement detection unit 64 serves as a learned DNN model for inference processing, and is a DNN for customer identification processing (customer ReID (Person Re-Identification) processing) based on the customer's face vector. It has a model.

In the above configuration, first, the face detection unit 61 (face detection model) of each signage 1 (for example, signage 1a, 1b, 1c) in the display control system 10 is reflected in the frame image input from the built-in camera 2. The face of the customer is detected (S11), and the coordinate position of the detected face (for example, the coordinates representing the center of the face and the coordinate range representing the width and height of the face) is output. Next, the gender / age estimation model included in the estimation unit 32 estimates the customer's attributes (gender and age (age)) using the cutout image of the customer's face detected by the face detection unit 61. (S12). Then, the vectorization model included in the face recognition information generation unit 62 performs vectorization processing on the cutout image (face image) of the customer's face detected by the face detection unit 61 (S13). The SoC 11 and the communication unit 17 of each signage 1 transmit the vector resulting from the above vectorization process to the signage management server 5 as a face vector (“face authentication information” in the claim).

The movement detection unit 64 of the signage management server 5 is installed with a certain signage 1 based on the face vector transmitted from each signage 1 using the DNN model for the customer identification process (customer ReID process) described above. It is detected whether or not the customer who was in front of the store has moved to the store where another signage 1 is installed (S14). Then, the selection condition including the gender and age of the customer who was in front of a certain signage 1 by the learning data set generation unit 65 of the signage management server 5 and the customer who was in front of the above-mentioned store where the signage 1 is installed are different. Based on the result of detection by the movement detection unit 64 whether or not the user has moved to the store where the signage 1 is installed, the learning data set 66 of the content selection NN38 is automatically generated (S15).

By the automatic generation process of S15, the selection condition including the gender and age of the customer who was in front of a certain signage 1 and another signage from this store (for example, restaurant A) at the time of this selection condition. Based on the detection result of whether or not the customer has moved to the installation store (for example, B movie theater), teacher data composed of a combination of the above selection conditions and the detection result can be collected. By learning the content selection NN38 using the learning data set 66 that collects such teacher data, the learned content selection NN38 can be used for a certain signage 1 (for example, a signage installed in the A restaurant). When entering the "selection condition" including the gender and age of the customer in front, the customer who meets this selection condition in the past installs another signage 1 from the front of this store (for example, A restaurant). It is possible to output the content selection probability according to the probability of moving to a store (for example, B movie theater).

As described above, according to the display control system 10 of the present embodiment and the display control program (signage side control program 20 (see FIG. 2) and server side display control program 27 (see FIG. 3)), two or more. Each of the signage 1 installed in the store has a selection condition including the gender and age of the customer in front of the signage 1, and the degree of congestion and the degree of danger of the store where the other signage 1 other than the signage 1 is installed. Based on the above, it is controlled to display the content (interruption content) related to the store where the signage 1 other than the signage 1 is installed. As a result, each of the above signage 1s installs other signage 1s other than the signage 1 without displaying the degree of congestion of the store where the signage 1 is installed (particularly, information that the store is vacant). It is possible to guide customers to a safe and uncrowded store that meets the above selection conditions.

Further, according to the display control system 10 of the present embodiment, in the content display control unit 34 on the signage 1 side and the display control unit 47 on the server side, each signage 1 installed in each store is installed with the signage 1. Content related to a store (other store) where another signage 1 other than the signage 1 is installed, based on the content selection information (content selection probability of another store) while displaying the content related to the new store (own store). Is controlled to be displayed. As a result, each signage 1 installed in each store is based on displaying the content related to the store (own store) in which the signage 1 is installed, and the signage 1 other than the signage 1 is installed. Among the stores (other stores), it is possible to guide customers to a safe store that is not crowded.

Further, according to the display control system 10 of the present embodiment, the selection condition used by the content selection information generation unit 37 (particularly, the content selection probability calculation unit 43) includes the gender and age of the customer in front of the signage 1. In addition, conditions such as the time zone at that time (when the content selection probability calculation process is executed), the weather in the region where the signage 1 installation store is located, and the location of the signage 1 installation store are included. As a result, the signage 1 is not only the gender and age of the customer in front of the signage 1, but also the time zone at that time (when the content selection probability calculation process is executed), and the region where the store where the signage 1 is installed is located. It is possible to display contents related to other stores (stores in which other signage 1 other than the signage 1 is installed) that meet the conditions such as the weather of the signage 1 and the location of the store in which the signage 1 is installed.

Further, according to the display control system 10 of the present embodiment, the selection condition including the gender and age (estimated by the estimation unit 32) of the customer who was in front of the signage 1 and the front of the store where the signage 1 is installed described above. The learning data set 66 of the content selection NN38 is automatically generated based on the result of detecting whether or not the customer who was in the office moved to another store where the signage 1 is installed by using the face recognition information (face vector). I tried to do it. When the learned content selection NN38 inputs the selection condition including the gender and age of the customer in front of a certain signage 1 by learning the content selection NN38 using such a learning data set 66. In the past, it is possible to output the content selection probability according to the probability that a customer who meets this selection condition has moved from the front of the signage 1 installation store to another signage 1 installation store.

Modification example:
The present invention is not limited to the configuration of each of the above embodiments, and various modifications can be made without changing the gist of the invention. Next, a modification of the present invention will be described.

Modification 1: Modification 1:
In the above embodiment, the content selection information generation unit 37 (mainly, the content selection probability calculation unit 43) has the gender of the customer in front of the signage 1 for each signage 1 installed in two or more stores. Content selection information (content selection probability) is generated based on the selection condition including age and the degree of congestion and risk of the store (other store) where the other signage 1 is installed. However, the content selection information generation unit does not necessarily have to generate the above-mentioned content selection information (content selection probability) based on both the degree of congestion and the degree of risk of other stores. That is, for each signage installed in two or more stores, the content selection information generation unit has selection conditions including the gender and age of the customer in front of the signage, and the store in which the other signage is installed (other stores). ) May be used to generate content selection information (content selection probability) based only on the degree of congestion.

Modification 2:
In addition, the content selection information generation unit has selected conditions including the gender and age of the customer in front of the signage for each signage installed in two or more stores, and the store where the other signage is installed (other stores). Content selection information (content selection probability) based on the congestion level and risk level of the store (own store) where the signage is installed, as determined by the congestion level / risk level determination unit in addition to the congestion level and risk level of). ) May be generated. As a result, for each signage installed in two or more stores, if the degree of congestion and risk of the own store where the signage is installed is high, the signage will be used by the other store where the other signage is installed. Of these, the probability of displaying content (interrupted content) related to stores with low congestion and risk that meet the above selection conditions is increased, and conversely, the congestion and risk of own stores where the signage is installed are low. In some cases, the signage can be controlled to reduce the probability of displaying the above-mentioned interrupt content.

In the above case as well, as in the case of the modification 1, the content selection information generation unit does not necessarily have to generate the above content selection information (content selection probability) based on both the degree of congestion and the degree of risk. .. That is, for each signage 1 installed in two or more stores, the content selection information generation unit has a selection condition including the gender and age of the customer in front of the signage 1 and a store in which another signage 1 is installed. Content selection information (content selection probability) may be generated based on the degree of congestion of the store (own store) in which the signage 1 is installed in addition to the degree of congestion of (other stores).

Modification 3:
Further, in the above embodiment, an example is shown in which the attributes included in the selection conditions used by the content selection information generation unit 37 (mainly the content selection probability calculation unit 43) are only the gender and age of the customer. , The attributes included in the above selection conditions are not limited to these, for example, the customer's clothing, whether or not the customer's jewelry (for example, sunglasses, earrings, etc.) is worn, and the customer's group attribute (whether a family of three or more). , A type of group such as a couple, friends of the same age, or one person).

Modification 4:
Further, in the above embodiment, the congestion degree / risk degree determination unit 42 of the signage management server 5 obtains the number of customers and the attribute information of the customers (whether or not a mask is worn, etc.) obtained from the customer information received from each signage 1. And whether or not disinfection was carried out), the degree of congestion and the degree of danger of the stores where each signage 1 was installed were determined. However, the present invention is not limited to this configuration, and the congestion degree / risk degree determination unit is provided on the signage side, and the congestion degree / risk degree determination unit on the signage 1 side is the number of customers estimated by the estimation unit. Based on the attribute information of the customer (whether or not the mask is worn and whether or not disinfection is performed), the degree of congestion and the degree of danger of the store where each signage 1 is installed are determined, and the information on the degree of congestion and the degree of danger of this store is obtained. , May be sent to the signage management server 5.

Modification 5:
Further, in the above embodiment, an example in which the content selection information in the present invention is the content selection probability (content selection probability) of the store (other store) in which the signage 1 other than the signage 1 is installed. As shown, the content selection information is not limited to this, and may be information that can be controlled to display the content of another signage-installed store (other store) on the signage based on this information. It's fine. For example, as for the content selection information, the content related to the store where the signage is installed (for example, restaurant A) is 11 seconds, and the content related to the store where the signage is installed (for example, according to the content list) is the first content (for example, the B movie theater). Content display time (combination) information such as 7 seconds for the introductory video of movie 1 and 2 seconds for the second content related to other signage stores (for example, the introductory video of movie 2 in B movie theater). There may be.

Modification 6:
Further, in the above embodiment, in the automatic generation process of the learning data set 66 of the content selection NN38, the movement detection unit 64 is a store in which a certain signage 1 is installed based on a face vector obtained by vectorizing a customer's face image. The customer who was in front of the company was made to detect whether or not the customer moved to the store where another signage 1 was installed. That is, the movement detection unit 64 uses a face vector obtained by vectorizing a customer's face image as authentication information used for customer identification. However, the present invention is not limited to this, and the movement detection unit may use a customer vector obtained by vectorizing an image of the entire customer's body as authentication information used for identifying the customer, or may use a customer's face or body features (for example,). Some feature quantity representing the contour of the face, the texture of the face (stains, wrinkles, sagging), the distance between the eyes, etc.) may be used.

1,1a, 1b, 1c signage (display device)
2 Built-in camera (shooting means)
20 Signage side display control program (display control program)
27 Server-side display control program (display control program)
32 Estimating unit (estimating means)
34 Content display control unit (display control means)
37 Content selection information generation unit (content selection information generation means)
38 Content selection NN (neural network)
42 Congestion degree / risk degree determination unit (congestion degree determination means)
47 Server-side display control unit (display control means)
62 Face recognition information generation unit (face recognition information generation means)
64 Movement detection unit (movement detection means)
65 Learning data set generation unit (learning data set generation means)
66 Learning dataset S, Sa, Sb, Sc, Sd Store (facility)

Claims (7)

Display devices with photographing means installed in each of at least two or more facilities, and
An estimation means that detects a person in front of each of the display devices from an image captured by the photographing means and estimates the attributes of the detected person.
Congestion degree determination means for determining the degree of congestion of the facility in which each of the display devices is installed, based on the number of people who have entered the facility in which each of the display devices is installed among the persons detected by the estimation means. When,
For each of the display devices installed in the two or more facilities, the selection condition including the attribute of the person in front of the display device estimated by the estimation means from the image taken by the photographing means of the display device, and the above. Based on the congestion degree of the facility where the display device other than the display device is installed, which is determined by the congestion degree determination means, the content selection information which is the information for selecting the content to be displayed on the display device is generated. Content selection information generation means and
Display control means for controlling each of the display devices installed in the two or more facilities to display contents related to the facility in which other display devices other than the display device are installed, based on the content selection information. Display control system with and. The display control means other than the display device based on the content selection information while each of the display devices installed in the two or more facilities displays the content related to the facility in which the display device is installed. The display control system according to claim 1, wherein another display device is controlled to display contents related to a facility in which the device is installed. In the content selection information generation means, for each of the display devices installed in the two or more facilities, a selection condition including an attribute of a person in front of the display device and the other display device are installed. 1. The display control system according to 2. The selection condition used by the content selection information generation means includes the attribute of the person in front of the display device, the time zone at that time, and the weather in the region where the facility in which the display device is installed is located. The display control system according to any one of claims 1 to 3, wherein at least one condition of the position of the facility where the display device is installed is included. The content selection information generation means is characterized by including a neural network that outputs the selection probability of content related to a facility in which a display device other than the display device is installed when the selection condition is vectorized and input. The display control system according to any one of claims 1 to 4. For each of the display devices installed in the two or more facilities, face recognition information generation for generating face recognition information for the person's face recognition from the person's face image in the image taken by the shooting means of the display device. Means and
Using the face authentication information generated by the face authentication information generation means, it is detected whether or not a person who was in front of a facility where a certain display device is installed has moved to a facility where another display device is installed. Movement detection means and
The selection condition including the attribute of the person who was in front of the display device and the person who was in front of the facility where the display device was installed, which was estimated by the estimation means from the image taken by the photographing means of the display device. , A training data set generation means that automatically generates a training data set of the neural network model based on the result of detecting whether or not the other display device has moved to the facility installed by the movement detection means. The display control system according to claim 5, further comprising: A display control program that controls switching of content to be displayed on a display device having a photographing means, which is installed in each of at least two or more facilities.
Computer,
An estimation means that detects a person in front of each of the display devices from an image captured by the photographing means and estimates the attributes of the detected person.
Congestion degree determination means for determining the degree of congestion of the facility in which each of the display devices is installed, based on the number of people who have entered the facility in which each of the display devices is installed among the persons detected by the estimation means. When,
For each of the display devices installed in the two or more facilities, the selection condition including the attribute of the person in front of the display device estimated by the estimation means from the image taken by the photographing means of the display device, and the above. Based on the congestion degree of the facility where the display device other than the display device is installed, which is determined by the congestion degree determination means, the content selection information which is the information for selecting the content to be displayed on the display device is generated. Content selection information generation means and
Display control means for controlling each of the display devices installed in the two or more facilities to display the content related to the facility in which the display device other than the display device is installed, based on the content selection information. Display control program to function as.

Images