WO2022091297A1

WO2022091297A1 - Spectator monitoring device, spectator monitoring system, spectator monitoring method, and non-transitory computer-readable medium

Info

Publication number: WO2022091297A1
Application number: PCT/JP2020/040633
Authority: WO
Inventors: 怜平田; 統山下; 健太藍野
Original assignee: 日本電気株式会社
Priority date: 2020-10-29
Filing date: 2020-10-29
Publication date: 2022-05-05
Also published as: JPWO2022091297A1

Abstract

A spectator monitoring device (1) according to the present disclosure is provided with: an acquisition unit (11) that acquires a first captured image of a spectator region where a plurality of spectators in a predetermined facility are included; a detection unit (12) that detects, from the first captured image, a non-wearer who is a spectator not wearing a mask; an identification unit (13) for identifying a partial region of the spectator region where the non-wearer is included; and a notification unit (14) that notifies an information terminal of operation personnel of the facility, of warning information about the identified partial region.

Description

Audience monitoring equipment, spectator monitoring systems, spectator monitoring methods and non-temporary computer-readable media

The present invention relates to spectator monitoring devices, spectator monitoring systems, spectator monitoring methods and non-temporary computer-readable media, in particular spectator monitoring devices, spectator monitoring systems, spectator monitoring methods and non-temporary spectator wearing masks. Regarding temporary computer-readable media.

In recent years, from the viewpoint of preventing the spread of infectious diseases, in principle, masks are required to be worn in facilities where a large number of spectators gather. Patent Document 1 discloses a technique of detecting a mask from a face image and urging the mask to be removed when the mask is worn.

International Publication No. 2017/212967

In the facility where a large number of spectators gathered, it was difficult for the facility staff (operation staff) to find the spectators who had their masks removed and to speak to them.

This disclosure is made in order to solve such a problem, and is an spectator monitoring device and spectator monitoring for the facility operating personnel to easily identify the spectator who is removing the mask from a large number of spectators. It is intended to provide systems, audience monitoring methods and non-temporary computer readable media.

The audience monitoring device according to the first aspect of the present disclosure is
An acquisition means for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
To prepare for.

The audience monitoring system according to the second aspect of the present disclosure is
An imaging device that photographs an audience area including multiple spectators in a predetermined facility,
Audience monitoring device connected to the photographing device and
Equipped with
The spectator monitoring device is
An acquisition means for acquiring a first captured image captured by the imaging device, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
To prepare for.

The audience monitoring method according to the third aspect of the present disclosure is
The computer
Acquire the first photographed image in which the spectator area including a plurality of spectators in a predetermined facility is photographed.
A non-wearer who is a spectator who is not wearing a mask is detected from the first captured image, and the mask is not worn.
A partial area of the audience area including the non-wearer is identified.
The information terminal of the operation staff of the facility is notified of the warning information about the specified partial area.

The non-temporary computer-readable medium containing the audience monitoring program according to the fourth aspect of the present disclosure is
An acquisition process for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection process for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image, and
Specific processing for specifying a partial area including the non-wearer in the audience area, and
Notification processing for notifying the information terminal of the facility operation personnel of warning information about the specified partial area, and
Let the computer run.

The present disclosure provides spectator monitoring devices, spectator monitoring systems, spectator monitoring methods and non-temporary computer-readable media for facility operators to easily identify spectators who are unmasked from a large number of spectators. be able to.

It is a block diagram which shows the structure of the spectator monitoring apparatus which concerns on this Embodiment 1. It is a flowchart which shows the flow of the audience monitoring method which concerns on this Embodiment 1. It is a block diagram which shows the whole structure of the audience monitoring system which concerns on this Embodiment 2. It is a figure which shows the example of the audience area and the camera installation position of the stadium which concerns on Embodiment 2. FIG. It is a block diagram which shows the structure of the authentication apparatus which concerns on this Embodiment 2. It is a flowchart which shows the flow of the face information registration processing which concerns on this Embodiment 2. It is a flowchart which shows the flow of the face recognition process by the authentication device which concerns on this Embodiment 2. It is a block diagram which shows the structure of the spectator monitoring apparatus which concerns on this Embodiment 2. It is a flowchart which shows the flow of the audience monitoring processing which concerns on this Embodiment 2. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 2. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 2. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 2. It is a block diagram which shows the structure of the spectator monitoring apparatus which concerns on this Embodiment 3. It is a flowchart which shows the flow of the audience monitoring processing which concerns on this Embodiment 3. It is a block diagram which shows the structure of the spectator monitoring apparatus which concerns on this Embodiment 4. It is a flowchart which shows the flow of the audience monitoring processing which concerns on this Embodiment 4. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 4. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 4. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 4. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 4. It is a flowchart which shows the flow of the audience monitoring processing which concerns on this Embodiment 5. It is a figure which shows the display example of the warning information of the non-wearer of the mask which concerns on this Embodiment 5. It is a flowchart which shows the flow of the audience monitoring processing which concerns on this Embodiment 6. It is a block diagram which shows the structure of the spectator monitoring apparatus which concerns on this Embodiment 7.

Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations are omitted as necessary for the sake of clarity of explanation.

<Embodiment 1>
FIG. 1 is a block diagram showing a configuration of an spectator monitoring device 1 according to the first embodiment. The spectator monitoring device 1 is an information processing device for monitoring the wearing of masks of a plurality of spectators in a predetermined facility. Here, the spectator monitoring device 1 is connected to a network (not shown). The network may be wired or wireless. Further, the photographing device and the information terminal are connected to the network. The photographing device photographs an spectator area including a plurality of spectators in a predetermined facility. The photographing device transmits the photographed image of the spectator area to the spectator monitoring device 1 via the network. The information terminal is an information processing device operated by facility operation personnel.

The spectator monitoring device 1 includes an acquisition unit 11, a detection unit 12, a specific unit 13, and a notification unit 14. The acquisition unit 11 acquires a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured. The detection unit 12 detects a non-wearer who is a spectator who is not wearing a mask from the first captured image. For example, the detection unit 12 determines whether or not a mask is present in the face region of a person in a captured image by image recognition. The specific unit 13 specifies a partial area of the audience area including the non-wearer of the mask. The notification unit 14 notifies the information terminal of the facility operation personnel of the warning information about the specified partial area. Here, the warning information may be display information for specifying (identifying) a partial area in the captured image, or may be voice information that can specify the partial area.

FIG. 2 is a flowchart showing the flow of the audience monitoring method according to the first embodiment. First, the acquisition unit 11 acquires a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured (S11). Next, the detection unit 12 detects a non-wearer who is a spectator who is not wearing a mask from the first captured image (S12). If no non-wearer is detected, the process returns to step S11 after a predetermined time.

When a non-wearer is detected in step S12, the specific unit 13 specifies a partial area including the non-wearer in the audience area (S13). Then, the notification unit 14 notifies the information terminal of the facility operation personnel of the warning information about the specified partial area (S14). That is, the notification unit 14 transmits warning information to the information terminal via the network. As a result, the information terminal receives the warning information from the spectator monitoring device 1 via the network, displays the warning information on the screen, or outputs the voice corresponding to the warning information.

As described above, according to the present embodiment, the facility operating personnel can recognize in real time that there is an spectator who is not wearing a mask in the spectator area via the information terminal. In particular, the operation personnel can easily identify the partial area of the spectator area where the non-wearer of the mask exists from the warning information. Therefore, it is possible to support the operation staff to call out to the spectators who have removed the mask to encourage them to wear the mask.

The spectator monitoring device 1 includes a processor, a memory, and a storage device as a configuration (not shown). Further, the storage device stores a computer program in which the processing of the spectator monitoring method according to the present embodiment is implemented. Then, the processor reads the computer program from the storage device into the memory and executes the computer program. As a result, the processor realizes the functions of the acquisition unit 11, the detection unit 12, the specific unit 13, and the notification unit 14.

Alternatively, the acquisition unit 11, the detection unit 12, the specific unit 13, and the notification unit 14 may each be realized by dedicated hardware. Further, a part or all of each component of each device may be realized by a general-purpose or dedicated circuitry, a processor, or a combination thereof. These may be composed of a single chip or may be composed of a plurality of chips connected via a bus. A part or all of each component of each device may be realized by the combination of the circuit or the like and the program described above. Further, as a processor, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (field-programmable gate array), a quantum processor (quantum computer control chip), or the like can be used.

Further, when a part or all of each component of the spectator monitoring device 1 is realized by a plurality of information processing devices, circuits, etc., the plurality of information processing devices, circuits, etc. may be centrally arranged. It may be distributed. For example, the information processing device, the circuit, and the like may be realized as a form in which each is connected via a communication network, such as a client-server system and a cloud computing system. Further, the function of the spectator monitoring device 1 may be provided in the SAAS (Software as a Service) format.

<Embodiment 2>
The second embodiment is a specific example of the first embodiment described above. FIG. 3 is a block diagram showing the overall configuration of the spectator monitoring system 1000 according to the second embodiment. The spectator monitoring system 1000 is an information system for monitoring the wearing of masks of a plurality of spectators existing in a plurality of spectator areas 101 to 10n (n is a natural number of 2 or more) in the stadium 100. The stadium 100 is an example of a predetermined facility. Further, it is assumed that the audience area 101 is divided into blocks 1011 to 101i (i is a natural number of 1 or more), and each block includes 1 or more seats. The same applies to the audience areas 102 (not shown) to 10n. For example, it is assumed that the audience area 10n is divided into a plurality of blocks 10n1 to 10nj (j is a natural number of 2 or more), and each block includes 1 or more seats. The audience area may be a standing seat without a seat.

The spectator monitoring system 1000 includes a camera 110, a staff terminal 120, a management terminal 130, an authentication device 200, and a spectator monitoring device 300. Each of the camera 110, the staff terminal 120, the management terminal 130, the authentication device 200, and the spectator monitoring device 300 is connected via the network N. Here, the network N is a wired or wireless communication line.

The stadium 100 includes a camera 110, a staff terminal 120, and a management terminal 130. The camera 110 is a photographing device that photographs each of the audience areas 101 to 10n in units of the audience area. Specifically, the camera 110 moves in the shooting direction (camera direction) at predetermined intervals, shoots 10n from the spectator area 101 in order for a predetermined time, shoots the spectator area 10n, and then again shoots the spectator area 101. Move in the shooting direction of, and repeat this. Further, the camera 110 transmits a photographed image of the audience area to the audience monitoring device 300 via the network N. At this time, the camera 110 transmits the captured image together with the imaging direction (direction of the camera or installation position of the camera) to the spectator monitoring device 300. The camera 110 may capture at least one of 10n from the audience area 101. For example, when the camera 110 shoots the spectator area 101, the other spectator area 10n and the like may be photographed by another camera. Then, the other cameras also transmit the photographed image and the photographed direction to the spectator monitoring device 300.

FIG. 4 is a diagram showing an example of the audience area and the camera installation position of the stadium 100 according to the second embodiment. For example, the camera 110a shows an example installed on the uppermost stage of the stand of the stadium 100. That is, the camera 110a is installed on the stand on the opposite side of the audience area 101 to 103 (stand) with the ground in between. In FIG. 4, the camera 110a shows that the direction of the audience area 101 is the shooting direction. Then, as described above, the camera 110a moves the shooting direction in the order of the

spectator areas

101, 102, 103 at regular intervals, shoots the spectator area in each shooting direction, and each time, the spectator monitoring device via the network N. The photographed image is transmitted to 300. Further, the camera 110b is another example of the camera installation position. The camera 110b is an example installed in front of the stand on the side of the audience area 101 to 103. Also in this case, the camera 110b may move the shooting direction in the order of the

spectator areas

101, 102, 103 at regular intervals, and may shoot the spectator area in each shooting direction. Alternatively, individual cameras may be installed in front of each of the audience areas 101 to 103.

Return to Fig. 3 and continue the explanation. The management terminal 130 is an information processing device operated by the administrator U2 in monitoring the audience. The manager U2 is an example of the operation staff of the stadium 100. The management terminal 130 is an example of the above-mentioned information terminal. The management terminal 130 receives the warning information described later from the spectator monitoring device 300 via the network N, and displays the warning information on the screen. Then, the manager U2 conveys the position of the non-wearer of the mask to the staff terminal 120 via the management terminal 130 based on the warning information displayed on the management terminal 130. For example, the management terminal 130 may transmit the voice of the non-wearer's seat position emitted by the manager U2 to the staff terminal 120 via the network N. Alternatively, the management terminal 130 may transfer the warning information to the staff terminal 120 selected by the administrator U2. As a result, the stadium staff U1 can confirm the warning information on the screen of the staff terminal 120, so that it becomes easier to grasp the non-wearer of the mask.

The staff terminal 120 is an information processing device operated by the stadium staff U1 which is an example of the operation personnel of the stadium 100. The staff terminal 120 receives and outputs voice information from the management terminal 130 via the network N. As a result, the stadium staff U1 can grasp the specific seat position of the non-wearer of the mask via the voice information. Therefore, the stadium staff U1 can go near the person who does not wear the mask and call for wearing the mask. The staff terminal 120 may be an example of the above-mentioned information terminal. In that case, the staff terminal 120 receives the warning information described later from the spectator monitoring device 300 via the network N instead of the management terminal 130 or together with the management terminal 130, and displays the warning information on the screen. Even in this case, the stadium staff U1 can grasp the specific seat position of the person who does not wear the mask.

The management terminal 130 and the staff terminal 120 are, for example, a mobile phone terminal, a smartphone, a tablet terminal, a PC (Personal Computer) on which or connected to a camera, and the like. When the warning information is voice information, the management terminal 130 and the staff terminal 120 output the received voice information.

Here, an example of pre-registration for realizing spectator monitoring in the spectator monitoring system 1000 will be described. The spectators make reservations for seats at the event of the specific day at the stadium 100 in advance. For example, the spectator purchases an event ticket on an event reservation site using an information processing device such as a PC or a smartphone, specifies a seat position at that time, and inputs personal information or the like as necessary. Then, the spectator acquires identification information (for example, a two-dimensional code) indicating the seat position and personal information reserved at the event. For example, the spectator's smartphone receives the two-dimensional code. Alternatively, the audience obtains the two-dimensional code by e-mail or the like and prints it.

Further, the stadium 100 shall be provided with an entrance terminal (not shown) for controlling the entrance of spectators at the entrance gate. The entrance terminal is an information terminal provided with a camera or the like, and is connected to the network N. Therefore, the audience monitoring system 1000 may include an entrance terminal.

The spectator arrives at the stadium 100 on the day of the event and presents the two-dimensional code to the entrance terminal. For example, the audience displays the two-dimensional code on the smartphone and holds the two-dimensional code over the camera of the entrance terminal. Alternatively, the spectator holds the paper on which the two-dimensional code is printed over the camera of the entrance terminal. The entrance terminal analyzes the two-dimensional code taken by the camera, extracts the reserved seat position and personal information, and makes a registration request including the seat position and personal information via the network N to the audience monitoring device 300. Send to. If there is no problem with the seat position, personal information, etc. included in the registration request, the spectator monitoring device 300 registers the seat position, personal information, etc. in association with each other, and returns a face information registration request to the entrance terminal. The entrance terminal displays a message indicating a registration request on the screen, photographs an area including the face of the spectator, and transmits a face information registration request including the captured image to the authentication device 200 or the spectator monitoring device 300 via the network N. .. As will be described later, the authentication device 200 performs face information registration processing in response to a face information registration request, and returns the registration result to the entrance terminal. If successfully registered, spectators can enter the stadium 100 through the entrance gate. Further, the authentication device 200 notifies the spectator monitoring device 300 of the user ID (audience ID) issued at the time of registration.

The spectator may also register his / her face image when reserving a seat. For example, the spectator information processing device may transmit a face information registration request including a captured image to the authentication device 200 or the spectator monitoring device 300 via the network N via the event reservation site. Alternatively, even if the spectator has not reserved a seat in advance, the seat will be confirmed by the admission procedure at the entrance terminal on the day of the event, and at that time, the registration request including the seat position and personal information and the photographed image will be taken. The face information registration request including the above may be transmitted to the audience monitoring device 300 via the network N.

The authentication device 200 is an information processing device that stores facial feature information of a plurality of people (audiences). Further, the authentication device 200 collates the face image or face feature information included in the request with the face feature information of each user in response to the face recognition request received from the outside, and requests the collation result (authentication result). Reply to the original.

FIG. 5 is a block diagram showing the configuration of the authentication device 200 according to the second embodiment. The authentication device 200 includes a face information DB (DataBase) 210, a face detection unit 220, a feature point extraction unit 230, a registration unit 240, and an authentication unit 250. The face information DB 210 stores the user ID 211 and the face feature information 212 of the user ID in association with each other. The face feature information 212 is a set of feature points extracted from the face image. The authentication device 200 may delete the face feature information 212 in the face feature DB 210 in response to a request from the registered user of the face feature information 212. Alternatively, the authentication device 200 may be deleted after a certain period of time has elapsed from the registration of the face feature information 212.

The face detection unit 220 detects the face area included in the registered image for registering face information and outputs it to the feature point extraction unit 230. The feature point extraction unit 230 extracts feature points from the face region detected by the face detection unit 220, and outputs face feature information to the registration unit 240. Further, the feature point extraction unit 230 extracts feature points included in the face image received from the information processing device, the entrance terminal, or the spectator monitoring device 300 of the spectator, and outputs the face feature information to the authentication unit 250.

The registration unit 240 newly issues the user ID 211 when registering the facial feature information. The registration unit 240 registers the issued user ID 211 and the face feature information 212 extracted from the registered image in the face information DB 210 in association with each other. The authentication unit 250 performs face authentication using the face feature information 212. Specifically, the authentication unit 250 collates the face feature information extracted from the face image with the face feature information 212 in the face information DB 210. If the verification is successful, the authentication unit 250 identifies the user ID 211 associated with the collated face feature information 212. The authentication unit 250 returns to the spectator monitoring device 300 as a face authentication result whether or not the face feature information matches. The presence or absence of matching of facial feature information corresponds to the success or failure of authentication. In addition, the fact that the facial feature information matches (with matching) means that the degree of matching is equal to or higher than a predetermined value. Further, the face authentication result shall include the specified user ID when the face authentication is successful.

FIG. 6 is a flowchart showing the flow of the face information registration process according to the second embodiment. Here, the information registration terminal (not shown) photographs the body including the face of the spectator (user), and transmits the face information registration request including the captured image (registered image) to the authentication device 200 via the network N. The information registration terminal is, for example, an information processing device such as a personal computer, a smartphone, or a tablet terminal. For example, the information registration terminal may be an entrance terminal installed at the entrance of the stadium 100. Alternatively, the information registration terminal may be an spectator monitoring device 300 that has received a face information registration request from the entrance terminal.

First, the authentication device 200 acquires the registered image included in the face information registration request (S21). For example, the authentication device 200 receives the face information registration request from the information registration terminal via the network N. Next, the face detection unit 220 detects the face region included in the registered image (S22). Next, the feature point extraction unit 230 extracts feature points from the face region detected in step S22, and outputs face feature information to the registration unit 240 (S23). Finally, the registration unit 240 issues the user ID 211, associates the user ID 211 with the face feature information 212, and registers the user ID 211 in the face information DB 210 (S24). The authentication device 200 may receive the face feature information 212 from the information registration terminal and register it in the face information DB 210 in association with the user ID 211. Further, the registration unit 240 may notify the spectator monitoring device 300 of the registered (issued) user ID.

FIG. 7 is a flowchart showing the flow of face recognition processing by the authentication device 200 according to the second embodiment. First, the feature point extraction unit 230 acquires a face image for authentication included in the face authentication request (S31). For example, the authentication device 200 receives a face authentication request from the audience monitoring device 300 via the network N, and extracts face feature information from the face image included in the face authentication request as in steps S21 to S23. Alternatively, the authentication device 200 may receive facial feature information from the spectator monitoring device 300. Next, the authentication unit 250 collates the acquired face feature information with the face feature information 212 of the face information DB 210 (S32). When the face feature information matches, that is, when the degree of matching of the face feature information is equal to or higher than a predetermined value (Yes in S33), the authentication unit 250 identifies the user ID 211 of the user whose face feature information matches (S34). , The user ID 211 identified as having succeeded in face authentication is returned to the audience monitoring device 300 (S35). When there is no matching face feature information (No in S33), the authentication unit 250 returns to the audience monitoring device 300 that the face authentication has failed (S36).

Return to Fig. 3 and continue the explanation. The spectator monitoring device 300 detects a non-wearer of the mask from the image captured by the camera 110 in the spectator area, identifies a partial area including the non-wearer, and notifies the management terminal 130 or the staff terminal 120 as warning information. It is an information processing device. The spectator monitoring device 300 may be redundant to a plurality of servers, or each functional block may be realized by a plurality of computers.

Next, the audience monitoring device 300 will be described in detail. FIG. 8 is a block diagram showing the configuration of the spectator monitoring device 300 according to the second embodiment. The spectator monitoring device 300 includes a storage unit 310, a memory 320, an IF (InterFace) unit 330, and a control unit 340. The storage unit 310 is an example of a storage device such as a hard disk and a flash memory. The storage unit 310 stores the program 311, the spectator ID 312, the seat position 313, the shooting direction 314, and the spectator area 315. The program 311 is a computer program in which the processing of the audience monitoring method according to the second embodiment is implemented.

The spectator ID 312 is the identification information of the spectators in the stadium 100. The spectator ID 312 may be spectator identification information at an event on a specific day. The seat position 313 is information for specifying the position of each seat within 10n from the spectator area 101. The spectator ID 312 and the seat position 313 are associated with each other. Further, the audience ID 312 corresponds to the user ID 211 in the face information DB 210 of the authentication device 200. Therefore, it can be said that the storage unit 310 stores the spectator ID 312 corresponding to the face feature information 212 in the face information DB 210 and the seat position 313 in association with each other.

The shooting direction 314 is information indicating the direction of the camera (or the installation position of the camera) in the camera 110. The spectator area 315 is information that defines the block and the seat position for each of the spectator areas 101 to 10n. Specifically, one spectator area 315 includes a plurality of blocks 3151 to 315i. And each block includes a plurality of seat positions. For example, block 3151 includes seat positions 31511 to 3151k (k is a natural number of 2 or more). Further, the shooting direction 314 and the audience area 315 are associated with each other.

The memory 320 is a volatile storage device such as a RAM (RandomAccessMemory), and is a storage area for temporarily holding information when the control unit 340 operates. The IF unit 330 is a communication interface with the network N.

The control unit 340 is a processor, that is, a control device that controls each configuration of the spectator monitoring device 300. The control unit 340 reads the program 311 from the storage unit 310 into the memory 320, and executes the program 311. As a result, the control unit 340 realizes the functions of the registration unit 341, the acquisition unit 342, the detection unit 343, the authentication control unit 344, the specific unit 345, and the notification unit 346.

The registration unit 341 registers the identification information (audience ID) corresponding to the facial feature information of each of the plurality of spectators and the seat position in the storage unit 310 in association with each other. For example, the registration unit 341 receives a registration request including the seat position, personal information, and the like from the entrance terminal via the network N, as in the above-mentioned pre-registration example. When the registration unit 341 confirms that the received seat position, personal information, etc. are not different from the reservation contents on the reservation site or the like, the registration unit 341 registers the seat position, personal information, etc. in the storage unit 310 in association with each other. Then, the registration unit 341 returns a request for registration of face information to the entrance terminal. When the registration unit 341 receives the face information registration request including the face image of the spectator from the entrance terminal, the registration unit 341 transmits the face information registration request to the authentication device 200 via the network N. Then, the registration unit 341 receives the user ID issued at the time of registration of the face information from the authentication device 200 via the network N, and stores the user ID as the spectator ID 312 in association with the seat position 313 registered in advance. It is stored in the unit 310. It should be noted that the registration unit 341 shall also process the registration request and the face information registration request from the information processing apparatus of the spectator in the same manner.

The acquisition unit 342 is an example of the acquisition unit 11 described above. The acquisition unit 342 acquires a captured image captured by any one of the audience area 101 and the like from the camera 110 via the network N. The acquisition unit 342 acquires the shooting direction together with the shot image from the camera 110.

The detection unit 343 is an example of the detection unit 12 described above. The detection unit 343 detects a non-wearer who is a spectator who is not wearing a mask from the captured image acquired by the acquisition unit 342. The detection unit 343 may analyze the captured image using a known technique to determine whether or not a mask is present in the face region of each person in the captured image. For example, the detection unit 343 may detect a non-wearer of a mask by using an AI (Artificial Intelligence) model that has learned a face image of a person wearing a mask.

The authentication control unit 344 controls face recognition with reference to the storage unit 310 and the authentication device 200 (these are collectively an example of the storage means) based on the face area of the non-wearer in the captured image. That is, the authentication control unit 344 causes the authentication device 200 to perform face recognition for the face feature information of the non-wearer. For example, the authentication control unit 344 extracts the face region corresponding to the non-wearer of the mask detected by the detection unit 343 from the captured image. Then, the authentication control unit 344 sends a face authentication request including an image of the extracted face area to the authentication device 200 via the network N, and the authentication device 200 outputs the face authentication result of the user (non-wearer). Receive. Alternatively, the authentication control unit 344 may extract the face feature information from the face region and include the face feature information in the face authentication request.

The specific unit 345 is an example of the above-mentioned specific unit 13. The specific unit 345 specifies a partial area of the audience area including the non-wearer of the mask. Here, the specific unit 345 specifies the seat position of the non-wearer as a partial area. That is, the specifying unit 345 specifies the user ID included in the received face authentication result as the spectator ID 312, and specifies the seat position 313 associated with the spectator ID 312 as a partial area. If the received face authentication result indicates a face authentication failure, the specific unit 345 cannot specify the audience ID. In that case, the specifying unit 345 may specify the seat position based on the shooting direction of the camera 110 that shot the shot image and the detection position of the non-wearer in the shot image. For example, the specifying unit 345 specifies the audience area 315 associated with the acquired shooting direction 314. Then, the specific unit 345 specifies the detection position in the captured image of the non-wearer detected by the acquisition unit 342. Then, the specifying unit 345 specifies the block corresponding to the detection position and the seat position as a partial area from the specified spectator area 315. The specific unit 345 may specify the block as a partial area without specifying the seat position. Alternatively, the specific unit 345 may specify the audience area itself, which is the object to be photographed of the photographed image, as a partial area.

The notification unit 346 is an example of the notification unit 14 described above. The notification unit 346 transmits warning information to the management terminal 130 or the staff terminal 120 via the network N. Here, the warning information includes the seat position specified by the specific unit 345. The warning information is information that can identify the seat position of the non-wearer in the photographed image. For example, the warning information may be a border surrounding the vicinity of the non-wearer's face. Therefore, the notification unit 346 may transmit the captured image on the management terminal 130 or the staff terminal 120 so as to display the warning information in the captured image. As a result, the manager U2 or the stadium staff U1 can easily identify the position of the non-wearer and can easily speak to the non-wearer. Alternatively, the warning information may be voice information of the seat position (seat number, etc.) of the non-wearer. Further, the warning information may include at least the block specified by the specific unit 345.

Note that the notification unit 346 may further notify the warning information to the predetermined output device in the stadium 100. The output device may be, for example, a speaker, an aurora vision (liquid crystal display), a digital signage, or the like. At this time, the notification unit 346 may use a speaker or digital signage near the block specified by the specific unit 345 as an output device. For example, a directional speaker may be used as the output device. In that case, the notification unit 346 may output warning information (voice) from the directional speaker toward the specified block. As a result, the audience can be notified directly. In addition, the warning information at this time may include voice or message information such as "Please cooperate in wearing a mask".

In the second embodiment, the detected seat position of the non-wearer does not necessarily have to be specified by face recognition. In that case, the spectator monitoring device 300 does not need the associated information of the spectator ID 312 and the seat position 313 and the authentication control unit 344. Then, as described above, the specifying unit 345 may specify the seat position as a partial region based on the shooting direction of the camera 110 that shot the shot image and the detection position of the non-wearer in the shot image. Alternatively, as described above, the specifying unit 345 may specify the block as a partial region based on the shooting direction of the camera 110 that shot the shot image and the detection position of the non-wearer in the shot image. In this case, even if the seat position cannot be specified, the notification unit 346 transmits an image of a partial area such as a block unit and an image showing a person who does not wear the mask with a red frame or the like as warning information to the staff terminal 120 or the like. It is good to do it.

FIG. 9 is a flowchart showing the flow of the audience monitoring process according to the second embodiment. Here, for example, it is assumed that the camera 110 shoots with respect to the shooting direction of the spectator area 101, and transmits the shot image and the shooting direction to the spectator monitoring device 300 via the network N. In response to this, the acquisition unit 342 acquires a captured image and a capture direction from the camera 110 via the network N (S41).

Next, the detection unit 343 determines whether or not the non-wearer of the mask is detected from the captured image acquired in step S41 (S42). For example, the detection unit 343 analyzes the captured image and identifies one or more spectator face regions included in the captured image. Then, the detection unit 343 determines whether or not the mask is worn for each of the specified face regions. If no non-wearer is detected, the detection unit 343 sends an instruction to the camera 110 via the network N to move the shooting direction of the camera 110 to the next spectator area (S43). ..

When one or more non-wearers are detected in step S42, the authentication control unit 344 and the specific unit 345 perform a partial area identification process for each non-wearer (S44). Hereinafter, it is assumed that one non-wearer is detected, but when two or more non-wearers are detected, step S44 shall be performed for each non-wearer.

Specifically, first, the authentication control unit 344 controls the face authentication of the non-wearer (S441). Specifically, the authentication control unit 344 extracts the face region of the non-wearer from the captured image and acquires the face feature information from the face region. Then, the authentication control unit 344 includes the face feature information in the face authentication request and transmits it to the authentication device 200 via the network N. Then, the authentication control unit 344 receives the face authentication result from the authentication device 200.

Then, the authentication control unit 344 determines whether or not the face authentication is successful (S442). Specifically, the authentication control unit 344 determines whether or not the received face authentication result includes information indicating success. When it is determined in step S442 that the face authentication is successful, the identification unit 345 specifies the seat position based on the spectator ID of the non-wearer (S443). Specifically, the specific unit 345 acquires a user ID that has succeeded in face authentication from the face authentication result as a spectator ID. Then, the identification unit 345 identifies the seat position 313 associated with the acquired spectator ID 312 from the storage unit 310.

If it is determined in step S442 that face recognition has failed, the identification unit 345 specifies the seat position based on the shooting direction (S444). Specifically, as described above, the specific unit 345 specifies the seat position based on the shooting direction and the detection position of the non-wearer.

After step S443 or S444, the notification unit 346 notifies the management terminal 130 or the staff terminal 120 via the network N of the warning information including the information indicating the specified seat position (S45). The warning information may include a photographed image and a seat position. The notification unit 346 may send warning information to the management terminal 130 or the staff terminal 120 so as to highlight and display the area where the non-wearer of the mask is detected in the map of the entire stadium 100. .. Thereby, for example, the management terminal 130 or the staff terminal 120 can display a map of the entire stadium 100 on the screen and emphasize the audience area or block (for example, in red) in which the non-wearer of the mask is detected. FIG. 10 is a diagram showing a display example of the warning information R0 of the non-wearer of the mask according to the second embodiment. Here, the management terminal 130 or the like displays a map of the entire stadium 100 on the screen, and indicates that the warning information R0 is highlighted and displayed in a partial area (block) in the spectator area 102. When the manager U2 or the stadium staff U1 accepts the selection of the warning information R0 or the spectator area 102, the management terminal 130 or the staff terminal 120 may display the screens of FIGS. 11 and 12, which will be described later.

FIG. 11 is a diagram showing a display example of the warning information R1 of the non-wearer of the mask according to the second embodiment. Here, the management terminal 130 or the staff terminal 120 indicates that the warning information R1 is displayed on the screen at the detected seat position in the captured image. The warning information R1 is an example including a frame line (dashed line) of the face area of the non-wearer and a seat number (for example, “S123”). For example, the manager U2 may check the warning information R1 displayed on the management terminal 130 and notify the staff terminal 120 of the seat position (seat number, etc.). As a result, the stadium staff U1 can grasp the seat position where the non-wearer of the mask exists by the staff terminal 120, and can go close to the non-wearer and urge him to wear the mask. Alternatively, the same applies even if the staff terminal 120 receives and displays the warning information directly from the spectator monitoring device 300. The management terminal 130 or the staff terminal 120 may display only one of the seat number and the frame line of the non-masked wearer as warning information R1 on the screen.

FIG. 12 is a diagram showing a display example of the warning information R11 of the non-wearer of the mask according to the second embodiment. That is, the warning information R1 in FIG. 11 is not limited to the non-wearer's frame line or seat number. In FIG. 12, the management terminal 130 and the like show that the warning information R11 is displayed on the screen in the block in the audience area where the non-wearer is detected in the captured image. The warning information R11 is an example including a frame line (broken line) and a block number (for example, “A23”) of a block in which a non-wearer exists in the audience area in the captured image. The management terminal 130 or the like may display only one of the block border and the block number as warning information R11 on the screen. The warning information R11 may surround the entire spectator area where the non-wearer is detected with a frame line. Alternatively, the warning information R11 may surround a partial area of a specific block in the captured image in which a non-wearer exists with a frame line. Alternatively, the warning information R11 may surround an area spanning a plurality of blocks in the audience area with a frame line. Further, a part or all of the above-mentioned warning information R1 and R11 may be combined.

As described above, in the present embodiment, the manager U2 and the stadium staff U1 can easily grasp the existence position of the non-wearer of the mask from a large number of spectators. Therefore, it is possible to support the operation staff to speak to the non-wearer. In particular, when the seat position for each spectator is pre-registered, the seat position can be accurately specified by using face recognition.

<Embodiment 3>
The third embodiment is a modification of the second embodiment described above. The spectator monitoring device according to the third embodiment further includes a determination unit for determining whether or not a predetermined condition is satisfied when a non-wearer is detected. Then, when it is determined that the predetermined condition is satisfied, the notification unit notifies the warning information. This limits excessive notification of warning information and allows the stadium staff U1 to call out to wear a mask when it is more necessary.

Further, it is desirable that the spectator monitoring device according to the third embodiment further includes a first measuring unit for measuring an index value relating to non-wearing of the mask when a non-wearer is detected. In this case, the determination unit determines whether or not the index value satisfies a predetermined condition. In this way, since the determination is made using the measured index value, it is possible to more accurately determine the necessity of notification of the warning information.

Since the overall configuration of the third embodiment is the same as that of FIG. 3, the illustration is omitted, and the differences will be described below. FIG. 13 is a block diagram showing the configuration of the spectator monitoring device 300a according to the third embodiment. In the storage unit 310 of the spectator monitoring device 300a, the program 311 is replaced with the program 311a as compared with the spectator monitoring device 300 described above. Further, the control unit 340 of the spectator monitoring device 300a has a measurement unit 347 and a determination unit 348 added as compared with the above-mentioned spectator monitoring device 300. Other configurations are the same as those of the spectator monitoring device 300.

Program 311a is a computer program in which the processing of the audience monitoring method according to the third embodiment is implemented.

The measuring unit 347 is an example of the above-mentioned first measuring unit. For example, the measuring unit 347 may measure the non-wearing time when the non-wearing person does not wear the mask as an index value. Alternatively, the measuring unit 347 may measure the number of times a specific spectator is detected as a non-wearer from a plurality of captured images taken a plurality of times with respect to the spectator area as an index value. In this case, the measuring unit 347 shall hold the number of detections in the memory 320 or the like. Alternatively, the measuring unit 347 may measure the number of non-wearers detected in the spectator area as an index value. Alternatively, when a non-wearer is detected, the measuring unit 347 may measure the congestion status in the spectator area as an index value. The measuring unit 347 may include the distance between the non-wearer and the surrounding spectators in the congestion situation.

The determination unit 348 is an example of the determination unit described above. For example, the determination unit 348 may determine that the predetermined condition is satisfied when the measured non-wearing time exceeds the predetermined time. As a result, while the spectator has just removed the mask, the mask may be re-worn after that, so that excessive notification of warning information can be suppressed.

Alternatively, the determination unit 348 may determine that the predetermined condition is satisfied when the measured number of detections exceeds the predetermined number. As a result, it is possible to call out to the spectators who have removed the mask many times even for a short time to wear the mask appropriately.

Alternatively, the determination unit 348 may determine that the predetermined condition is satisfied when the measured number of detected persons exceeds the predetermined number. As a result, when the number of non-wearers is small, it is possible to suppress excessive notification of warning information.

Alternatively, the determination unit 348 may determine that the predetermined condition is satisfied when the measured congestion status exceeds a predetermined degree. For example, the determination unit 348 may determine whether or not the congestion status exceeds a predetermined degree depending on whether or not another spectator is present within a predetermined range of the non-wearer among the measured congestion status. As a result, when the audience area is dense, it is possible to call out to the non-wearer to wear an appropriate mask.

FIG. 14 is a flowchart showing the flow of the audience monitoring process according to the third embodiment. Since steps S41 to S44 and S45 are the same as those in FIG. 9 described above, the differences from FIG. 9 will be described below. After step S44, the measuring unit 347 measures an index value relating to non-wearing of the mask (S46). Here, the index value includes, for example, the above-mentioned non-wearing time, the number of detections, the number of detected people, or the congestion situation, but is not limited thereto. Further, the measuring unit 347 may measure by combining two or more of the non-wearing time, the number of detections, the number of detected people, and the congestion status.

Then, the determination unit 348 determines whether or not the index value measured in step S46 satisfies a predetermined condition (S47). For example, when the index value is the non-wearing time, the determination unit 348 determines whether or not the non-wearing time exceeds a predetermined time. When the index value is the number of detections, the determination unit 348 determines whether or not the number of detections exceeds a predetermined number. When the index value is the number of detected persons, the determination unit 348 determines whether or not the number of detected persons exceeds a predetermined number. Further, when the index value is a congestion status, the determination unit 348 determines whether or not the congestion status exceeds a predetermined degree.

When it is determined in step S47 that the index value satisfies a predetermined condition, the notification unit 346 notifies the warning information as described above (S45). On the other hand, if it is determined in step S47 that the index value does not satisfy the predetermined condition, the process proceeds to step S43.

Note that the third embodiment does not necessarily have to include the measuring unit 347. In that case, for example, the determination unit 348 may determine that the predetermined condition is satisfied when the detected non-wearer is not eating or drinking. That is, the determination unit 348 determines whether or not the detected non-wearer is eating or drinking. For example, the determination unit 348 may determine whether or not the detected non-wearer is eating or drinking by using the AI model that has learned the image around the face where the person is eating and drinking. As a result, it is possible to suppress excessive notification of warning information when eating or drinking. Alternatively, the administrator U2 determines whether or not the non-wearer is eating or drinking after seeing the photographed image and the warning information displayed on the management terminal 130, and the determination result is determined by the audience monitoring device via the management terminal 130. You may input to 300. In that case, the determination unit 348 determines that the predetermined condition is not satisfied when the input determination result indicates that the food or drink is being eaten. That is, even a person who does not wear the mask is excluded from the warning target. At this time, the notification unit 346 may give an instruction to cancel the warning information already notified. Therefore, in this case as well, it is possible to suppress excessive notification of warning information. However, the determination unit 348 may count the number of times the detected non-wearer eats and drinks. Then, the determination unit 348 may determine that the predetermined condition is satisfied when the number of times of eating and drinking exceeds the predetermined value.

Alternatively, the determination unit 348 may determine that the predetermined condition is satisfied when the specified partial area is a designated area where the mask is required to be worn. For example, in the eat space in the stadium 100, it may be permitted not to wear a mask. Therefore, the spectator monitoring device 300 shall register in advance a space (other than the eat space) in which a mask is required to be worn, such as a normal bleachers, as a designated area. In this case, the determination unit 348 determines whether or not the seat position or block specified by step S44 is in the designated area. As a result, the notification of warning information can be suppressed in the eat space.

<Embodiment 4>
The fourth embodiment is a modification of the second or third embodiment described above. When a non-wearer is detected, the spectator monitoring device according to the fourth embodiment determines a second measuring unit that measures an index value related to non-wearing of the mask and warning information indicating the degree of warning according to the index value. It is further provided with a decision-making unit to be used. As a result, the operation staff can adjust the frequency and degree of the call by receiving the notification according to the degree of warning, and can respond in detail. In addition, the number of notification destinations may be increased or decreased or selected according to the degree of warning.

Since the overall configuration of the fourth embodiment is the same as that of FIG. 3, the illustration is omitted, and the differences will be described below. FIG. 15 is a block diagram showing the configuration of the spectator monitoring device 300b according to the fourth embodiment. In the storage unit 310 of the spectator monitoring device 300b, the program 311a is replaced with the program 311b as compared with the above-mentioned spectator monitoring device 300a. Further, in the control unit 340 of the spectator monitoring device 300b, the determination unit 348 is replaced with the determination unit 349 as compared with the above-mentioned spectator monitoring device 300a. The measuring unit 347 according to the fourth embodiment is an example of the above-mentioned second measuring unit. Other configurations are the same as those of the spectator monitoring device 300a. The spectator monitoring device 300b can be realized even if the spectator monitoring device 300 according to the second embodiment is improved.

Program 311b is a computer program in which the processing of the audience monitoring method according to the fourth embodiment is implemented.

The decision unit 349 is an example of the above-mentioned decision unit. For example, the determination unit 349 may determine the color of the frame line surrounding the face area of the non-wearer so that the warning degree becomes stronger in the order of blue, yellow, and red. For example, the determination unit 349 may determine the warning degree (for example, the color of the border) of the warning information according to the non-wearing time. That is, the determination unit 349 may determine the warning information so that the warning degree becomes stronger (higher) as the non-wearing time becomes longer. As a result, the spectators (and their locations) who have been unmasked for a long time among non-wearers can be easily identified, so that the stadium staff U1 can refer to the spectators who have been unmasked for a long time. You can give priority to calling out.

Alternatively, the determination unit 349 may determine the warning degree (for example, the color of the frame line) of the warning information according to the number of detections. That is, the determination unit 349 may determine the warning information so that the warning degree becomes stronger (higher) as the number of detections increases. This makes it possible to easily identify the spectators (and their whereabouts) who have removed the mask many times even for a short time, so the stadium staff U1 gives priority to the spectators. It can be performed.

Alternatively, the determination unit 349 may determine the warning degree (for example, the color of the frame line) of the warning information according to the number of detected persons. That is, the determination unit 349 may determine the warning information so that the warning degree becomes stronger (higher) as the number of detectors increases. As a result, the stadium staff U1 can make a call according to the number of detected people who are not wearing the mask. Further, when the number of detected persons exceeds a predetermined number, the determination unit 349 may add a speaker or the like near the specified partial area to the notification destination of the warning information. As a result, the burden on the stadium staff U1 to speak to a large number of people can be reduced.

Alternatively, the determination unit 349 may determine the warning degree (for example, the color of the border) of the warning information according to the congestion situation. That is, the determination unit 349 determines the warning information so that the closer the non-wearer and the other spectators are, and the closer the spectators are, the stronger (higher) the warning level is. May be good. As a result, the stadium staff U1 can easily grasp that there is a non-wearer of the mask in a place where the spectator area is dense, and can give a higher priority to the voice.

Note that the determination unit 349 may include the index value itself such as the measured non-wearing time, the number of detections, the number of detected persons, or the congestion status in the warning information. As a result, the management terminal 130 or the like that has received the warning information can display the index value on the screen. Therefore, the manager U2 and the stadium staff U1 can visually recognize the color and the index value to more easily grasp the non-wearer.

FIG. 16 is a flowchart showing the flow of the audience monitoring process according to the fourth embodiment. Since steps S41 to S44, S46 and S45 are the same as those in FIG. 14 described above, the differences from FIG. 14 will be described below. After step S46, the determination unit 349 determines warning information indicating the degree of warning according to the index value (S48). Here, as described above, the determination unit 349 determines the warning information indicating the degree of warning according to any one or a combination of the non-wearing time, the number of detections, the number of detections, and the congestion situation. After that, the notification unit 346 notifies the determined warning information as described above (S45).

17 to 20 are diagrams showing an example of displaying warning information of a non-wearer of a mask according to the fourth embodiment. FIG. 17 shows an example in which when the non-wearing time is measured as an index value, the notation “mask non-wearing time 5 minutes!” Is displayed as the warning information R1a. FIG. 18 shows an example in which when the number of detections is measured as an index value, the notation “Number of times the mask is not worn 5 times / 15 minutes!” Is displayed as the warning information R1b. FIG. 19 shows an example in which when the number of detected people per spectator area is measured as an index value, the notation “3 non-mask wearers!” Is displayed as warning information R1c. FIG. 20 shows an example in which when the congestion situation is measured as an index value, the notation “3 people around the person who does not wear the mask!” Is displayed as the warning information R1d. Note that FIG. 20 shows an example in which the area surrounding the non-masked wearer is surrounded by a frame line (broken line), but the frame line is not essential. Further, in FIGS. 17 to 20, instead of the frame line of the face area of the non-wearer of the warning information R1a to R1d, the frame line or the block number of the block unit or the like as in the warning information R11 of FIG. May be used.

The display example of the warning information in FIGS. 17 to 20 may be applied in the third embodiment.

When the detection unit 343 detects that the non-wearer of the mask makes a loud voice, cheers, talks, stands up, etc., the determination unit 349 determines the warning information so that the warning level becomes high. You may. For example, the detection unit 343 may detect that the detected non-wearer is having a conversation or the like by using an AI model that has learned an image of a face around a person having a conversation. Alternatively, the stadium 100 may be provided with a microphone that collects the voices of the spectators for each spectator area or each block. In this case, even if the detection unit 343 detects that the detected non-wearer is having a conversation or the like when the volume collected from the microphone corresponding to the specified block is equal to or higher than a predetermined value. good.

<Embodiment 5>
The fifth embodiment is a modification of the above-mentioned embodiments 2 to 4. The spectator monitoring device according to the fifth embodiment further includes a third measuring unit for measuring the number of detected non-wearers detected for each of the plurality of partial areas in the spectator area. Then, the specific unit identifies the partial area in which the number of detected persons is larger among the plurality of partial areas, and the notification unit gives priority to the specified partial area to notify the warning information. As a result, the stadium staff U1 can give priority to the block having a higher risk of infectious disease. Further, the notification unit may add a speaker corresponding to the specified block to the notification destination. The third measuring unit may measure an index value related to non-wearing of the mask for each of the plurality of partial regions in the audience area. The index value may be, for example, the non-wearing time, the number of detections of the non-wearing person, the congestion situation around the non-wearing person, or the like, as in the second measuring unit described above. In this case, the specific unit identifies the partial area having the larger index value among the plurality of partial areas, and the notification unit gives priority to the specified partial area to notify the warning information. That is, the specific unit may determine the priority for notifying the warning information based on the index value for each partial area.

Since the overall configuration of the fifth embodiment is the same as that of FIG. 3, the illustration is omitted. Further, since the configuration of the spectator monitoring device of the fifth embodiment is the one in which the measurement unit 347 is added to FIG. 8, the illustration is omitted, and the differences will be described below. The spectator monitoring device of the fifth embodiment can be realized by improving the

spectator monitoring devices

300, 300a or 300b according to the second to fourth embodiments.

FIG. 21 is a flowchart showing the flow of the audience monitoring process according to the fifth embodiment. Since steps S41 to S43 are the same as those in FIG. 9 described above, the differences from FIG. 9 will be described below.

When one or more non-wearers are detected in step S42, the measuring unit 347 measures the number of non-wearers detected for each of the plurality of blocks in the audience area (S51). Then, the specifying unit 345 identifies a block having a larger number of detected people among the plurality of blocks (S52). Then, the notification unit 346 notifies the management terminal 130 or the staff terminal 120 of the warning information for identifying the specified block via the network N (S53). At this time, the notification unit 346 may further notify the warning information to the speaker corresponding to the specified block via the network N.

After that, the specific unit 345 determines whether or not there is an unnotified block in step S53 among the blocks detected by the non-wearer (S54). When it is determined that there is an unnotified block, the identification unit 345 identifies a block having a larger number of detected people among the unnotified blocks (S55). Then, the notification unit 346 notifies the warning information as described above (S53). Then, when it is determined in step S54 that there is no unnotified block, the audience monitoring process is terminated.

FIG. 22 is a diagram showing a display example of warning information of a non-wearer of the mask according to the fifth embodiment. FIG. 22 shows an example in which three non-wearers are detected in the block on the left side from the front of the audience area, and one non-wearer is detected in the block on the right side. According to the flow of FIG. 21, the warning information R2a is first displayed in the block on the left side, and then the warning information R2b is displayed in the block on the right side. However, the warning information R2a and R2b may be displayed at the same time. The warning information R2a indicates the notation "(1) three non-mask wearers!", And the warning information R2b indicates the notation "(2) one non-mask wearer!". Therefore, it is shown that the warning information R2a including the notation of "(1)" has a higher priority than the warning information R2b including the notation of "(2)". Further, in this example, the character size of the warning information R2a is larger than the character size of the warning information R2b. This also indicates that the warning information R2a has a higher priority than the warning information R2b. That is, the administrator U2 and the like can grasp the block having a higher density of non-wearers by at least one of the display order of the warning information R2a and the warning information R2b, the number at the beginning, or the character size. In other words, the administrator U2 and the like can grasp the high-priority block for calling out to wear the mask by at least one of the display order of the warning information, the number at the beginning, and the character size. Therefore, the stadium staff U1 can give priority to the block corresponding to the warning information R2a.

As described above, according to the present embodiment, the management terminal 130 or the staff terminal 120 can receive warning information in order from the block in which the number of non-masked wearers is large. Therefore, the manager U2 and the stadium staff U1 can grasp the block having a high priority for calling out. Therefore, the stadium staff U1 can call out to effectively reduce the risk of infectious diseases.

<Embodiment 6>
The sixth embodiment is a modification of the above-mentioned embodiments 2 to 5. In the sixth embodiment, once the warning information is notified, the notification is given according to the initially detected situation of the spectator. For example, the acquisition unit according to the sixth embodiment acquires a second captured image in which the (same) spectator area is captured after the warning information is notified. Then, when the spectator detected as a non-wearer in the first captured image is not detected as a non-wearer in the second captured image, the notification unit may notify the information terminal to that effect. As a result, when the spectator once removes the mask and then re-wears the mask, the stadium staff U1 can grasp the fact and can stop talking to the spectator. Therefore, it is possible to suppress excessive voice calls and reduce the burden on the stadium staff U1.

Further, the acquisition unit according to the sixth embodiment acquires a third photographed image in which the (same) spectator area is photographed after the warning information is notified. The second captured image and the third captured image may be the same. Then, it is assumed that the detection unit detects that the spectator detected as a non-wearer in the first captured image has moved from the specified partial region in the third captured image. At this time, the notification unit may notify the information terminal that the movement has been detected. This makes it possible to notify that the spectator has left with the mask removed. Therefore, the stadium staff U1 can give priority to the audience.

In this case, the specific part should further specify the position of the spectator who has been detected to have moved after the movement. At this time, the notification unit may notify the information terminal of the specified position after the movement. This makes it possible to notify the immediate presence position of the spectator with the mask removed. Therefore, the stadium staff U1 can make a call after tracking the spectator. Therefore, the burden on the stadium staff U1 can be reduced.

Since the overall configuration of the sixth embodiment and the configuration of the spectator monitoring device are the same as those in FIGS. 3 and 8, the illustrations are omitted, and the differences will be described below.

FIG. 23 is a flowchart showing the flow of the audience monitoring process according to the sixth embodiment. The same reference numerals are given to the processes equivalent to those in FIG. As a premise, for example, it is assumed that the non-wearer has already been detected in the audience area 101 by the process of FIG. 9, and the warning information has already been notified. Then, it is assumed that the spectator monitoring device 300 holds the detected spectator ID or seat position (as well as the detection or notification time) of the non-wearer in the memory 320.

After that, it is assumed that the camera 110 again shoots in the shooting direction of the spectator area 101, and transmits the shot image and the shooting direction to the spectator monitoring device 300 via the network N. In response to this, the acquisition unit 342 acquires (second and third) captured images and shooting directions from the camera 110 via the network N (S41).

Next, the detection unit 343 determines whether or not a non-wearer of the mask was detected last time in the acquired shooting direction (S61). For example, the detection unit 343 refers to the memory 320 and determines whether or not a non-wearer is detected in the previously captured image in the audience area 101. Further, when it is determined that the detection is detected, the detection unit 343 identifies the detected non-wearer's spectator ID or seat position.

When it is determined that the non-wearer of the mask was detected last time (YES in step S61), the detection unit 343 determines whether or not the previous non-wearer was detected as the wearer of the mask in the captured image this time. Judgment (S62). For example, the detection unit 343 detects a non-wearer in the captured image as in step S42, and if detected, whether or not the detected non-wearer exists at the seat position specified above. To judge. Alternatively, the detection unit 343 may determine whether or not the face area of the seat position specified above in the captured image is a non-wearer of the mask.

When the previous non-wearer is detected as the wearer this time (YES in step S62), the notification unit 346 indicates that the previous non-wearer subsequently wore the mask (hence, no voice call is required). Is notified to the management terminal 130 or the staff terminal 120 (S63).

When the previous non-wearer is detected as a non-wearer this time as well (NO in step S62), the detection unit 343 determines whether or not the previous non-wearer has moved (S64). For example, the detection unit 343 performs tracking using face recognition for the non-wearer, and identifies the position of the non-wearer (audience) after movement (S65). At this time, the detection unit 343 may track (measure, detect, specify) the movement of the non-wearer by performing face recognition with a plurality of cameras. Then, the notification unit 346 notifies the management terminal 130 or the staff terminal 120 of the position of the non-wearer of the mask after the movement (S66).

When it was determined that the non-wearer of the mask was not detected last time (NO in step S61), the spectator monitoring device 300 performs steps S42 to S45 in the same manner as in FIG. It should be noted that steps S42 to S45 may be executed regardless of the determination result of step S61.

As described above, in the sixth embodiment, the stadium staff U1 can take an appropriate response by once notifying the warning information and then giving the notification according to the situation of the initially detected spectator. For example, it is possible to suppress excessive voice to the audience who re-wears the mask. In addition, when the non-wearer moves, the stadium staff U1 can give priority to calling out. In particular, at that time, the stadium staff U1 can grasp the position of the destination of the non-wearer, so that he / she can effectively speak out. In addition, these can reduce the burden on the stadium staff U1.

<Embodiment 7>
The seventh embodiment is a modification of the above-mentioned embodiments 2 to 6. In the seventh embodiment, the biometric authentication function is built into the spectator monitoring device. Since the spectator monitoring system according to the seventh embodiment is the same as the spectator monitoring system 1000 described above in which the authentication device 200 is built in the spectator monitoring device 300c, illustration and description thereof will be omitted.

FIG. 24 is a block diagram showing the configuration of the spectator monitoring device 300c according to the seventh embodiment. In the storage unit 310 of the spectator monitoring device 300c, as compared with the above-mentioned spectator monitoring device 300, the program 311 is replaced with the program 311a, and the spectator ID 312 is associated with the face feature information 316. Further, in the control unit 340 of the spectator monitoring device 300c, the authentication control unit 344 is replaced with the authentication control unit 344c as compared with the above-mentioned spectator monitoring device 300.

Program 311a is a computer program in which the processing of the audience monitoring method according to the seventh embodiment is implemented.

The face feature information 316 corresponds to the face feature information 212 of the above-mentioned authentication device 200. The face feature information 316 is associated with the audience ID 312 (user identification information). That is, the storage unit 310 includes the above-mentioned face information DB 210.

The authentication control unit 344c controls face recognition by collating the face feature information of a plurality of persons with the face feature information of the non-wearer of the mask. That is, the authentication control unit 344c collates the detected face feature information extracted from the face area of the non-wearer with the face feature information 316 stored in the storage unit 310 to perform face recognition, thereby performing face recognition. Get the authentication result.

In this embodiment, step 441 in FIG. 9 described above is replaced with face recognition processing in the audience monitoring device 300c by the authentication control unit 344c.

As described above, the same effect as that of the above-mentioned second embodiment can be obtained in the seventh embodiment as well. Therefore, by making the same improvements to the spectator monitoring devices according to the third to sixth embodiments, the same effects as those of the third to sixth embodiments can be obtained.

<Other embodiments>
In the above-described embodiment, the description is given as a hardware configuration, but the present invention is not limited to this. The present disclosure can also be realized by causing the CPU to execute a computer program for arbitrary processing.

In the above example, the program can be stored and supplied to the computer using various types of non-transitory computer readable medium. Non-temporary computer-readable media include various types of tangible storage mediums. Examples of non-temporary computer-readable media include magnetic recording media (eg, flexible disks, magnetic tapes, hard disk drives), magneto-optical recording media (eg, magneto-optical disks), CD-ROMs (ReadOnlyMemory), CD-Rs, Includes CD-R / W, DVD (DigitalVersatileDisc), semiconductor memory (eg, mask ROM, PROM (ProgrammableROM), EPROM (ErasablePROM), flash ROM, RAM (RandomAccessMemory)). The program may also be supplied to the computer by various types of transient computer readable medium. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Note that this disclosure is not limited to the above embodiment, and can be appropriately changed without departing from the spirit. Further, the present disclosure may be carried out by appropriately combining the respective embodiments.

Some or all of the above embodiments may also be described, but not limited to:
(Appendix A1)
An acquisition means for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
Audience monitoring device equipped with.
(Appendix A2)
The spectator monitoring device according to Appendix A1, wherein the specific means specifies the seat position of the non-wearer as the partial area.
(Appendix A3)
The specific means is described in Appendix A2 for specifying the seat position based on the shooting direction of the shooting device that shot the first shot image and the detection position of the non-wearer in the first shot image. Audience monitoring device.
(Appendix A4)
The registration means for registering the identification information corresponding to the facial feature information of each of the plurality of spectators and the seat position in the storage means in association with each other.
An authentication control means for controlling face recognition with reference to the storage means based on the face area of the non-wearer in the first captured image, and an authentication control means.
Further prepare
The spectator monitoring device according to Appendix A2, wherein the specific means specifies a seat position associated with a non-wearing spectator who has succeeded in face recognition as the partial area.
(Appendix A5)
When the non-wearer is detected, a determination means for determining whether or not a predetermined condition is satisfied is further provided.
The spectator monitoring device according to any one of Supplementary A1 to A4, which notifies the warning information when it is determined that the predetermined condition is satisfied.
(Appendix A6)
When the non-wearer is detected, the first measuring means for measuring the index value regarding the non-wearing of the mask is further provided.
The spectator monitoring device according to Appendix A5, wherein the determination means determines whether or not the index value satisfies the predetermined condition.
(Appendix A7)
The first measuring means measures the non-wearing time when the non-wearing person does not wear the mask as the index value.
The spectator monitoring device according to Appendix A6, wherein the determination means determines that the predetermined condition is satisfied when the non-wearing time exceeds the predetermined time.
(Appendix A8)
The first measuring means measures the number of times a specific spectator is detected as a non-wearer from a plurality of captured images taken a plurality of times with respect to the spectator area as the index value.
The spectator monitoring device according to Appendix A6, wherein the determination means determines that the predetermined condition is satisfied when the number of detections exceeds a predetermined number.
(Appendix A9)
The first measuring means measures the detected number of non-wearers detected in the audience area as the index value.
The spectator monitoring device according to Appendix A6, wherein the determination means determines that the predetermined condition is satisfied when the number of detected persons exceeds a predetermined number.
(Appendix A10)
When the non-wearer is detected, the first measuring means measures the congestion state of the spectator area as the index value.
The spectator monitoring device according to Appendix A6, wherein the determination means determines that the predetermined condition is satisfied when the congestion situation exceeds a predetermined degree.
(Appendix A11)
The spectator monitoring device according to Appendix A5, wherein the determination means determines that the predetermined condition is satisfied when the detected non-wearer does not eat or drink.
(Appendix A12)
The spectator monitoring device according to Appendix A5, wherein the determination means determines that the predetermined condition is satisfied when the specified partial region is a designated region where the mask is required to be worn.
(Appendix A13)
When the non-wearer is detected, a second measuring means for measuring an index value relating to non-wearing of the mask, and a second measuring means.
Further provided with a determination means for determining warning information indicating the degree of warning according to the index value.
The spectator monitoring device according to any one of Supplementary A1 to A4, wherein the notification means notifies the determined warning information.
(Appendix A14)
The second measuring means measures the non-wearing time when the non-wearing person does not wear the mask as the index value.
The spectator monitoring device according to Appendix A13, wherein the determination means determines the warning information according to the non-wearing time.
(Appendix A15)
The second measuring means measures the number of times a specific spectator is detected as a non-wearer from a plurality of captured images taken a plurality of times with respect to the spectator area as the index value.
The spectator monitoring device according to Appendix A13, wherein the determination means determines the warning information according to the number of detections.
(Appendix A16)
The second measuring means measures the detected number of non-wearers detected in the audience area as the index value.
The spectator monitoring device according to Appendix A13, wherein the determination means determines the warning information according to the number of detected persons.
(Appendix A17)
When the non-wearer is detected, the second measuring means measures the congestion state of the spectator area as the index value.
The spectator monitoring device according to Appendix A13, wherein the determination means determines the warning information according to the congestion situation.
(Appendix A18)
A third measuring means for measuring the detected number of non-wearers detected is further provided for each of the plurality of partial areas in the spectator area.
The specific means identifies a partial region having a larger number of detected people among the plurality of partial regions.
The spectator monitoring device according to any one of Supplementary A1 to A4, wherein the notification means gives priority to the specified partial area to notify the warning information.
(Appendix A19)
The acquisition means acquires a second captured image in which the audience area is captured after the warning information is notified.
When the spectator detected as the non-wearer in the first photographed image is not detected as the non-wearer in the second photographed image, the notification means informs the information terminal to that effect. The spectator monitoring device according to any one of the appendices A1 to A18 to be notified.
(Appendix A20)
The acquisition means acquires a third captured image in which the audience area is captured after the warning information is notified.
The detection means detects that the spectator detected as the non-wearer in the first captured image has moved from the specified partial region in the third captured image, and detects that the spectator has moved from the specified partial region.
The spectator monitoring device according to any one of Supplementary A1 to A19, wherein the notification means notifies the information terminal that the movement has been detected.
(Appendix A21)
The specific means identifies the position of the spectator after the movement when the movement is detected, and the identification means determines the position of the spectator after the movement.
The spectator monitoring device according to Appendix A20, wherein the notification means notifies the information terminal of the specified position after movement.
(Appendix A22)
The spectator monitoring device according to any one of Supplementary A1 to A21, wherein the notification means further notifies the warning information to a predetermined output device in the facility.
(Appendix B1)
An imaging device that photographs an audience area including multiple spectators in a predetermined facility,
Audience monitoring device connected to the photographing device and
Equipped with
The spectator monitoring device is
An acquisition means for acquiring a first captured image captured by the imaging device, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
Audience monitoring system equipped with.
(Appendix B2)
The spectator monitoring system according to Appendix B1, wherein the specific means specifies the seat position of the non-wearer as the partial area.
(Appendix C1)
The computer
Acquire the first photographed image in which the spectator area including a plurality of spectators in a predetermined facility is photographed.
A non-wearer who is a spectator who is not wearing a mask is detected from the first captured image, and the mask is not worn.
A partial area of the audience area including the non-wearer is identified.
Notify the information terminal of the facility operator of the warning information about the specified partial area.
Audience monitoring method.
(Appendix D1)
An acquisition process for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection process for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image, and
Specific processing for specifying a partial area including the non-wearer in the audience area, and
Notification processing for notifying the information terminal of the facility operation personnel of warning information about the specified partial area, and
A non-temporary computer-readable medium containing an audience monitoring program that causes a computer to run.

Although the invention of the present application has been described above with reference to the embodiments (and examples), the invention of the present application is not limited to the above embodiments (and examples). Various changes that can be understood by those skilled in the art can be made within the scope of the invention of the present application in terms of the configuration and details of the invention of the present application.

1 Audience monitoring device 11 Acquisition unit 12 Detection unit 13 Specific unit 14 Notification unit 1000 Audience monitoring system 100 Stadium 101 Audience area 1011 block 101i block 10n Audience area 10n1 block 10nj block 110 Camera 110a Camera 110b Camera 120 Staff terminal 130 Management terminal 200 Authentication Device 210 Face information DB
211 User ID
212 Face feature information 220 Face detection unit 230 Feature point extraction unit 240 Registration unit 250 Authentication unit 300 Audience monitoring device 300a Audience monitoring device 300b Audience monitoring device 300c Audience monitoring device 310 Storage unit 311 Program 311a Program 311b Program 311c Program 312 Audience ID
313 Seat position 314 Shooting direction 315 Audience area 3151 block 31511 Seat position 3151k Seat position 315i block 316 Face feature information 320 Memory 330 IF part 340 Control part 341 Registration part 342 Acquisition part 343 Detection part 344 Authentication control part 344c Part 346 Notification part 347 Measurement part 348 Judgment part 349 Decision part R0 Warning information R1 Warning information R11 Warning information R1a Warning information R1b Warning information R1c Warning information R1d Warning information R2a Warning information R2b Warning information N Network U1 Stadium staff U2 Administrator

Claims

An acquisition means for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
Audience monitoring device equipped with.
The spectator monitoring device according to claim 1, wherein the specific means specifies the seat position of the non-wearer as the partial area.
According to claim 2, the specifying means specifies the seat position based on the shooting direction of the shooting device that shot the first shot image and the detection position of the non-wearer in the first shot image. Audience monitoring device as described.
The registration means for registering the identification information corresponding to the facial feature information of each of the plurality of spectators and the seat position in the storage means in association with each other.
An authentication control means for controlling face recognition with reference to the storage means based on the face area of the non-wearer in the first captured image, and an authentication control means.
Further prepare
The spectator monitoring device according to claim 2, wherein the specific means specifies a seat position associated with a non-wearing spectator who has succeeded in face recognition as the partial area.
When the non-wearer is detected, a determination means for determining whether or not a predetermined condition is satisfied is further provided.
The spectator monitoring device according to any one of claims 1 to 4, wherein the notification means notifies the warning information when it is determined that the predetermined condition is satisfied.
When the non-wearer is detected, the first measuring means for measuring the index value regarding the non-wearing of the mask is further provided.
The audience monitoring device according to claim 5, wherein the determination means determines whether or not the index value satisfies the predetermined condition.
The first measuring means measures the non-wearing time when the non-wearing person does not wear the mask as the index value.
The spectator monitoring device according to claim 6, wherein the determination means determines that the predetermined condition is satisfied when the non-wearing time exceeds the predetermined time.
The first measuring means measures the number of times a specific spectator is detected as a non-wearer from a plurality of captured images taken a plurality of times with respect to the spectator area as the index value.
The audience monitoring device according to claim 6, wherein the determination means determines that the predetermined condition is satisfied when the number of detections exceeds a predetermined number.
The first measuring means measures the detected number of non-wearers detected in the audience area as the index value.
The audience monitoring device according to claim 6, wherein the determination means determines that the predetermined condition is satisfied when the number of detected persons exceeds a predetermined number.
When the non-wearer is detected, the first measuring means measures the congestion state of the spectator area as the index value.
The audience monitoring device according to claim 6, wherein the determination means determines that the predetermined condition is satisfied when the congestion situation exceeds a predetermined degree.
The spectator monitoring device according to claim 5, wherein the determination means determines that the predetermined condition is satisfied when the detected non-wearer does not eat or drink.
The spectator monitoring device according to claim 5, wherein the determination means determines that the predetermined condition is satisfied when the specified partial region is a designated region where the mask is required to be worn.
When the non-wearer is detected, a second measuring means for measuring an index value relating to non-wearing of the mask, and a second measuring means.
Further provided with a determination means for determining warning information indicating the degree of warning according to the index value.
The spectator monitoring device according to any one of claims 1 to 4, wherein the notification means notifies the determined warning information.
The second measuring means measures the non-wearing time when the non-wearing person does not wear the mask as the index value.
The spectator monitoring device according to claim 13, wherein the determination means determines the warning information according to the non-wearing time.
The second measuring means measures the number of times a specific spectator is detected as a non-wearer from a plurality of captured images taken a plurality of times with respect to the spectator area as the index value.
The audience monitoring device according to claim 13, wherein the determination means determines the warning information according to the number of detections.
The second measuring means measures the detected number of non-wearers detected in the audience area as the index value.
The spectator monitoring device according to claim 13, wherein the determination means determines the warning information according to the number of detected persons.
When the non-wearer is detected, the second measuring means measures the congestion state of the spectator area as the index value.
The spectator monitoring device according to claim 13, wherein the determination means determines the warning information according to the congestion situation.
A third measuring means for measuring the detected number of non-wearers detected is further provided for each of the plurality of partial areas in the spectator area.
The specific means identifies a partial region having a larger number of detected people among the plurality of partial regions.
The spectator monitoring device according to any one of claims 1 to 4, wherein the notification means gives priority to the specified partial area to notify the warning information.
The acquisition means acquires a second captured image in which the audience area is captured after the warning information is notified.
When the spectator detected as the non-wearer in the first photographed image is not detected as the non-wearer in the second photographed image, the notification means informs the information terminal to that effect. The spectator monitoring device according to any one of claims 1 to 18 to be notified.
The acquisition means acquires a third captured image in which the audience area is captured after the warning information is notified.
The detection means detects that the spectator detected as the non-wearer in the first captured image has moved from the specified partial region in the third captured image, and detects that the spectator has moved from the specified partial region.
The spectator monitoring device according to any one of claims 1 to 19, wherein the notification means notifies the information terminal that the movement has been detected.
The specific means identifies the position of the spectator after the movement when the movement is detected, and the identification means determines the position of the spectator after the movement.
The spectator monitoring device according to claim 20, wherein the notification means notifies the information terminal of the specified position after movement.
The spectator monitoring device according to any one of claims 1 to 21, wherein the notification means further notifies the warning information to a predetermined output device in the facility.
An imaging device that photographs an audience area including multiple spectators in a predetermined facility,
Audience monitoring device connected to the photographing device and
Equipped with
The spectator monitoring device is
An acquisition means for acquiring a first captured image captured by the imaging device, and
A detection means for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image,
Specific means for identifying a partial area including the non-wearer in the audience area, and
A notification means for notifying the information terminal of the operation staff of the facility of warning information about the specified partial area, and
Audience monitoring system equipped with.
The spectator monitoring system according to claim 23, wherein the specific means specifies the seat position of the non-wearer as the partial area.
The computer
Acquire the first photographed image in which the spectator area including a plurality of spectators in a predetermined facility is photographed.
A non-wearer who is a spectator who is not wearing a mask is detected from the first captured image, and the mask is not worn.
A partial area of the audience area including the non-wearer is identified.
Notify the information terminal of the facility operator of the warning information about the specified partial area.
Audience monitoring method.
An acquisition process for acquiring a first captured image in which a spectator area including a plurality of spectators in a predetermined facility is captured, and
A detection process for detecting a non-wearer who is a spectator who is not wearing a mask from the first captured image, and
Specific processing for specifying a partial area including the non-wearer in the audience area, and
Notification processing for notifying the information terminal of the facility operation personnel of warning information about the specified partial area, and
A non-temporary computer-readable medium containing an audience monitoring program that causes a computer to run.