JP7209472B2 - Information processing system, information processing method, and program - Google Patents

Description

The present invention relates to an information processing system, an information processing method, and a program.

Conventionally, when taking a record of the implementation and attendance of a training session, etc., the manager visually identifies the part where the relevant participant is shown in the group photo of the participants of the meeting, and the registered face photo of the participant There is disclosed a technique for confirming attendance by confirming the corresponding relationship between the attendees and preventing fraudulent acts such as substitution and falsification (for example, Patent Document 1).

JP-A-2006-53597

However, in the attendance confirmation method described above, if there is a registered face photo, it is possible to match it with the group photo of the participants. , the attendance cannot be confirmed and an error occurs. In this case, in order to confirm attendance, it is necessary to confirm the identity verbally or newly register a photograph of the face. Moreover, such procedures impose a heavy burden on the administrator.

SUMMARY OF THE INVENTION The present invention has been made in consideration of such circumstances, and an object of the present invention is to provide an information processing system, an information processing method, and a program that reduce the workload in confirming the attendance of persons whose face images have not been registered. be one.

(1) The present invention has been made to solve the above problems, and as one aspect of the present invention, the roll call is performed based on imaging data including one or more call targets obtained from an input device. a face image detection unit that detects a face image of a subject at the time of roll call; a storage unit that stores the face image of the subject of the roll call; the face image of the subject of the roll call stored in the storage; a determining unit for determining whether or not the target person for the roll call is a person who is scheduled to be the target person for the roll call based on the face image detected by the face image detecting unit; a face image registration unit that stores the face image of the detected roll call target person at the time of the roll call in the storage unit when it is not determined that the person is scheduled to be the roll call target person, A judgment result by the judging unit , the face image detected by the face image detecting unit and stored in the storage unit when the person is not judged to be the person expected to be the roll call target. a display unit for displaying the face image of the target person for roll call stored in the storage unit when there is a candidate face image that is assumed to be the same person as the face image of the target person for roll call . An information processing system characterized by comprising:

(2) Further, as an aspect of the present invention, the display unit displays the image data detected by the face image detection unit when the person is not determined to be the person scheduled as the roll call target. The information processing system according to (1), wherein the attribute information of the subject of (1) and the candidate face image are both displayed.

(3) Further, as one aspect of the present invention, a roll call recording unit for storing age information of the person to be called is further provided, and the determination unit detects a plurality of face images at the time of roll call by the face image detection unit. In the case where the age information of the target person is stored in the roll call recording unit, deriving the expected age of the target person from the face image detected by the face image detection unit at the time of the call, It is determined whether the assumed age and the age information of the roll call candidate stored in the roll call recording unit are similar or different, and if one of the roll call candidates and the assumed age are close, In addition, when the assumed age is different from the other said roll call target person, the roll call target person stored in the roll call recording unit and the one said roll call target person at the time of the roll call match. (1) or (2) for estimating that the possibility is high, and displaying the estimation result estimated by the determination unit together with the facial image of the person to be called, which is stored in the storage unit ; The information processing system described.
(4) Further, as one aspect of the present invention, the determination result by the determination unit is a result determined from facial images corresponding to the position information and the determination timing at which the roll call was performed, and the information processing system of (1) to (3), further comprising a verification unit that verifies that the face image is correctly associated with the roll call subject based on the determination result Any one information processing system.

( 5 ) Further, as one aspect of the present invention, the storage unit classifies and stores the face images of the roll call target person based on a predetermined classification rule (1) to ( 4 ). ).

( 6 ) Further, as one aspect of the present invention, the determination unit performs the determination based on the imaging data corresponding to the classification according to the presence or absence of equipment around the face of the subject of roll call. The information processing system according to ( 5 ) characterized by the above.

( 7 ) Further, as one aspect of the present invention, the facial image registration further includes an input content reception unit that receives an operation input or a voice input of position information and information related to determination timing by a user of the input device. The information processing system according to any one of (1) to ( 7 ), wherein the unit stores the face image in the storage unit based on the content input to the input content reception unit. is.

(8) Further, as one aspect of the present invention, the computer detects the face image of the person to be called based on the imaging data including one or more persons to be called, obtained from the input device, storing the face image of the person to be called, and determining the person to be called as the person to be called based on the stored face image of the person to be called and the detected face image at the time of the call. If it is not determined that the person is the person scheduled as the roll call target, the face image of the detected roll call target at the time of the roll call is stored. In addition, when the person is not determined to be the person scheduled as the person to be called, and the detected face image at the time of the call is the same as the stored face image of the person to be called. In the information processing method, when there is a candidate face image assumed to be a person, the stored face image of the roll call target person is displayed.

(9) Further, as one aspect of the present invention, the computer detects the face image of the person to be called based on the imaging data including one or more persons to be called, obtained from the input device,
The face image of the person to be called is stored, and the person to be called is scheduled to be the person to be called based on the stored face image of the person to be called and the detected face image at the time of the call. If the person is not determined to be the person scheduled as the roll call target, the face image of the detected roll call target at the time of the roll call is stored. In addition, when the person is not determined to be the person scheduled as the person to be called, and the detected face image at the time of the call is the same as the stored face image of the person to be called. A program for displaying the stored face image of the roll call target when there is a face image of a candidate assumed to be a person .

According to the present invention, it is possible to provide an information processing system capable of reducing the workload in confirming the attendance of a person whose face image has not been registered.

1 is a block diagram showing a functional configuration of an information processing system 1 according to a first embodiment; FIG. 5 is a diagram showing a display example of a roll call result displayed on a display unit 520. FIG. FIG. 10 is a diagram showing an operation screen image displayed on a display unit 520 when a roll-call facial image FN of a mismatched person N1 is stored as a facial image FW. 7 is a diagram showing an example of classification of determination results by a determination unit 140; FIG. 4 is a diagram showing an example of a classification rule for matching image groups by a storage unit 110. FIG. 4 is a flow chart showing part of the flow of processing in which a supervisor X confirms a roll call using the information processing system 1. FIG. 4 is a flow chart showing part of the flow of processing in which a supervisor X confirms the consistency of a roll call using the information processing system 1. FIG. It is a block diagram of information processing system 1A concerning a 2nd embodiment.

Hereinafter, embodiments of an information processing system, an information processing method, and a program according to the present invention will be described with reference to the drawings. An information processing system 1 according to the present invention manages roll call records of workers at, for example, a construction site.

(First embodiment)
[overall structure]
FIG. 1 is a block diagram showing the functional configuration of an information processing system 1 according to the first embodiment. The information processing system 1 includes, for example, an arithmetic device 100 and an input device 500 . The computing device 100 and the input device 500 are connected to a network NW, for example, and communicate with each other via a communication line. The network NW is, for example, a wireless LAN (Local Area Network), Bluetooth (registered trademark), LTE (Long Term Evolution) (registered trademark), or the like. Further, the computing device 100 and the input device 500 may be connected to each other via a USB cable (Universal Serial Bus) or the like to communicate with each other.

The computing device 100 includes, for example, a storage unit 110, a communication unit 120, a recognition unit 130, a determination unit 140, and a roll call recording unit 150.

The storage unit 110 stores one or more face images FW in a state classified according to a predetermined classification rule for each subject person. The face image FW is image data used when performing face authentication of a person who is a subject. The face image FW is, for example, image data from which part or all of facial parts such as the eyes, nose and mouth of a single subject can be recognized. The predetermined classification rule is, for example, a rule for classifying face images FW based on whether or not the subject person is wearing equipment. The storage unit 110, for example, collects only face images FW in which the subject is wearing a helmet, classifies them into one system, and stores them. In addition, the storage unit 110 stores information based on the presence or absence of equipment around the face, such as when the subject wears glasses or goggles, wears a mask, or wears nothing. , and a group of images for matching composed of one or more face images FW are classified into respective systems and stored. In addition, the storage unit 110 stores the face image FW based on the shooting location (for example, outdoors, indoors, a dark place, etc.) and whether or not some kind of correction (for example, size change or backlight correction) has been performed on the imaged data. may be classified and stored.

Further, the storage unit 110 derives the feature amount of the face image FW and stores it together with the face image FW. A feature amount is numerical data used when face authentication processing is performed using the face image FW, and is a numerical representation of the features of the face of the subject of the face image FW. be. When there are a plurality of face images FW of the same person, the storage unit 110 may derive one feature amount from the plurality of face images FW, or may derive a feature amount for each face image FW.

The communication unit 120 communicates with the input device 500, which will be described later. The communication unit 120 receives imaging data from the input device 500 and receives operation instructions from the user of the input device 500, for example. The communication unit 120 converts the reception result into a predetermined data format and outputs it to the recognition unit 130 .

The communication unit 120 includes an input content receiving unit 122, for example. The input content reception unit 122 converts the input signal received from the input device 500 into a predetermined data format recognizable by the recognition unit 130 and outputs the data to the recognition unit 130 . An input signal received by the input content receiving unit 122 is, for example, an audio signal.

The recognition unit 130 detects the face of each person to be called from the imaging data output from the communication unit 120, cuts out an area including the face based on the detection result, and recognizes it as a face image FN at the time of the call. Recognition section 130 outputs face image FN at roll call to determination section 140 . Note that the recognition unit 130 causes the storage unit 110 to store the face image FN at roll call based on the determination result of the determination unit 140, which will be described later.

Further, the recognition unit 130 recognizes a call target person based on an operation instruction output from the communication unit 120 and outputs an instruction to register or update call information to the call recording unit 150 . The roll call information is, for example, a record of when, where, and the presence of which roll call target person was confirmed. The recognition unit 130 is an example of a “face image detection unit” and a “face image registration unit”.

The determination unit 140 determines that the subject of the roll call face image FN is a scheduled call target person based on the roll call face image FN output from the recognition unit 130 and the face image FW stored in the storage unit 110 . Determine whether or not there is The planned roll call target is, for example, a list of roll call targets prepared in advance by the user of the information processing system 1 (hereinafter referred to as a roll call list), attributes such as affiliation and name that identify the target roll call. The person whose information is listed.

The determining unit 140 determines, for example, whether or not the subject of the facial image FN during roll call is a scheduled person to be called based on the face authentication processing of the facial image FN and the facial image FW at the time of call. The determination processing by the determination unit 140 is performed based on, for example, the feature amount of the face image FW included in the storage unit 110 . The determining unit 140 derives the probability (hereinafter referred to as probability) that the person of the subject of the roll call face image FN and the face image FW is estimated to be the same person as a result of the face authentication processing.

If the derived accuracy is equal to or greater than a preset threshold, the determining unit 140 determines that the subjects of the roll call facial image FN and the facial image FW are the same person. Further, when the derived accuracy is less than the threshold, the determination unit 140 determines that the subject person could not be identified from the face image FN during roll call, and communicates the user of the input device 500 via the communication unit 120. request to verify the identity of the person. The accuracy threshold is adjusted by the administrator Y of the information processing system 1, for example. Further, derivation of accuracy is performed by AI (Artificial Intelligence), for example. The determination unit 140 outputs an instruction to the recognition unit 130 to register the face image FN at roll call in the storage unit 110 based on the confirmation result of the corresponding person of the user of the input device 500 .

The roll call recording unit 150 records roll call information based on the determination result of the determination unit 140 . The call recording unit 150 records the call information for each person to be called based on, for example, a call list prepared in advance by the manager Y of the information processing system 1 .

In addition, when the user of the information processing system 1 has not prepared a roll call list, the roll call recording unit 150, based on the result of face authentication processing of the face image FN and the face image FW at the time of the call by the determination unit 140, A roll call list may be created.

The input device 500 is, for example, a terminal device such as smart glasses, a tablet terminal, or a smartphone. Smart glasses are, for example, head-mounted display type wearable computers that have input functions such as a microphone and a touch panel on the temples of the glasses. The input device 500 includes a camera 510, a display section 520, and an input section 530, for example.

The camera 510 captures images of one or more roll call subjects based on the operation of the user (hereinafter referred to as supervisor) who operates the input device 500 . For example, if the input device 500 is equipped with a GPS (Global Positioning System) receiver, the imaging data of the camera 510 may be provided with positional information determined by the GPS receiver.

The display unit 520 displays, for example, imaged data of the camera 510 and processing results of the arithmetic device 100 . The input unit 530 generates an input signal of roll call information based on the operation input of the user of the input device 500 . The input unit 530 is configured using an input function such as a touch panel, mouse, keyboard, headset, or the like. For example, when a voice is input through a headset or the like, the input unit 530 transmits the generated voice signal to the input content reception unit 122 of the arithmetic device 100 via the network NW.

[Usage scene 1]
An example of a usage scene in which the information processing system 1 is used will be described below. For example, a supervisor X at a construction site Z conducts a roll call using the information processing system 1 . At the construction site Z, the workers W1 to W6 who are subject to roll calls may work all day, only in the morning, or only in the afternoon. Therefore, the supervisor X confirms the roll call a plurality of times (for example, not only the morning meeting, but also the afternoon meeting, evening meeting, etc.). In addition, below, when not distinguishing a worker in particular, the worker W may be called.

The supervisor X, for example, convenes the workers W at the construction site Z and holds a morning assembly. The supervisor X confirms the roll call at any timing of the morning assembly. A supervisor X photographs one or more workers W with a camera 510 at a morning meeting. The supervisor X may photograph all of the workers W at the same time, may photograph a plurality of workers in units of role or affiliation, or may photograph each worker individually.

The supervisor X uses the input unit 530 to input, for example, the start of roll call for the morning assembly of the construction site Z on April 10th. The input device 500 transmits the input contents of the supervisor X to the arithmetic device 100 . The communication unit 120 converts the input content of the supervisor X into an input signal and outputs the input signal to the recognition unit 130 . The recognition unit 130 recognizes from the input signal output from the communication unit 120 that the roll call for the morning assembly of the construction site Z on April 10th will start, and the roll call recording unit 150 indicates that the roll call timing is April 10th. It is the morning meeting of the construction site Z, and a command is output to call up a roll call list whose position information corresponds to the construction site Z. The determination unit 140 retrieves the face image FW of the worker W to be called from the storage unit 110 based on the roll call name list retrieved by the roll call recording unit 150 . The roll call timing is an example of "determination timing".

Supervisor X operates camera 510 and input unit 530 to transmit imaging data to arithmetic device 100 . The communication unit 120 outputs the imaging data received from the input device 500 to the recognition unit 130 . The recognition unit 130 recognizes the face image FN at roll call, which is obtained by cutting out the periphery of the face of the worker W from the imaging data received from the input device 500 . The determination unit 140 collates the facial image FN during roll call with the facial image FW stored in the storage unit 110 to determine whether the facial image FN is the worker W to be called.

If the roll-call facial image FN and the facial image FW already stored in the storage unit 110 can be matched, the determination unit 140 determines that the matching is successful. In addition, when the accuracy of the face image FN and the face image FW at the time of call call is not equal to or higher than a preset threshold, or when the face image FW corresponding to the face image FN at the time of call call is not found, the determination unit 140 When the correct feature amount cannot be derived from the face image FN and the determination process cannot be performed, it is determined that the matching fails. The determination unit 140 transmits the determination result to the input device 500 via the communication unit 120 .

The computing device 100 performs face authentication processing of the worker W from the imaging data, and transmits the processing result to the input device 500 . FIG. 2 is a diagram showing an example of the processing result output to the display unit 520 as a result of performing face authentication processing based on the imaging data of the worker W imaged by the supervisor X using the input device 500. is.

As shown in FIG. 2, the imaging data of the worker W needs to be imaged so that the subject's face (the positional relationship of eyes and nose, etc.) can be recognized, for example, so that the feature amount can be derived. However, the subject does not necessarily have to face the front, and may, for example, be slanting (N1) or looking down (W4), or may be wearing a helmet, mask, glasses, or the like.

Workers W1 to W4 in FIG. 2 are workers W who have been successfully collated as a result of the determination processing by the determination unit 140. FIG. In addition, N1 and N2 in FIG. 2 are the workers W who failed in collation. The display unit 520 displays information identifying the worker W (eg, name, affiliation, worker ID, etc.) with a solid line around the face of the worker W to be successfully matched. In addition, the display unit 520 displays information for identifying the worker W (for example, a temporary ID, etc.) by surrounding the face of the worker W who is a target of verification failure with a dashed line. Note that the display unit 520 may express, for example, a solid line surrounding the worker W for whom the matching is successful and a broken line surrounding the worker W for the matching failure with different thicknesses and colors. In addition, the display unit 520 may display, for example, that there are two workers who could not confirm the roll call, and that the workers are W5 and W6. In addition, when the correct feature amount cannot be derived from the face image FN at roll call and the determination process cannot be performed, the display unit 520 includes a message directed to the supervisor X as information identifying the worker W for whom collation is unsuccessful. , a message may be displayed to prompt the user to re-capture the facial image FN during roll call.

For example, if there are unmatched persons that could not be recognized, such as the unmatched persons N1 and N2 shown in FIG. The additional information about the non-matching person is, for example, the assumed age and assumed sex of the subject person derived by the determination unit 140 from the face image FN during roll call. The display unit 520 displays, for example, additional information such as "male in his 50s" for the unmatched person N1 and "male in his 20s" for the unmatched person N2. In addition, when there is a non-matching person that could not be recognized and there is a candidate face image FW that is assumed to be the same person as the non-matching person, the display unit 520 displays the face image FW and the face image FW. Information such as the name of the image FW may be displayed.

Further, when the roll call list includes information on the ages of workers W5 and W6, the determination unit 140 estimates from the additional information which of the workers W5 and W6 is the non-matching person N1, and the estimation result is may be displayed. For example, the determination unit 140 determines that the assumed age of the mismatched person N1 in FIG. If there is a divergence from , it is estimated that there is a high possibility that the unmatched person N1 is the worker W6, and the estimation result is displayed on the display unit 520 as additional information. The determination unit 140 may infer that the unmatched person N2 in FIG. 2 is also W5 from the real age information in the roll call list, like the unmatched person N1.

The supervisor X confirms with the person himself/herself that the dissimilar person N1 is the worker W6 and the dissimilar person N2 is the worker W5 at an arbitrary timing. Supervisor X inputs the confirmation result from the input unit 530 . At that time, the display unit 520 displays a face image FN corresponding to the roll call information being input by the supervisor X, which is cut out only around the non-matching person N1, as shown in FIG. 3, for example. The recognition unit 130 associates the roll-call facial images FN of the non-matching persons N1 and N2 with their affiliations and names in the storage unit 110 based on the input content of the supervisor X received via the communication unit 120. It is stored as a face image FW. Note that the roll-call facial images FN of the workers W1 to W7 may or may not be stored in the storage unit 110 as the facial images FW.

The supervisor X convenes the roll call target again and holds a lunch meeting. The supervisor X also takes an image of part or all of the subject of the roll call with the camera 510 during the lunch meeting, and confirms the roll call. For example, the arithmetic device 100 compares the result of the roll call confirmation with the roll call information of the morning meeting, and finds the workers who were absent at the morning meeting but participated in the afternoon meeting, and the workers who participated in the morning meeting but If there is an absent worker, that person is detected.

In addition, since the arithmetic device 100 can determine the worker W6 who is determined to be the non-matching person N1 at the morning meeting based on the face image FN at the time of the roll call at the time of the morning meeting, there is a low possibility that the worker W6 will be the subject of verification failure. Similarly, the computing device 100 can determine the worker W5, who is determined to be a non-matching worker N2 at the morning meeting, based on the face image FN at the time of the roll call at the time of the morning meeting.

In addition to the morning assembly, for example, the supervisor X explains the outline of the construction site Z when the worker W newly joins the work at the construction site Z, and at the time of acceptance training, etc. When it is desired to keep an attendance record, the information processing system 1 is similarly used to take a roll call.

In the above-described usage scene, even if supervisor X does not finish confirming dissimilar persons N1 and N2 by the start of the lunch meeting, the roll call recording unit 150 uses a temporary ID or the like to identify dissimilar persons. and record the roll call up to the lunch meeting, and when the non-matching persons N1 and N2 are confirmed, the roll call information with the temporary ID is overwritten with the worker ID that identifies the worker, so that the accurate roll call record can be recorded. This is the end of the description of the planned usage scene.

[Classification of Determination by Determination Unit 140]
FIG. 4 is an example of classification of the result of determination processing by the determination unit 140 . When the determination processing by the determination unit 140 is derived from the answers of the AI installed in the determination unit 140, the answers can be classified into four types as shown in FIG.

A correct answer among the answers of the determination unit 140 can be classified as, for example, “1. applicable person” or “2. no applicable person”. "1. There is an applicable person" is, for example, a case where the certainty of the face image FN and the face image FW at roll call is equal to or higher than a threshold. "2. No applicable person" means, for example, that the face image FW of the subject of the face image FN at roll call is stored in the storage unit 110, but the accuracy with the face image FN at roll call is equal to or higher than the threshold. This is the case where there is no such face image FW, or the case where the face image FW of the subject of the roll call face image FN is not stored in the storage unit 110 .

In addition, an incorrect answer among the answers of the determination unit 140 can be classified as, for example, "3. Misidentification as a different person" or "4. Misrecognition as unregistered." "3. Misidentification as a different person" means, for example, erroneously determining that the face image FN during roll call is associated with a different person from the subject of the face image FN during roll call, and that the accuracy is equal to or greater than a threshold. Just in case it gets lost. "4. Misidentification as unregistered" means, for example, that although the face image FW of the subject of the roll call face image FN is stored in the storage unit 110, it is determined that the person is not the same person for reasons such as different systems. This is the case when These erroneous answers are, for example, when the number of face images FW stored in the storage unit 110 is small, when the face images FW or the face images FN at roll call are unclear, or when the administrator Y of the information processing system 1 This can occur when the confidence threshold setting is not suitable.

The erroneous answer by AI shown in FIG. By learning the correction contents of the roll call record, it is gradually resolved. Therefore, in usage scene 1, supervisor X checks the roll call list after completing the registration work for dissimilar workers N1 and N2, and checks whether the results of the judgment of workers W1 to W7 are as expected. It is desirable to verify that incorrect answers are not included.

[System example of storage unit 110]
An example in which the storage unit 110 systematically stores face images FW will be described below with reference to FIG.

FIG. 5 is an example of a system for storing face images FW associated with a single target person for roll call. For example, as shown in FIG. 5, the storage unit 110 classifies the face images FW into groups of matching images into groups A, B, and C based on the presence or absence of equipment around the face of the subject. . System A is a face image FW without any equipment, system B is a face image FW with a helmet on, and system C is a face image FW with a mask on. The storage unit 110 classifies and organizes the face images FW for each system, for example, according to the face direction of the subject person, such as front, oblique left, oblique right, upward, and downward.

When the storage unit 110 classifies the facial images FW into three systems of A system, B system, and C system, the recognition unit 130 selects which system the face image FN at roll call can be classified into the same system. do. Further, when the system of the face image FN at roll call cannot be determined, the recognition unit 130 selects which system is suitable for determination. Recognition section 130 outputs the recognition result to determination section 140 . The determination unit 140 calls the face image FW of the system recognized by the recognition unit 130 from the storage unit 110 and performs determination processing.

Although FIG. 5 shows an example in which one face image FW is associated with each face orientation, a plurality of face images FW may be associated with each face orientation. In this case, for example, the determination unit 140 sequentially selects the face image FW that has been determined to have a high degree of accuracy based on the results of the processing so far, among the plurality of face images FW stored in the storage unit 110, and the facial image FN at the time of roll call. It has a learning function such as performing judgment processing.

[Processing flow]
The flow of processing of the information processing system 1 will be described below with reference to the drawings. 6 and 7 are flowcharts showing an example of the operation of the information processing system 1 according to the embodiment of the invention. FIG. 6 is a flow chart showing an example of the flow of processing from the start of the roll call confirmation to the completion of the confirmation.

The supervisor X takes a picture of the worker W with the camera 510 (step S100). Next, the recognition unit 130 detects the face of the person to be called from the imaging data, and cuts out the face image FN at the time of the call (step S102). Next, the determination unit 140 selects a matching image group of face images FW of a suitable system (step S104). Next, the determination unit 140 collates the face image FW with the roll call face image FN (step S106). Next, the determination unit 140 determines whether or not the derived probability is equal to or greater than a threshold as a result of matching the facial image FW and the facial image FN at roll call (step S108). When determining that the degree of accuracy is equal to or higher than the threshold, the determination unit 140 causes the display unit 520 to display that the target roll-call face image FN is a target of matching success (step S110). The supervisor X determines whether or not the attribute information displayed on the display unit 520 and associated with the target of matching success is correct (step S112). If it is determined that the display is correct, the roll call recorder 150 updates the roll call record based on the result of the supervisor X's determination that the display is correct (step S114). If it is determined that the display is not correct, the process of step S118 is performed.

If it is not determined in the processing of step S108 that the accuracy is equal to or greater than the threshold value, the determination unit 140 causes the display unit 520 to display that the target roll-call face image FN is a matching failure target (step S116). Next, the supervisor X confirms the affiliation and name of the subject of the corresponding face image FN during roll call (step S118). Next, the supervisor X inputs the confirmed result into the input device 500 and instructs the registration in the arithmetic device 100 (step S120). Next, the recognition unit 130 causes the storage unit 110 to register the face image FN at roll call based on the confirmation result of the supervisor X (step S122), and performs the process of step S114.

FIG. 7 is a flow chart showing an example of the flow of processing in which the supervisor X confirms that there is no erroneous recognition after completion of the roll call confirmation. Supervisor X inquires the roll call list (step S200). The supervisor X checks the roll call list and determines whether or not there is erroneous recognition data (step S202). If it is determined that there is erroneous recognition data, the supervisor X compares the roll call record with other information (for example, attribute information obtained by asking the worker W himself) (step S204), and records the roll call list. Update for correction is performed (step S206). This completes the description of the processing of this flowchart.

As described above, the information processing system 1 according to the first embodiment recognizes the facial image FN of the person to be called on the basis of the imaging data including the person to be called, obtained from the input device 500. A face image FW of a person to be called, which is stored in advance in the recognizing unit 130, and a face image FN at the time of call, which is recognized by the recognizing unit, is subjected to face collation processing, so that a person who is scheduled to be a person to be called is identified. By creating a roll call record with the determination unit 140 that determines whether the A high level of roll call record management is possible.

Further, the information processing system 1 according to the first embodiment does not determine that the subject of the face image FN during roll call is not the person expected to be the target person for the roll call. is stored in the storage unit 110, it is possible to reduce the workload of collating the target person from the next roll call.

(Second embodiment)
Next, an information processing system 1A according to the second embodiment will be described. In the following description, portions having functions similar to those described in the first embodiment will be given the same names and reference numerals, and detailed descriptions of the functions will be omitted.

FIG. 8 is a diagram showing an example of the configuration of an information processing system 1A according to the second embodiment. As shown in FIG. 8, an information processing system 1A of the second embodiment differs from the first embodiment in that an arithmetic device 100 includes a verification unit 160, and the verification unit 160 is connected to a verification terminal 200. . Therefore, the verification unit 160 and the verification terminal 200 will be mainly described below.

The verification unit 160 reads one or more roll call records specified by the manager Y from the roll call recording unit 150 and outputs them to the verification terminal 200 . Verification terminal 200 is, for example, a PC (Personal Computer). The verification terminal 200 is used by the manager Y to aggregate roll call record data and verify the consistency thereof.

[Usage scene 2]
An example of a usage scene in which the information processing system 1A is used will be described below. For example, the information processing system 1A is used at a plurality of construction sites. The manager Y uses the verification terminal 200 to output the roll call records of all bases, and information other than the information processing system 1A (for example, the worker list of the worker W and the report of the supervisor X) and , make a match check. The worker list includes, for example, attribute information such as the affiliation and name of the worker W, and information such as when the worker W is scheduled to work at the construction site and until when.

It is expected that it will be difficult for the supervisor X to detect all of the incorrect answers given by the AI at the timing of the morning assembly or the like. Therefore, it is desirable that the administrator Y of the information processing system 1A double-checks whether there is any misrecognition of the call information at any timing, extracts the wrong answer, and corrects the call record. An example of the double-checking method of the roll call record by the manager Y will be described below.

The verification unit 160 , for example, extracts a call record in which there is a possibility of contradiction in the call information, and outputs it to the verification terminal 200 . A contradiction is an erroneous answer by the AI shown in FIG. A contradiction is, for example, that one worker W is associated with roll call records of construction sites at the same time zone on the same day at multiple bases. Manager Y uses the verification terminal 200 to check the roll call record that may contain a contradiction. or who was mistakenly recognized as the worker W. The administrator Y corrects the roll call records so that they are correctly associated with each other based on the specified contents.

When the manager Y finds the contradiction as described above, the face image FW of the worker W stored in the storage unit 110 is mixed with the face image FW of the worker W. It is desirable to request the supervisor X or the worker W himself to confirm whether there is any However, there are many face images FW of the worker W stored in the storage unit 110, and confirmation by the supervisor X or the worker W himself is difficult, or confirmation is not possible because the construction period in which the worker W is involved has already ended. If it is difficult, the storage unit 110 disables all of the stored facial images FW of the worker W, or lowers the value of the accuracy of the erroneously recognized facial images FW. In this case, the storage unit 110 tries to prevent the recurrence of an incorrect answer by the determination unit 140 by adopting the roll-call facial image FN acquired when the worker W participates in the construction work next time as the facial image FW. can be done.

The verification unit 160 also extracts roll call information to be noted and outputs it to the verification terminal 200 . Noteworthy roll call information is roll call information that contains more than a certain percentage of incorrect answers by AI, unlike the above-mentioned roll call records that may have contradictions. be. A roll call that requires attention is, for example, a case where one worker W has roll call records of different bases in the morning and afternoon of the same day. When the verification unit 160 finds the record, for example, it derives the estimated travel time from the location information of the base, determines whether the roll call record is correct (whether it is possible to move between bases), If the roll call record is not determined to be correct, it may be output as information requiring confirmation by manager Y, similar to the discrepancy described above.

In addition, the verification unit 160, based on the job type and skills of the worker W, when the roll call record of the worker W is associated with a construction site that does not particularly require the skills possessed by the worker W, roll call information to be noted can be extracted as The information extracted by the verification unit 160 may be the entire roll call record, or the manager Y may set extraction conditions.

As described above, the information processing system 1A according to the second embodiment has the same effects as those of the first embodiment. Furthermore, it is possible to perform roll call record management with higher accuracy.

Although the embodiments of the present invention have been described in detail above, the specific configurations are not limited to those described above, and various design changes can be made without departing from the gist of the present invention. be.

In the above-described embodiment, an example of performing roll call by face authentication processing is shown. A probability derivation may be performed based on Further, the accuracy may be derived based on the result of recognizing character information (for example, printed company name or name tag) provided on the subject's clothes by OCR (Optical Character Recognition/Reader) or the like.

Further, in the above-described embodiment, the determination unit 140 may use all the face images FW stored in the storage unit 110 as objects to be compared with the facial image FN at roll call, or faces satisfying a predetermined condition. The image FW may be extracted and used. Predetermined conditions are, for example, information on the worker W's place of work and place of residence. For example, when there is a possibility that the worker W will participate in the construction site Z away from the place of work on a business trip, the administrator Y determines that all the faces stored in the storage unit 110 are included in the determination processing by the determination unit 140 Set to use image FW. In addition, when the worker W is unlikely to participate in the construction site Z away from the work place, etc., the manager Y sets a predetermined condition (for example, if the work place etc. The surrounding area of the site Z) is extracted and used.

Note that the determination unit 140 fails to collate the facial image FN during a roll call when a target person to be matched with the facial image FN during a roll call is not found from the extracted facial images FW of the workers W satisfying the predetermined condition. The determination process may be terminated by determining that the face image is the object, or the determination process may be performed again using all the face images FW stored in the storage unit 110 . In addition, the administrator Y may set to determine that the facial image FN during roll call is successfully matched if a target person to be matched with the facial image FN during roll call is found as a result of performing the determination process again. In addition, as a result of performing the determination process again, the manager Y confirms with the supervisor X or the like that it is the worker W himself, just in case a target person to be compared with the face image FN at the time of roll call is found. If it is desired to make the determination, the determination unit 140 is set to determine that the target is a matching failure target. In addition, if manager Y wishes to have supervisor X or the like confirm that it is Worker W himself, he may say, "Mr. XX usually works in XX district. may be set to be displayed on the display unit 520.

Further, in the above-described embodiment, the usage scene in which the supervisor X prepares the roll call record alone has been described. A form may be adopted in which the leader (foreman) prepares a roll call record for each group, and the supervisor X double-checks the roll call record of the foreman.

Further, in the above-described embodiment, an example in which the input device 500 includes the display unit 520 and the input unit 530 is shown, but the computing device 100 may include the display unit and the input unit. In this case, the supervisor X saves the imaging data of the input device 500 in a portable storage medium, reads the imaging data of the storage medium from the input unit of the computing device 100, and performs roll call recording processing.

Note that part or all of the information processing system in the above-described embodiments may be implemented by a computer. In that case, a program for realizing this control function may be recorded in a computer-readable recording medium, and the program recorded in this recording medium may be read into a computer system and executed.

The "computer system" here is a computer system built into the information processing system, and includes hardware such as an OS and peripheral devices. The term "computer-readable recording medium" refers to portable media such as flexible discs, magneto-optical discs, ROMs and CD-ROMs, and storage devices such as hard discs incorporated in computer systems.

Furthermore, "computer-readable recording medium" means a medium that dynamically retains a program for a short period of time, such as a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line. It may also include a volatile memory inside a computer system that serves as a server or client in that case, which holds the program for a certain period of time. Further, the program may be for realizing part of the functions described above, or may be capable of realizing the functions described above in combination with a program already recorded in the computer system.

Further, the arithmetic device 100 in the above-described embodiments may be implemented as an integrated circuit such as LSI (Large Scale Integration). Each functional block of the information processing system may be individually processorized, or part or all may be integrated and processorized. Also, the method of circuit integration is not limited to LSI, but may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integration circuit technology that replaces LSI appears due to advances in semiconductor technology, an integrated circuit based on this technology may be used.

Reference Signs List 1 information processing system 100 arithmetic device 110 storage unit 120 communication unit 122 input content receiving unit 130 recognition unit 140 determination unit 150 roll call recording unit 160 verification unit 200 ... verification terminal, 500 ... input device, 510 ... camera, 520 ... display section, 530 ... input section

Claims (9)

a face image detection unit that detects the face image of the target person for the roll call based on the imaging data including one or more target people for the roll call obtained from the input device;
a storage unit that stores the face image of the roll call target;
The person to be called is scheduled as a person to be called based on the face image of the person to be called stored in the storage unit and the face image detected by the face image detection unit at the time of the call. a determination unit that determines whether the person is a person;
face image registration, in which, when the determining unit does not determine that the person is the person scheduled as the target person for the roll call, the face image of the detected target person for the roll call at the time of the call is stored in the storage unit; and
A judgment result by the judging unit , the face image detected by the face image detecting unit and stored in the storage unit when the person is not judged to be the person expected to be the roll call target. a display unit for displaying the face image of the target person for roll call stored in the storage unit when there is a candidate face image that is assumed to be the same person as the face image of the target person for roll call . prepare
Information processing system. The display unit displays the attribute information of the subject of the imaging data detected by the face image detection unit and the candidate face image when the person is not determined to be the person scheduled as the roll call target. 2. The information processing system according to claim 1, wherein both are displayed. Further comprising a call recording unit that stores the age information of the call target,
When the facial image detection unit detects a plurality of facial images at the time of the roll call, and when the age information of the subject of the roll call is stored in the roll call recording unit, the determination unit determines that the facial image detection unit An assumed age of the person to be called is derived from the face image detected at the time of the call, and whether the assumed age and the age information of the person to be called stored in the call recording unit are similar or divergence is determined. If one said roll call target person and said assumed age are close to each other and if said other said roll call target person and said assumed age diverge, said roll call stored in said roll call recording unit estimating that there is a high possibility that the target person and the one target person at the time of the roll call match,
The information processing system according to claim 1 or 2, wherein the display unit displays the estimation result estimated by the determination unit together with the face image of the person to be called for roll call stored in the storage unit . The determination result by the determination unit is a result determined from the face image corresponding to the position information and the determination timing at which the roll call was performed,
The information processing system further includes a verification unit that verifies that the face image is correctly associated with the roll call target person based on the determination result.
The information processing system according to any one of claims 1 to 3. The storage unit classifies and stores the face image of the person to be called based on a predetermined classification rule.
The information processing system according to any one of claims 1 to 4. The determination unit is
making the determination based on the imaging data corresponding to the classification according to the presence or absence of equipment around the face of the roll call subject;
The information processing system according to claim 5. further comprising an input content reception unit that receives an operation input or voice input of information related to position information and determination timing by the user of the input device,
The face image registration unit stores the face image in the storage unit based on the content input to the input content reception unit.
The information processing system according to any one of claims 1 to 6. the computer
Detecting the face image of the person to be called based on the imaging data including one or more persons to be called, obtained from the input device;
storing the face image of the roll call target;
Based on the stored face image of the person to be called and the detected face image at the time of the call, it is determined whether or not the person to be called is a person scheduled to be the person to be called. ,
storing the face image of the detected roll call target at the time of the roll call when the person is not determined to be the person scheduled as the roll call target;
When it is not determined that the person scheduled to be the roll call target is the same person, and the detected face image at the time of the roll call and the stored face image of the roll call target are the same person. an information processing method comprising: displaying the stored face image of the target person when there is a candidate face image assumed to be absent . to the computer,
Detecting the face image of the target person during the roll call based on the imaging data including one or more target persons obtained from the input device,
storing the face image of the roll call target;
Based on the stored face image of the person to be called and the detected face image at the time of the call, it is determined whether the person to be called is a person scheduled to be the person to be called. ,
storing the face image of the detected roll call target at the time of the roll call when the person is not determined to be the person scheduled as the roll call target;
When it is not determined that the person scheduled to be the roll call target is the same person, and the detected face image at the time of the roll call and the stored face image of the roll call target are the same person. when there is a candidate face image assumed to be absent, displaying the stored face image of the roll call target ;
program.

Images