JP7256364B2 - Information processing device, its control method and program - Google Patents

Description

The present invention relates to an information processing apparatus capable of detecting fraudulent act of spoofing using a photograph during personal authentication as early as possible, its control method, and program technology.

Telework (working from home), which has become popular in recent years, has the problem of the need to guarantee whether or not teleworkers are working and whether or not another person is working.

In order to solve the problem, the face detection frequency is used to estimate whether the person is "present" or "absent" from the image captured by the camera connected to the work PC (for example, the in-camera of the PC), There is a technique for estimating whether a person is the "real person" or "another person" by using the degree of similarity to a person's face that has been registered in advance.
Moreover, it is necessary to determine whether the person in the image is a living person or a pseudo one such as a photograph, in addition to whether the person is the person himself/herself.

Japanese Patent Laid-Open No. 2004-100000 discloses a method for identifying a person to be spoofed while preventing spoofing by judging whether the face of a person to be authenticated is a pseudo face such as a photograph or the like. A technique for a face authentication device, which will be described later, is disclosed, and a technique is disclosed for detecting the presence or absence of movement of facial parts from images of a plurality of frames and performing biometric identification from changes in eye regions.

JP-A-2006-330936

In a system that constantly authenticates the identity of teleworkers while they are working so as not to interfere with their work, there is a problem in determining whether a photo is a photo based on changes in the eye area, that is, whether or not a person blinks. Humans blink their eyes for an average of 0.1 seconds each time, about 20 times per minute. In a use case where all resources can be devoted to anomaly detection, such as surveillance cameras, it is possible to quickly determine whether a photograph is a photograph by checking the presence or absence of blinks at a frequency shorter than the blink time per blink. . However, when performing behind-the-scenes work of teleworkers on their PCs, it is necessary to check the presence or absence of blinking with low resources, that is, at a low frame rate, so as not to hinder the work. Since the probability of capturing the moment of blinking is reduced by confirming the blinking of each time, it is necessary to take time to determine the presence or absence of blinking. For example, when checking only once per second, even if no blink is detected for one minute, the probability of not blinking is 87%, and the possibility of blinking remains with a probability of 13%. There is a need for a system that can quickly determine the quality of a photo while avoiding erroneous determination that it is a photograph.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a scheme capable of detecting fraudulent act of spoofing using a photograph during personal authentication as quickly as possible without consuming PC resources of a PC operator.

The present invention is based on calculation means for calculating the likelihood of a plurality of states set in advance based on a photographed image, and accumulation of detection results of movement of a person's face region detected from the photographed image. , the current state specified based on the control means for controlling the probability of transition to another state, the likelihood calculated by the calculation means, and the transition probability controlled by the control means is the predetermined state. and notification means for notifying based on a predetermined state in the case of an information processing apparatus.

According to the present invention, in a system for constantly authenticating whether a PC operator is the person himself/herself, it is possible to detect unauthorized acts such as spoofing using photographs during personal authentication as soon as possible without interfering with the worker's original work. can be provided.

1 is a system configuration diagram showing an example of the configuration of a telework management system of the present invention; FIG. 2 is a block diagram showing the hardware configuration of an information processing device applicable to the trail transmission terminal 100, the trail inspection terminal 110, and the trail management server 120 shown in FIG. 1; FIG. 2 is a block diagram showing a functional configuration necessary for the trail sending terminal 100, the trail auditing terminal 110, and the trail management server 120 shown in FIG. 1; FIG. FIG. 4 is an image showing the relationship between the transition probability, the estimated value, the observed value, and the output transition probability used in photo spoofing estimation using the hidden Markov model of FIG. 9 is a flow chart showing a process for outputting a determination result of the current user's state; 10 is a flowchart showing face tracking success/failure determination processing; 6 is a flowchart showing face selection processing; 10 is a flow chart showing photo spoofing estimation using Hidden Markov Models. An image diagram showing the relationship between an acquired camera image, each determination method, and the user state estimated from the determination result. Focus frame with continuous time User status displayed and updated in real time on the trail sending terminal 100 An example of an audit screen

First, the outline of the present invention will be explained.

The telework management system of the present invention uses face recognition technology to detect events of teleworkers (labor events such as "presence" and "leaving", security incidents such as "spoofing" and "peeping" by a third party).・It is a system that records and allows administrators to view.

The above is the outline of the telework management system according to the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a system configuration diagram showing an example of the configuration of the telework management system of the present invention.

1, one or more trail sending terminals 100, one or more trail auditing terminals 110, and one or more trail management servers 120 are connected via a local area network (LAN) 130, a router 140, and the Internet 150. It is configured to be

The trail transmission terminal 100 detects labor events and security incidents of teleworkers to be used, and transmits the trail to the trail management server 120 . Also, the teleworker's feature amount data (face image and feature amount obtained therefrom) used for event detection is transmitted to the trail management server 120 .

The trail audit terminal 110 performs an operation of approving the feature amount data recorded in the trail management server 120 and an audit operation of the trail recorded in the trail management server 120 .

The trail management server 120 records the trail received from the trail sending terminal 100 and processes audit operations of the trail auditing terminal 110 for the trail. Also, it records the feature amount data received from the trail sending terminal 100, and processes the approval operation of the trail inspection terminal 110 for the feature amount data.

A hardware configuration of an information processing apparatus applicable to the trail sending terminal 100, the trail auditing terminal 110, and the trail management server 120 shown in FIG. 1 will be described below with reference to FIG.

FIG. 2 is a block diagram showing the hardware configuration of an information processing device applicable to the trail sending terminal 100, trail auditing terminal 110, and trail management server 120 shown in FIG.

In FIG. 2, reference numeral 201 denotes a CPU that controls devices and controllers connected to a system bus 204 in an integrated manner. The ROM 203 or the external memory 212 stores the BIOS (Basic Input/Output System), which is the control program for the CPU 201, an operating system program (hereinafter referred to as OS), and the functions executed by each server or each PC. Various programs, etc., which will be described later, are stored.

A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . The CPU 201 loads programs and the like necessary for executing processing from the ROM 203 or the external memory 212 to the RAM 202, and executes the loaded programs to realize various operations.

An input controller 205 controls input from a keyboard (KB) 209, a camera device 210 (imaging device), and a pointing device such as a mouse (not shown).
A video controller 206 controls display on a display device such as a CRT display (CRT) 211 or the like. Although the CRT 211 is shown in FIG. 2, the display may be not only the CRT but also other display such as a liquid crystal display. These are for use by administrators as needed.

A memory controller 207 stores a boot program, various applications, font data, user files, edit files, various data, and the like in an external storage device (hard disk (HD)), a flexible disk (FD), or a PCMCIA card slot. It controls access to an external memory 212 such as Compact Flash (registered trademark) memory connected via an adapter.

A communication I/F controller 208 connects and communicates with an external device via a network (for example, the LAN 130 shown in FIG. 1), and executes communication control processing in the network. For example, communication using TCP/IP is possible.

It should be noted that the CPU 201 enables display on the CRT 211 by, for example, rasterizing an outline font to a display information area in the RAM 202 . The CPU 201 also allows the user to issue instructions with a mouse cursor (not shown) on the CRT 211 .

Various programs described later for realizing the present invention are recorded in the external memory 212 and are executed by the CPU 201 by being loaded into the RAM 202 as necessary. Furthermore, definition files and various information tables used when executing the above programs are also stored in the external memory 212, and detailed descriptions thereof will be given later.

Next, a functional block diagram of the trail sending terminal 100, trail auditing terminal 110, and trail management server 120 of the present invention will be described with reference to FIG.

Note that detailed control performed by each functional block will be described later with reference to a flowchart.

First, the functional configuration of the trail transmitting terminal 100 will be described.

The video input unit 101 acquires video from the camera device 210 and transmits the video to the feature amount data registration unit 102 and the event detection unit 106 .

The feature amount data registration unit 102 supplies the frame (still image) of the image sent from the image input unit 101 to the face recognition unit 103, and stores the feature amount data obtained from the face recognition unit 103 in the feature amount data storage unit. 104.

The face recognition unit 103 performs face detection on the frames sent from the feature amount data registration unit 102 or the event detection unit 106 . Also, feature amount data is calculated for the detected face. Furthermore, when the feature amount data to be compared is received, the degree of similarity between the two feature amount data is calculated.
The feature amount data storage unit 104 stores feature amount data consisting of a face image and its feature amount.

The feature data transmission unit 105 transmits the feature data sent from the feature data registration unit 102 to the feature data reception unit 121 of the trace management server 120 via the communication I/F controller 208 .

The event detection unit 106 provides the frame of the video obtained from the video input unit 101 and the feature amount data obtained from the feature amount data storage unit 104 to the face recognition unit 103, and the face detection/identification result obtained from the face recognition unit 103. Based on this, events such as presence, absence, spoofing, and peeping are detected. A trace of the detected event is transmitted to the event control unit 107 .

The event control unit 107 stores the trail obtained from the event detection unit 106 in the trail storage unit 108 . When a specific transmission condition is satisfied (for example, 3 minutes have passed since the leaving event occurred), the trail recorded in the trail storage unit 108 is transmitted to the trail transmission unit 109 . Further, any trail that has been transmitted or no longer needs to be transmitted (for example, a presence event occurred within 3 minutes after an leaving event occurred) is erased from the trail storage unit 108 . Also, if necessary, control corresponding to the event (such as locking the screen when a spoofing event occurs) is performed.

The trail storage unit 108 stores event trails.

The trail sending unit 109 sends the trail obtained from the event control unit 107 to the trail receiving unit 125 of the trail management server 120 via the communication I/F controller 208 .

The feature amount data update unit 10A receives the result of the feature amount data management operation processing unit 124 of the trail management server 120 and updates the feature amount data in the feature amount data storage unit 104 .

Next, the functional configuration of the trace inspection terminal 110 will be described.

The feature amount data management operation unit 111, via the feature amount data management operation processing unit 124 of the trail management server 120, browses the feature amount data, and checks the approval status (waiting for approval, approval, denial, rejection) and identifier. Relevance (identification ID) is updated.

The trail audit operation unit 112 browses event trails and labor situation statistical information derived from the events via the trail audit operation processing unit 127 of the trail management server 120 .

The update notification reception unit 113 receives update notifications from the update notification transmission unit of the trail management server 120 . Finally, the functional configuration of the trail management server 120 will be described.

The feature amount data receiving unit 121 receives the feature amount data sent from the feature amount data transmitting unit 105 of the trail transmitting terminal 100 and stores the feature amount data in the feature amount data storage unit 122 . It also requests the update notification transmission unit 123 to notify that new feature amount data has been registered.

The feature amount data storage unit 122 stores the feature amount data received by the feature amount data reception unit 121 . It also stores identifier information associated with the identification ID of the feature amount data.

The update notification transmission unit 123 transmits an update notification to the update notification reception unit 113 of the trace inspection terminal 110 when the feature data and the trail are received.

The feature amount data management operation receiving unit 124 receives a command from the feature amount data management operation unit 111 of the trace inspection terminal 110, displays the feature amount data in the feature amount data storage unit 122 on the screen, or performs an operation on the feature amount data. process operations.

The trail receiving unit 125 receives the trail transmitted from the trail transmitting unit 109 of the trail transmitting terminal 100 and stores it in the trail storing unit 126 . If the trail event is "spoofing" or "peeping", it requests the update notification transmitting unit 123 to notify that the security incident trail has been registered.

The trail storage unit 126 stores trails of labor events such as "presence" and "absence" and security incidents such as "spoofing" and "peeping".

The trail audit operation processing unit 127 receives commands from the trail audit operation unit 112 of the trail audit terminal 110, displays the trail in the trail storage unit 126 and labor statistical information calculated from the trail on the screen, and performs operations on the trail. to process.

The account storage unit 128 stores account information of teleworkers and administrators.

It also has a function to perform account authentication necessary for teleworkers and administrators to use the system, and a function to register their account information, but detailed explanation is omitted as it is out of the scope of the present invention. .

Hereinafter, a flowchart showing the processing for outputting the determination result of the current user status (present, away, peeping, spoofing others, spoofing photos) will be described. In the embodiment, the hidden Markov model is used for the determination of the state, but other probability models or other methods may be used for the determination.

Description will be made from FIG.

The processing of each step in FIG. 5 is executed by the CPU of the trace transmission terminal (telework user terminal).

In S501, the current state to be output is set to unknown. Set the face tracking state to FALSE. Set the state probability reset flags of all hidden Markov models to TRUE.

In S502, it is checked whether or not there is a process end event for outputting the current state. If there is an end event (TRUE), the process ends. If there is no end event (FALSE), the process proceeds to S503.

In S503, a camera image (still image) is acquired for personal authentication.

In S505, it is confirmed whether or not the user is in a face tracking state (a state in which even if a low degree of similarity is detected by tracking the authenticated face, it is difficult to transition from the presence state to the impersonation state). If FALSE (not in face tracking state), proceed to S509; if TRUE (in face tracking state), proceed to S506.

In S506, face tracking success/failure determination processing, the details of which are shown in FIG. 6, is performed.

FIG. 6 will be explained.

In S601, the tracking area of the current camera image is calculated from the tracking area of the previous camera image. (using a tracking algorithm such as MeanShift)
In S602, the difference between the histograms of the tracking area of the previous camera image and the tracking area of the current camera image is calculated.

In S603, it is determined whether or not the calculated difference is below a predetermined threshold. If it falls below, the process proceeds to S604. If not, the process proceeds to S608.

In S604, it is determined whether or not the ratio threshold of the skin color area, which is regarded as tracking success, is 0 or more. If it is 0 or more, the process proceeds to S605. If it is 0, the process proceeds to S607.

In S605, the proportion of the skin color area in the tracking area of the current camera image is calculated.

In S606, it is determined whether or not the calculated ratio of skin-colored areas in the tracking area of the current camera image exceeds a skin-colored area ratio threshold of 0 or more, which is regarded as successful tracking. If it exceeds, the process proceeds to S607. If not, the process proceeds to S608.

In S607, it is determined that face tracking has succeeded. Also, the face tracking area is updated to the tracking area of the current camera image.

In S608, it is determined that face tracking has failed.

Returning to the description of FIG.

In S507, as a result of the face tracking success/failure determination processing in FIG. 6, it is determined whether or not the face tracking has succeeded. If face tracking is successful, the process proceeds to S509. If face tracking fails, the process proceeds to S508.

In S508, face tracking is stopped and the face tracking state is set to FALSE. Set the state probability reset flags of all hidden Markov models to TRUE.

In S509, a face is detected from the current camera image. It calculates not only the position, size, and angle of the face, but also the extent to which the eyes are closed (eye-closed score).

In S510, it is determined whether or not there remains a face that has not yet been identified (personally authenticated) among the faces detected in S509. If there is a face that has not yet been identified (personally authenticated) (TRUE), the process proceeds to S511. If not (FALSE), proceed to S512.

In S511, the degree of similarity of the detected face is calculated using (the feature amount data of) the registered face image. There may be multiple registered face images, in which case multiple similarities are calculated. After that, the process returns to S510.

In S512, face selection processing, the details of which are shown in FIG. 7, is performed. This process may be performed before S510.

FIG. 7 will be explained.

In S701, the sorted face list is made into an empty array.

In S702, the faces detected in S509 are sorted in descending order of size (face area).

In S703, it is determined whether or not there are still unselected faces among the detected faces sorted in descending order of size in S702. If it remains (TRUE), the process proceeds to S704. If not, the process ends.

In S704, it is determined whether or not the size of the detected face is equal to or less than a threshold. If it is equal to or less than the threshold, the process proceeds to S707. If it is not equal to or less than the threshold, the process proceeds to S705.

In S705, it is determined whether or not the selected face is included in any estimated body region. An estimated body region is a trapezoidal or parabolic region estimated to be located on the body, which should be under the face. If it is included, the process proceeds to S707. If not included, the process proceeds to S706.

In S706, the selected face is added to the selected face list. After that, the process returns to S703. In S707, the detected face is regarded as unsorted and is not added to the sorted face list. After that, the process returns to S703.

Returning to the description of FIG.

In S513, the face selected in S512 is used as an input to estimate whether or not it is peeping (hidden Markov model). Using the processing result of FIG. 7, an observed value is obtained as to whether the number of selected faces (detected faces) is two or more or one or less. Next, according to the obtained observed values, the probabilities of each state of the estimated values (peeping and non-peeping) are calculated, and the plausible estimated values are selected.

In S514, subsequent processing is switched according to the estimation result of S513. If it is peeping, the process proceeds to S515, and if it is not peeping (other than that), the process proceeds to S522.

At S515, the internal state is set to "peeping".

At S516, it is determined whether or not the currently displayed output state matches the internal state (peeping) set at S515. If they match (TRUE), the process proceeds to S521. If they do not match (FALSE), the process proceeds to S518.

In S522, the face selected in S512 is used as an input to estimate (hidden Markov model) whether the person is present, away from the seat, or impersonating another person. The number of selected faces (detected) is 0, and the feature of one selected face (detected) is the person himself (the similarity exceeds the threshold, that is, the registered face image ), and the feature of one selected face (detected) is that of another person (the similarity is below the threshold, that is, it is not similar to the registered face image). demand. When there are a plurality of registered face images and the degree of similarity is calculated for each, determination may be made using only the maximum value, or determination may be made using another calculation method such as an average value or a median value. Next, according to the obtained observed values, the probabilities of each state of the estimated values (presence, impersonation, and absent) are calculated, and a plausible estimated value is selected.

In S523, subsequent processing is switched according to the estimation result of S522. If the person is present, the process proceeds to S524, and if the person is impersonating another person, the process proceeds to S534. If the user is away from the seat, the process proceeds to S535.

At S534, the internal state is set to "impersonation of another person".

At S535, the internal state is set to "leave".

In S536, the face tracking state is confirmed. If not in the tracking state (FALSE), proceed to S537; if in the tracking state (TRUE), proceed to S538.

In S537, it is determined whether or not the currently displayed output state matches the current internal state of "impersonating another person or away from seat". If they match (TRUE), the process proceeds to S521. If they do not match (FALSE), the process proceeds to S520.

In S538, it is determined whether or not a certain period of time has elapsed since the person was identified (determined whether or not the face image resembles a registered face image) at the time of the previous tracking. If it has passed (TRUE), the process proceeds to S539. If it has not elapsed (FALSE), the process proceeds to S540.

In S539, in order to determine whether or not to continue face tracking, it is determined whether the detected face is the person himself/herself or another person based on whether the degree of similarity is below a threshold. The threshold used in this determination is set lower than the threshold used in S529, so that face tracking is continued unless it is clearly determined that the person is someone else. If it falls below (TRUE), the process proceeds to S519. If not (NO), the process proceeds to S540.

In S540, it is determined whether or not a certain period of time has passed since the last tracked face area correction. If it has passed (TRUE), the process proceeds to S541. If not (FALSE), the process proceeds to S521.

In S524, a motion score is calculated from the detected face position, face size, face angle, and the like. In addition, the calculated motion score and eye-closed score are temporarily stored for a plurality of continuous camera images.

In S525, presence/absence information of an observed value (blinking, moving, or not moving) is determined from the motion scores and eye-closed scores of a plurality of temporarily held camera images. For example, if the difference between the maximum value and the minimum value of the eye closing score is equal to or greater than a threshold, it is assumed that the user has blinked. For example, if the maximum and minimum motion scores are the same, then there is no motion. For example, if it cannot be said that the object blinks or does not move, it is assumed that the object is moving.

In S526, photo spoofing estimation is performed using a hidden Markov model whose details are shown in FIG. As described above, photo spoofing may be estimated by using a probability model other than the hidden Markov model or by using another method.

FIG. 8 will be explained.

In S801, a reset flag for resetting each estimated value of the state probability (probability of being present, probability of photo spoofing (handheld), probability of photo spoofing (fixed)) to an initial value (initial probability) is set. Determine presence/absence. If there is a reset flag (TRUE), the process proceeds to S805, and if there is no reset flag (FALSE), the process proceeds to S802.

In S805, the state probabilities (presence probability, photo spoofing (handheld) probability, photo spoofing (fixed) probability), which are estimated values, are reset to initial values (initial probabilities).

In S806, a reset flag is set to reset each estimated state probability (probability of being present, probability of photo spoofing (handheld), probability of photo spoofing (fixed)) to an initial value (initial probability). change to nothing.

In S802, from the observed value (blinking, motion, no motion), the state transition probability, and the output transition probability, the current estimated value of each state (should be present, photo spoofing (hand-held) Compute the probabilistic likelihood of photo spoofing (should be fixed).

In other words, according to the accumulated detection frequency of the observation results (blink detected, motion detected, motion not detected), the current estimated value of each state (should be present. Photo spoofing) There should be (handheld), photo spoofing (fixed)) probability likelihood will gradually change.

In S803, based on the calculated probability likelihood, the probability of each estimated state (probability of being present, probability of photo spoofing (handheld), probability of photo spoofing (fixed)) is temporarily updated. do.

In S804, the estimated value of the maximum probability is output among the probabilities of states that are temporarily updated estimated values. In some cases, in order to prevent state changes from occurring frequently, if the probability of the previously output state does not fall below a certain condition, or if the maximum probability does not exceed a certain condition, the previously output state will continue. You can also output it as

FIG. 4 will be explained.

FIG. 4 is an image showing the relationship between the transition probability, the estimated value, the observed value, and the output transition probability used in photo spoofing estimation using the hidden Markov model of FIG. Note that the described values such as the transition probability are only examples, and other values may be used according to the required performance.

The estimated value is a possible value of the state to be estimated, and here, there are three states of presence, photo spoofing (handheld), and photo spoofing (fixed).

Observed values are values that can be observed as data, and here are three observed states (blinking, motion, no motion) calculated from a plurality of continuous camera images.

A transition probability is the probability that an estimate is assumed to switch to another estimate. Here, we assume that there are few malicious users who attempt photo spoofing, and that the probability of transitioning from being seated, photo spoofing (handheld), or photo spoofing (fixed) to another state is quite low. there is

The output probability is the ratio of observed values assumed to be observed when the estimated values are determined. The probability of detecting blinking is low when the person is present (because the person blinks only for about 2 to 3 seconds per minute), but it is assumed that movement can be observed with a high probability. In the case of photo spoofing (hand-held), almost no blinking can be observed, and in the case of photo spoofing (fixed), it is assumed that almost no movement is observed.

In S527, as a result of the photograph spoofing estimation in S526, it is a step to distinguish whether the current internal state is presence or photo spoofing. If it is estimated that the person is present, the process advances to S529;

At S528, the internal state is set to "photo impersonation".

In S518, the face tracking state is checked. If it is determined that the face is not in the tracking state (FALSE), the process proceeds to S520, and if it is determined that the face is in the tracking state (TRUE), the process proceeds to S519.

In S519, face tracking is stopped and the face tracking state is set to FALSE (no). Furthermore, the state probability reset flags of all hidden Markov models are set to TRUE.

In S520, the current state to be output (presence, photo spoofing, other person's spoofing, away from seat, peeping) is converted to the current internal state (presence, photo spoofing, other person's spoofing). leaving the seat, peeping).

In S521, the display contents are updated to the current state to be output (presence, photo spoofing, other person's spoofing, away from seat, peeping). After that, the process returns to S502 to repeat the processing up to this point, and the current state to be constantly output (presence, photo spoofing, other person's spoofing, leaving seat, peeping) is repeatedly determined.

At S529, the internal state is set to "presence".

In S530, it is checked whether the face tracking state is set. If not in the tracking state (FALSE), proceed to S531; if in the tracking state (TRUE), proceed to S538.

In S531, it is determined whether or not the currently displayed output state matches the current internal state of "present". If they match (TRUE), the process proceeds to S532. If they do not match (FALSE), the process proceeds to S520.

In S532, it is determined whether the maximum similarity of the current face is equal to or greater than a threshold. This threshold value is set to a value higher than the threshold value used in determining whether the person is the real person in S522, and the face tracking is started only when the confirmation of the person's identity is considerably high. If it is equal to or greater than the threshold (TRUE), the process proceeds to S533. If it is not equal to or greater than the threshold (FALSE), the process proceeds to S521.

In S533, face tracking is started. The detected face area is set as a face tracking area, and the face tracking state is set to TRUE.

In S541, the area to be tracked as a face is corrected. The tracking area of the current camera image is replaced with the detected face area instead of the area determined in S506.

FIG. 9 is an image diagram showing the relationship between an acquired camera image, each determination method, and the user state estimated from the determination result.

FIG. 10 is an image diagram showing the relationship between the movement of the user's face within the face region and the user state estimated from the detection result of whether or not the user is blinking within the face region when detecting a face region with continuous time. is.

In FIG. 10, the solid line frame surrounding the face area indicates the presence state, and the dotted line frame surrounding the face area (photograph) indicates the spoofing state.

FIG. 11 shows the user's status displayed and updated in real time on the trace transmission terminal 100, and shows an example of a display indicating that the user is present and an example of a display indicating that the user is impersonating (photograph).

FIG. 12 shows that when the trail sending terminal 100 detects a (photograph) spoofing state, the trail data (for example, a driver's license) acquired by the trail sending terminal 100, the fraudulent state (detected content), and the time are displayed in the trail. An example of an inspection screen displayed on the inspection terminal 110 is shown.

It goes without saying that the configuration and content of the various data described above are not limited to this, and may be configured in various configurations and content according to the application and purpose.

Further, the program in the present invention is a program that causes a computer to execute the processes shown in FIGS. 5 to 8. FIG. The program in the present invention may be a program for each process shown in FIGS. 5 to 8. FIG.

As described above, a recording medium recording a program for realizing the functions of the above-described embodiments is supplied to a system or device, and the computer (or CPU or MPU) of the system or device reads the program stored in the recording medium. Needless to say, the object of the present invention can also be achieved by reading and executing.

In this case, the program itself read from the recording medium implements the novel functions of the present invention, and the recording medium recording the program constitutes the present invention.

Examples of recording media for supplying programs include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, magnetic tapes, non-volatile memory cards, ROMs, EEPROMs, silicon A disk or the like can be used.

Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also based on the instructions of the program, the OS (operating system) and the like running on the computer are actually executed. Needless to say, a case where part or all of the processing is performed and the functions of the above-described embodiments are realized by the processing are included.

Furthermore, after the program read from the recording medium is written in the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is read according to the instruction of the program code. It goes without saying that a case where a CPU or the like provided in a function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

Moreover, the present invention may be applied to a system composed of a plurality of devices or to an apparatus composed of a single device. Further, it goes without saying that the present invention can be applied to the case where it is achieved by supplying a program to a system or device. In this case, by loading a recording medium storing a program for achieving the present invention into the system or device, the system or device can enjoy the effects of the present invention.

Furthermore, by downloading and reading out the program for achieving the present invention from a server, database, etc. on the network using a communication program, the system or device can enjoy the effects of the present invention. It should be noted that all configurations obtained by combining each of the above-described embodiments and modifications thereof are also included in the present invention.

100 Trace transmission terminal (telework user terminal)
110 trail management server (telework management device)
120 Trace audit terminal (administrator terminal)

Claims (6)

a calculation means for calculating the likelihood of a plurality of states set in advance based on the photographed image;
a control means for controlling the probability of transition to another state based on accumulation of detection results of movement of the person's face area detected from the photographed image;
If the current state specified based on the likelihood calculated by the calculation means and the transition probability controlled by the control means is a predetermined state, notification means for notifying that the current state is a predetermined state. and,
An information processing device comprising: The notification means, when a value based on the likelihood calculated by the calculation means and the transition probability controlled by the control means satisfies a first condition, in another state to which the current state has transitioned. 2. The information processing apparatus according to claim 1 , wherein notification is made based on a certain event. The notification means, when the value based on the likelihood calculated by the calculation means and the transition probability controlled by the control means satisfies a second condition, the current state continues to be a predetermined state. 3. The information processing apparatus according to claim 1, wherein the notification is made based on the fact that. 4. The information according to any one of claims 1 to 3 , wherein the predetermined state is a state in which the person in the image is the real person, and the other state is a state in which the person in the image is an impersonator. processing equipment. a calculation step in which the calculation means of the information processing device calculates the likelihood of a plurality of preset states based on the photographed image;
a control step in which the control means of the information processing apparatus controls the probability of transition to another state based on the accumulation of detection results of the movement of the person's face area detected from the photographed image;
If the current state specified based on the likelihood calculated in the calculation and the transition probability controlled in the control step is a predetermined state, the notification means of the information processing device is in the predetermined state a notification step of making a notification based on
A control method for an information processing device, comprising: information processing equipment,
a calculation means for calculating the likelihood of a plurality of states set in advance based on the photographed image;
a control means for controlling the probability of transition to another state based on accumulation of detection results of movement of the person's face area detected from the photographed image;
If the current state specified based on the likelihood calculated by the calculation means and the transition probability controlled by the control means is a predetermined state, notification means for notifying that the current state is a predetermined state. and,
program to function as

Images