JP2018092287A - Device, method and program for personal identification - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a personal identification technique by which personal identification is possible for a period of timer longer than that in conventional techniques.SOLUTION: A personal identification device comprises: a personal image acquisition unit 1 which acquires a time series of images of an identification object person; a facial recognition unit 2 which uses the acquired time series of images of the identification object person and a preliminarily registered image of the person to determine whether the identification object person is the same person as the person of which the image is preliminarily registered or not by facial recognition technology; a tracking unit 3 which tracks the identification object person determined as the same person, on the basis of the time series of images of the identification object person; a motion estimation unit 4 which, in the case of a failure in tracking, estimates a motion of an image region included in the images of the identification object person on the basis of the time series of images of the identification object person; and a personal identification unit 5 which determines whether the identification object person has been changed to another person or not on the basis of the estimated motion of the image region and, if not determining that the identification object person has been changed, causes the tracking unit to restart tracking.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a technique for identifying a remote person using techniques such as a face authentication technique and a tracking technique.

  When conducting examinations, etc. on the Internet, it is difficult to supervise the test locally, unlike the venue test, so it is illegal for the test supervisor to remotely monitor the camera image using the camera of the examinee's terminal. It is conceivable to prevent this. Since the number of people who can be supervised at a time is limited in the monitoring of camera images by human visual inspection, the supervisor is identified as non-regular if it can be identified automatically by applying face recognition technology etc. It is possible to focus attention only on those who have taken the test, and the supervisory work will be made more efficient.

  The face authentication technique is a technique for comparing the feature amount of a face image registered in advance with the feature amount of the face image of the person to be authenticated at the time of authentication to confirm the authenticity of the person to be authenticated (for example, non-patent) Reference 1). The accuracy of face authentication is very high in the adjusted environment, and on the other hand, it has a characteristic that it easily deteriorates due to the influence of lighting conditions, face orientation, facial expressions, shielding objects, and the like.

  Therefore, when face authentication technology is applied to a remote test, the test subject can be identified with high accuracy in situations where the test subject is facing the front and the entire face is shown, but the test subject is facing down and with a cheek cane. In a situation where a part of the face is hidden, face authentication is likely to fail, and the period during which the examinee can be correctly identified is limited. Therefore, it is conceivable to improve the period in which the examinee can be identified by combining the face authentication technique and the tracking technique.

  The tracking technique is a technique for tracking a target person or object by searching for a similar image region from images given in time series using the feature amount (see, for example, Non-Patent Document 2). .) Since it is generally considered that the appearance of the tracked object changes from moment to moment, a tracking technology that is robust against changes in the appearance has been proposed. Even if tracking fails in a certain image, tracking can be resumed if a similar image area is found in subsequent images.

  In the combination of the face authentication technique and the tracking technique in the remote test, when face authentication fails, the examinee is tracked starting from the image that succeeded in face authentication last. Thereby, in addition to the period when face authentication was successful, it becomes possible to identify the examinee during the period when tracking was successful.

Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato and Lior Wolf, “DeepFace: Closing the Gap to Human-Level Performance in Face Verification”, Conference on Computer Vision and Pattern Recognition (CVPR) June 24, 2014. Zdenek Kalal, Krystian Mikolajczyk and Jiri Matas, “Tracking-Learning-Detection”, IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, VOL. 6, NO. 1, JANUARY 2010.

  However, with the technology that combines the face authentication technology and tracking technology, if the examinee temporarily turns sideways, and the appearance of the face is largely hidden by holding the eyes, the examinee's tracking will occur. May fail.

  When tracking is stopped after tracking has failed once, the problem is that the examinee cannot be identified until the next successful face authentication.

  Therefore, an object of the present invention is to provide a person identification device, method, and program that can identify a person such as a test taker longer than before.

  A person identification device according to an aspect of the present invention is pre-registered with a person image acquisition unit that acquires a time series of images of an identification target person that is a person to be identified, and a time series of acquired images of the identification target person. Based on the time series of the image of the identification target person, and a face authentication unit that determines whether or not the identification target person and the person registered in advance are the same person The tracking unit that tracks the identification target person determined to be the same person, and the movement of the image area included in the identification target person image based on the time series of the identification target person image when tracking fails. Based on the motion estimation unit to be estimated and the estimated motion of the image area, it is determined whether or not the person to be identified has been replaced by another person. If it is not determined that the person has been replaced, It includes a personal identification unit to resume tracking King unit.

  The time during which a person can be identified becomes longer than before.

The block diagram for demonstrating a person identification device. The flowchart for demonstrating a person identification method. The figure which shows an example of the hardware constitutions of a person identification device.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  As shown in FIG. 1, the person identification device includes, for example, a person image acquisition unit 1, a face authentication unit 2, a tracking unit 3, a motion estimation unit 4, and a person identification unit 5. The person identification method is realized by each unit of the person identification apparatus performing the processing from step S1 to step S5 described below with reference to FIG.

<Human Image Acquisition Unit 1>
The person image acquisition unit 1 acquires a time series of images of a person to be identified who is a person to be identified (step S1). The acquired time series of the images of the identification target person is output to the face authentication unit 2, the tracking unit 3, and the motion estimation unit 4.

  The person image acquisition unit 1 acquires a time series of images of identification target persons by acquiring images of identification target persons at predetermined time intervals.

  An example of the person to be identified is a student taking a test at a remote place using a terminal device such as a PC. In this case, an image of an examinee who is a person to be identified is photographed by a camera provided in a terminal device such as a PC, and the photographed image is transferred to the person image acquisition unit 1 of the person identifying device via a communication network such as the Internet. Sent. For example, the person image acquisition unit 1 acquires the image of the person to be identified in this way.

  The person to be identified may be a person other than the examinee as long as an image of the person can be acquired.

<Face authentication unit 2>
The face authentication unit 2 uses the time series of the images of the identification target person acquired by the person image acquisition unit 1 and the image of the person registered in advance, and the identification target person and the person registered in advance are the same person. It is determined by face authentication technology whether it exists (step S2). The determination result is output to the tracking unit 3.

  The face authentication technique is a technique for comparing the feature amount of a face image registered in advance with the feature amount of the face image of the person to be authenticated at the time of authentication to confirm the authenticity of the person to be authenticated. As the face authentication technique, for example, an existing face authentication technique such as the technique described in Non-Patent Document 1 can be used.

  When the face authentication unit 2 succeeds in face authentication, in other words, when it is determined that the person to be identified and the person registered in advance are the same person, the person to be identified who has succeeded in the face authentication. Information about the image area is output to the tracking unit 3.

<Tracking unit 3>
The tracking unit 3 tracks the identification target person determined by the face authentication unit 2 to be the same person as the person registered in advance in the face authentication unit 2 based on the time series of the images of the identification target person (step) S3). In other words, the tracking unit performs tracking on the image area of the person to be identified who has succeeded in face authentication in the face authentication unit 2. The tracking result is output to the motion estimation unit 4.

  Tracking is to track a target person or object by searching for similar image regions from images given in time series using the feature amount. Tracking can be performed by an existing tracking technique such as the technique described in Non-Patent Document 2, for example.

  When tracking fails, the tracking unit 3 stores information about an image region that has been a tracking target in an image immediately before the image that failed tracking.

  The tracking unit 3 may determine that tracking has failed if the moving distance is equal to or greater than the moving distance between the position of the image area of the previous image and the position of the image area of the current image. . As a result, the person identifying unit 5 determines that the person is a regular person only when the movement distance of the person to be identified is within a certain range, and can identify the person under more severe conditions.

<Motion estimation unit 4>
When the tracking unit 3 fails to track the identification target person, the motion estimation unit 4 estimates the movement of the image area included in the identification target person image based on the time series of the identification target person image (step S <b> 3). S4). Information about the estimated movement of the image area is output to the person identification unit 5.

  The image area is an arbitrary image area included in the image of the person to be identified. For example, when looking at the time series of images, if there are moving objects that move in the image, moving objects that enter from outside the image, and moving objects that move out of the image, Each image portion of these moving bodies is an image region that is a motion estimation target. An example of the image region is an image of a person to be identified.

  An existing motion estimation technique can be used for the motion estimation. The motion estimation technique is a technique for estimating the direction and distance in which an image area has moved using a difference between two temporally adjacent images in a time-series image. This moving direction and distance is called a motion vector.

  The motion estimation unit 4 calculates the motion vector using the time series of the images of the person to be identified, and thereby the image region that enters from the outside of the image and the image that goes out of the image It is estimated whether there was at least one of the areas. Whether or not there is at least one of an image area that enters from the outside of the image and an image area that exits from the image is an example of information about the motion of the estimated image area. The information about the estimated motion of the image area may be the motion vector itself.

<Person identification unit 5>
Based on the motion of the image area estimated by the motion estimation unit 4, the person identification unit 5 determines whether the identification target person has been replaced with another person, and if it is not determined that the person has been replaced, Determines that the person registered in advance is the same person, and causes the tracking unit 3 to resume the tracking (step S5).

  For example, when there is at least one of an image area entering from outside the image and an image area going out from the image, the person identification unit 5 determines that the person to be identified is another person. Judged to have been replaced. The person identification unit 5 determines that the person to be identified has been replaced with another person when there are both an image area entering from outside the image and an image area exiting from the image. May be.

  The result of determination as to whether or not the replacement has been performed is output to the tracking unit 3.

  When the tracking unit 3 receives the determination result indicating that the tracking has not been performed, the tracking unit 3 determines in advance an image region similar to the image region of the identification target person that has been the target of tracking in the image immediately before the image that failed to be tracked. Tracking is resumed assuming that the image area is a registered person.

Conventionally, once tracking fails, tracking is not resumed until face authentication succeeds. For this reason, the person cannot be identified until the face authentication is successful. The result can be obtained prior to face authentication, and by resuming tracking based on the determination of whether or not the person has been replaced, the time that the person can be identified becomes longer than before. Then, even if there is an image area similar to the image area of the person to be identified that was the target of tracking in the image immediately before the image that failed to be tracked, the image area is Suppose that the image area of the person is not registered in advance, and tracking for the image area is not resumed.

  Even if tracking fails once, if a similar image area is found in subsequent images, it can be regarded as a legitimate examinee and tracking can be resumed. However, even if a person change occurs between tracking failure and resumption, tracking a non-regular person as a regular person continues, and oversight of the change becomes a problem.

  If the tracking result is not restarted when the determination result indicating that the replacement has been received is received, it is possible to prevent the replacement of the person from when the tracking fails until the restart.

[Hardware configuration and modifications]
For example, the person identification device according to one embodiment of the present invention may function as a computer. FIG. 3 is a diagram illustrating an example of a hardware configuration of the person identification device according to the embodiment of the present invention. The person identification device described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, and the like.

  In the following description, the term “apparatus” can be read as a circuit, a device, a unit, or the like.

  Each function in the person identification device reads predetermined software (program) on hardware such as the processor 1001 and the memory 1002, so that the processor 1001 performs calculation, and communication by the communication device 1004, memory 1002, and storage 1003. This is realized by controlling the reading and / or writing of data.

  For example, the processor 1001 controls the entire computer by operating an operating system. The processor 1001 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like. For example, the information processing unit 11 described above may be realized by the processor 1001.

  Further, the processor 1001 reads a program (program code), software module, and data from the storage 1003 and / or the communication device 1004 to the memory 1002, and executes various processes according to these. As the program, a program that causes a computer to execute at least a part of the operations described in the above embodiments is used. For example, the face authentication unit 2 of the person identification device may be realized by a control program stored in the memory 1002 and operating on the processor 1001. Other functional blocks may be similarly realized. Although the above-described various processes have been described as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. Note that the program may be transmitted from a network via a telecommunication line.

  The memory 1002 is a computer-readable recording medium and includes, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. May be. The memory 1002 may be called a register, a cache, a main memory (main storage device), or the like. The memory 1002 can store a program (program code), a software module, and the like that can be executed to implement the method for controlling the person identification device according to the embodiment of the present invention.

  The storage 1003 is a computer-readable recording medium, such as an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (eg, a compact disc, a digital versatile disc, a Blu-ray). (Registered trademark) disk, smart card, flash memory (for example, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, and the like. The storage 1003 may be referred to as an auxiliary storage device. The storage medium described above may be, for example, a database, server, or other suitable medium including the memory 1002 and / or the storage 1003.

  The communication device 1004 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as a network device, a network controller, a network card, a communication module, or the like.

  The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, or the like) that accepts an external input. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 1005 and the output device 1006 may have an integrated configuration (for example, a touch panel).

  Each device such as the processor 1001 and the memory 1002 is connected by a bus 1007 for communicating information. The bus 1007 may be configured with a single bus or may be configured with different buses between apparatuses.

  The person identification device includes hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). A part or all of each functional block may be realized by the hardware. For example, the processor 1001 may be implemented by at least one of these hardware.

  Each aspect / embodiment described herein includes LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G, 5G, FRA (Future Radio Access), W-CDMA. (Registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi), IEEE 802.16 (WiMAX), IEEE 802.20, UWB (Ultra-WideBand), The present invention may be applied to a Bluetooth (registered trademark), a system using another appropriate system, and / or a next generation system extended based on the system.

  Information or the like can be output from the upper layer (or lower layer) to the lower layer (or upper layer). Input / output may be performed via a plurality of network nodes.

  The determination and determination may be performed by a value (0 or 1) represented by 1 bit, may be performed by a true / false value (Boolean: true or false), or may be performed by comparing numerical values (for example, (Comparison with a predetermined value).

  Input / output information and the like may be stored in a specific location (for example, a memory) or may be managed by a management table. Input / output information and the like can be overwritten, updated, or additionally written. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

  Although the present invention has been described in detail above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the scope of claims. Therefore, the description of the present specification is for illustrative purposes and does not have any limiting meaning to the present invention.

  Software, whether it is called software, firmware, middleware, microcode, hardware description language, or other names, instructions, instruction sets, codes, code segments, program codes, programs, subprograms, software modules , Applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc. should be interpreted broadly.

  Also, software, instructions, etc. may be transmitted / received via a transmission medium. For example, software may use websites, servers, or other devices using wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission media.

  Information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, commands, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these May be represented by a combination of

  Note that the terms described in this specification and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meaning.

  In addition, information, parameters, and the like described in this specification may be represented by absolute values, may be represented by relative values from a predetermined value, or may be represented by other corresponding information. . For example, the radio resource may be indicated by an index.

  As used herein, the terms “determining” and “determining” may encompass a wide variety of actions. “Judgment” and “determination” are, for example, judgment, calculation, calculation, processing, derivation, investigating, looking up (eg, table , Searching in a database or another data structure), considering ascertaining as “determining”, “deciding”, and the like. In addition, “determination” and “determination” include receiving (for example, receiving information), transmitting (for example, transmitting information), input (input), output (output), and access. (accessing) (e.g., accessing data in a memory) may be considered as "determined" or "determined". In addition, “determination” and “decision” means that “resolving”, “selecting”, “choosing”, “establishing”, and “comparing” are regarded as “determining” and “deciding”. May be included. In other words, “determination” and “determination” may include considering some operation as “determination” and “determination”.

  As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”

  As long as the terms “including”, “comprising”, and variations thereof are used herein or in the claims, these terms are inclusive of the term “comprising”. Intended to be Furthermore, the term “or” as used herein or in the claims is not intended to be an exclusive OR.

  Throughout this disclosure, if articles are added by translation, for example, a, an, and the in English, these articles must be clearly indicated otherwise in context, Including multiple things.

Claims (3)

A person image acquisition unit that acquires a time series of images of identification target persons that are persons to be identified;
Using the time series of the acquired images of the identification target person and the pre-registered person image, it is determined by face authentication technology whether the identification target person and the pre-registered person are the same person A face authentication unit
A tracking unit that tracks the identification target person determined to be the same person based on the time series of the images of the identification target person;
A motion estimation unit that estimates a motion of an image area included in the identification target person image based on a time series of the identification target person image when the tracking fails;
Based on the estimated movement of the image area, it is determined whether or not the person to be identified has been replaced with another person, and if it is not determined that the person has been replaced, the person identifying unit that causes the tracking unit to resume the tracking When,
A person identification device including: A person image acquisition unit in which a person image acquisition unit acquires a time series of images of a person to be identified who is a person to be identified;
Whether the identification target person and the pre-registered person are the same person by using a time series of the acquired identification target person image and a pre-registered person image. A face authentication step to be determined by face authentication technology;
A tracking step for tracking the identification target person determined to be the same person based on the time series of the image of the identification target person;
A motion estimation step for estimating a motion of an image area included in the image of the person to be identified based on a time series of the image of the person to be identified when the motion estimation unit fails in the tracking;
The person identification unit determines whether the person to be identified has been replaced with another person based on the estimated movement of the image area. If the person identification unit has not been determined to have replaced, the tracking unit performs the tracking. A person identification step to resume,
Person identification method including   The program for functioning a computer as each part of the person identification apparatus of Claim 1.

Images