JP2023071510A - Information processor, authentication system, authentication method, program, and storage medium - Google Patents

Abstract

To provide an information processor, an authentication system, an authentication method, a program, and a storage medium that suppress output of erroneous authentication due to an image being adjusted.SOLUTION: In an authentication system, at least one imaging device, an image recording server, an image analysis server, and a terminal device are communicably connected via a network. Therein, an image analysis server 104, which is an information processor, comprises: an acquisition unit 301 that acquires a first image imaged by the imaging device; a detection unit 302 that detects a subject from the first image; an authentication unit 308 that, when the detection unit 302 detects the subject, authenticates the subject on the basis of information on the first image and information on a second image stored in a storage unit; a determination unit 303 that determines whether the first image is an image being adjusted; and an output unit 310 that, when the determination unit 303 determines that the first image is the image being adjusted, does not output an authentication result of the authentication unit 308, or otherwise, when it determines that the first image is not the image being adjusted, outputs the authentication result.SELECTED DRAWING: Figure 3

Description

The present invention relates to an information processing device, an authentication system, an authentication method, a program, and a storage medium.

Conventionally, there has been disclosed a technique for switching between face detection and face identification according to a photographing function (Patent Document 1). Also, a technique is disclosed in which an authentication system has exposure adjustment means (Patent Document 2).

JP 2008-270896 JP 2007-94535

A problem to be solved by the present invention is to suppress output of erroneous authentication due to an image being adjusted.

In order to solve the above problems, an information processing apparatus according to an aspect of the present invention is an information processing apparatus capable of communicating with an imaging device, comprising: an acquisition unit configured to acquire a first image captured by the imaging device;
a detection unit that detects a subject from the first image; and when the detection unit detects the subject, based on information regarding the first image and information regarding the second image stored in a storage unit, an authentication unit that authenticates the subject; a determination unit that determines whether the first image is an image being adjusted; and if the determination unit determines that the first image is an image being adjusted, and an output unit that does not output the authentication result of the authentication unit and outputs the authentication result when the determination unit determines that the first image is not an image being adjusted.

According to the present invention, it is possible to suppress output of erroneous authentication due to an image being adjusted.

The figure which shows an example of a structure of the authentication system which concerns on 1st Embodiment. 2 is a diagram showing the hardware configuration of each device according to the first embodiment; FIG. 1 is a diagram showing an example of a functional configuration of an information processing apparatus according to a first embodiment; FIG. FIG. 2 is a diagram showing an example of the functional configuration of an operation terminal according to the first embodiment; FIG. 4A and 4B are views showing an example of image adjustment operation of the authentication system according to the first embodiment; FIG. 4 is a flow chart showing an example of the operation of the information processing apparatus according to the first embodiment; FIG. 5 is a diagram showing an example of how an authentication result is output according to an image adjustment state by the information processing apparatus according to the first embodiment; 9 is a flow chart showing an example of the operation of the information processing apparatus according to the second embodiment; FIG. 11 is a diagram showing an example of how an authentication result is output according to an image adjustment state by the information processing apparatus according to the second embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions, and the present invention is the following embodiments. is not limited to Also, a part of each embodiment described later may be appropriately combined.

<Embodiment 1>
(System configuration)
FIG. 1 is a block diagram showing an example of the configuration of a face authentication system according to this embodiment.

The authentication system 100 has at least one imaging device 102 and information processing devices 103 , 104 and 105 . The information processing device 103 can be configured as a recording server, the information processing device 104 can be configured as an image analysis server, and the information processing device 105 can be configured as an operation terminal. These information processing devices may be configured as physically independent devices, or may be configured integrally.

An imaging device 102 and information processing devices 103 , 104 and 105 are communicably connected via a network 101 . The network 101 is, for example, a LAN (Local Area Network), a WAN (Wide Area Network), or the like. A physical connection form to the network 101 may be wired or wireless. Furthermore, in FIG. 1, one or two imaging devices 102 and information processing devices 103, 104, and 105 are connected to the network 101, but the number of connected devices is not limited to the number shown in FIG.

The imaging device 102 captures an image of a subject existing within the recognition target space at a predetermined angle of view. The imaging device 102 has a function of transmitting captured images (simply referred to as images) to the information processing devices 103 , 104 and 105 via the network 101 .

The image recording server 103 receives images transmitted from the imaging device 102 via the network 101 and stores and accumulates the images. In addition, the recording server 103 transmits the stored captured images to each device in response to requests received from the image analysis server 104 and the operation terminal 105 .

The image analysis server 104 receives an image or the like of a subject to be detected from the captured image from the operation terminal 105 via the network, generates a feature amount necessary for face detection processing of the subject, and uses it to detect the face of the subject. Save as feature information. It should be noted that the present embodiment will be described assuming that the subject is a person.

The image analysis server 104 generates authentication notification information when detecting a face to be detected in the captured image, and transmits the generated authentication notification information to the operation terminal 105 . The transmission of the authentication notification information may be executed based on a request from the operation terminal 105. If the operation terminal 105 is set to receive the detection result in real time, the image analysis server 104 sends the analysis result every time analysis processing ends. may be transmitted to the operation terminal 105 at the same time. In the following description of this embodiment, it is assumed that the image analysis server 104 receives an image captured by the network camera 102 via the image recording server 103, but the same applies to receiving directly from the network camera 102 as well. process can be executed. In the following description, it is assumed that the image analysis server 104 holds analysis results and registration information, but they may be saved in another database or operation terminal 105 via a network.

The operation terminal 105 includes a display device (display), and displays captured images received from the camera 102, playback/display of image data recorded in the image recording server 103, and authentication notification information received from the image analysis server 104. It has a display control function for The operation terminal 105 includes a user interface and an input unit for face detection, and is used for setting parameters related to image analysis processing for face detection, setting registration information, and setting various parameters for authentication notification information. have a function.

(Device configuration)
The device configuration of the imaging device according to this embodiment will be described. FIG. 2 is a diagram showing an example of the hardware configuration of the imaging device 102. As shown in FIG. The imaging device 102 includes a CPU (Central Processing Unit) 201, a ROM (Read Only Memory) 202, a RAM (Random Access Memory) 203, an external memory 204, an imaging unit 205, an input unit 206, a communication I/F 207, and a system bus 208. can be configured to include A configuration other than this may be further included.

The CPU 201 is a control unit that controls the operation of the imaging apparatus 102 in an integrated manner, and controls the operation of each component (202 to 207) via the system bus 208. FIG. A ROM 202 is a non-volatile memory that stores control programs and the like necessary for the CPU 201 to execute processing. Note that the program may be stored in the external memory 204 or a removable storage medium (not shown). A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 . That is, the CPU 201 loads necessary programs and the like from the ROM 202 to the RAM 203 when executing processing, and executes the programs and the like to realize various functional operations.

The external memory 204 stores, for example, various data and information necessary for the CPU 201 to perform processing using programs. The external memory 204 also stores various data and information obtained by the CPU 201 performing processing using a program or the like, for example. The image capturing unit 205 captures an image of the face and generates a captured image.

The imaging unit 205 can include, for example, an imaging element such as a CMOS (Complementary Metal Oxide Semiconductor) or a CCD (Charge Coupled Device), an A/D converter, an image processing unit, and the like.

An input unit 206 includes a power button, a setting button, and the like, and the operator of the imaging device 102 can give instructions to the imaging device 102 via the input unit 206 . The communication I/F 207 is an interface for communicating with an external device (here, the information processing device 103). Communication I/F 207 is, for example, a LAN interface. A system bus 208 connects the CPU 201, ROM 202, RAM 203, external memory 204, imaging unit 205, input unit 206, and communication I/F 207 so as to be able to communicate with each other. The functions of each unit of the imaging apparatus 102 are realized by the CPU 201 executing programs stored in the ROM 202 or the external memory 204 .

In this embodiment, the hardware configuration of each of the information processing apparatuses 103 to 105 can also conform to the hardware configuration shown in FIG. However, in that case, a hardware configuration such as a display unit can be provided. An information processing device such as a tablet or a smartphone having both the imaging unit 205 and a display unit may be used. Further, an information processing apparatus such as a PC having a display unit instead of the imaging unit 205 may be used. Here, the display unit is composed of a monitor such as a liquid crystal display (LCD). Further, the information processing apparatuses 103 to 105 are equipped with a pointing device such as a keyboard and a mouse as an input unit 206, and are configured so that an operator can give instructions to the information processing apparatuses 103 to 105. FIG. External memory 204 may also include a database for storing various information.

A functional configuration of the information processing apparatus according to this embodiment will be described. FIG. 3 is a block diagram showing an example of a functional configuration of an image analysis server (information processing device). The image analysis server 104 includes an acquisition unit 301, a detection unit 302, a determination unit 303, an adjustment unit 304, a feature amount acquisition unit 305, a subject information registration unit 306, a storage unit 307, an authentication unit 308, an authentication result information management unit 309, an output It has a section 310 , a creation section 311 and a notification control section 312 .

Each functional component shown in FIG. 3 can be realized by CPU 201 of image analysis server 104 executing a corresponding processing program. However, at least some of the elements shown in FIG. 3 may be configured to be realized by dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 201, but part of the functions may be installed in another device. For example, part of the functions may be installed in the imaging device 102 or may be installed in another information processing device including the image recording server 103 . In this embodiment, the image analysis server 104 mainly detects a person's face, but the subject to be detected is not limited to a person's face.

The acquisition unit 301 receives the captured image transmitted from the imaging device 102 via the image recording server 103, decompresses and decodes it, and acquires an image (moving image or still image) (first image). The acquisition unit 301 sequentially sends the acquired images to the detection unit 302 . Here, the captured image supplied from the imaging device 102 in this embodiment may be a still image or a moving image. Also, the supply source of the captured image is not particularly limited. The supply source may be the imaging device 102, the image recording server 103, or other imaging device that supplies captured images via a cable or wirelessly. Alternatively, the acquisition unit 301 may acquire the captured image from a memory (for example, the external memory 204) within the same device. In the following description, the acquisition unit 301 transmits one image to the detection unit 302 regardless of whether the acquisition unit 301 acquires a moving image or a still image. . In the former case, the one image corresponds to each frame forming a moving image, and in the latter case, the one image corresponds to a still image. In the case of moving images, similar processing may be repeated for each frame. Also, the imaging device 102 may transmit information indicating whether the first image is being adjusted, and the obtaining unit 301 may obtain information indicating whether the first image is being adjusted together with the first image.

The detection unit 302 performs subject detection processing on the captured image received by the acquisition unit 301 using moving object detection, a matching pattern dictionary, and the like. As a result, a subject such as a face or a human body is detected in an area in the captured image, and a subject area including the subject in the captured image is extracted.

Here, the subject detection processing in the present embodiment may be any method as long as the subject can be detected from an image, managed by an identifier such as an ID capable of identifying an area including the detected subject, and tracked. good too. A method of detecting a moving object and managing it with an ID that identifies the object of the moving object may be used, or a method of detecting the area of the human body by Ω tracking and tracking the area of the human body may be used. Alternatively, a method of detecting a face region instead of a moving body or human body region and tracking the same face may be used. Furthermore, if a human body is detected first during image adjustment, the face may be temporarily hidden due to tracking of the human body, or due to changes in the image during image adjustment during video recording, making face detection temporarily difficult. It is also possible to continue tracking the human body and continue identifying the face of the same person even if there is a frame with the same number of frames.

A detection unit 302 detects a subject for each image and assigns a unique ID (Identifier) as an identifier of the subject area. If the subject is continuously detected in past images under a predetermined condition, the same ID as that in the past image is given. If the subject has no continuity with the past image, an ID different from that of the past image is given and the detected subject area is managed. For example, even if the subject is not detected for a certain period of time (the number of images), if it can be determined that the subject is the same as the person in the past image, the same ID as in the past image is given. As another example, even if there is an image in which the direction or size of the subject in the image changes and it is difficult to detect the subject area, it is possible to regard it as the same person by estimating the movement of a moving body, etc., and assign the same ID. .

By the above method, the object is detected and tracked in the images continuously acquired by the acquisition unit 301 .

The determination unit 303 determines whether the captured image (first image) captured by the imaging device 102 in the authentication system according to this embodiment is being adjusted. This determination may be made by receiving information indicating whether or not the image is being adjusted from the imaging device 102, or may be determined in accordance with an instruction from the adjustment unit 304, which will be described later. During the adjustment period, after it is determined that an image that is not suitable for authentication has been acquired in the area of the subject that is the authentication target in the captured image captured by the imaging device 102, an image adjustment instruction is issued to the imaging device 102, Until it is determined that an image suitable for authentication that does not require image adjustment has been acquired.

The adjustment unit 304 controls adjustment of the captured image (first image). For example, when the captured image of the imaging device 102 cannot be correctly authenticated due to the influence of backlight or the like, AE correction may be instructed. If the subject is a face, the average brightness of each pixel in the face area used for authentication is calculated. Then, whether or not the calculated average luminance value is within a predetermined range that does not interfere with authentication is determined to determine whether or not image adjustment is necessary, and the normal shooting state and the image adjusting state are transitioned. . When a plurality of subjects are present in the captured image, determination can be made by averaging luminance values of the plurality of subjects. An example of AE correction has been shown as an example of the adjusting state of the present embodiment, but other image adjusting means can be used in the same manner as long as it determines and adjusts an image suitable for subject authentication. When adjusting a captured image using a function of the imaging apparatus 102 such as AE correction, the adjustment unit 304 instructs the imaging apparatus 102 to adjust the image, so that the imaging apparatus 102 adjusts the captured image. When the captured image acquired by the imaging device 102 is acquired by the image analysis server 104 and then subjected to various image processing for image adjustment, the image adjustment unit 304 can also instruct the acquisition unit 301 to adjust the captured image.

A feature amount acquisition unit 305 performs feature amount extraction processing for extracting a feature amount (information about the first image) based on a predetermined calculation from the subject image (first image) detected by the detection unit 302. . In this embodiment, the feature amount of the face is extracted as the feature amount. Any feature amount of the subject may be used as long as it is used for authentication of the subject.

A subject information registration unit 306 receives information including a face image to be authenticated from the operation terminal 105 via the LAN 101, generates a feature amount (information about the second image) necessary for face detection processing, and stores face features. The information (hereinafter referred to as “facial feature information”) is stored in the storage unit 307 . The feature amount generation here can be performed in the same manner as the process performed by the feature amount acquisition unit 305 . Further, the subject information registration unit 306 stores information on the face posture with respect to the gravity direction in the storage unit 307 as face posture information (information on the second image) of the registered face image based on the user's input. Any information can be used as the face posture information of the registered face image as long as it can identify the tendency of face deformation from the face posture in the real space. The pose of the face can be represented by a three-dimensional vector, or the orientation of the face can be specified in terms of general human body orientations such as standing, supine, supine, and prone. Alternatively, an image of a desired subject (second image) may simply be stored in the storage unit 307 .

The storage unit 307 stores the facial feature amount and image to be registered and the facial posture information of the registered image generated by the subject information registration unit 306 together with information such as the registration name and registration time as facial feature information (information about the second image). ). Information stored in the storage unit 307 is not limited to facial feature information. Information on the body shape of the subject as a whole or characteristic parts (arms, legs, etc.) may also be used. Further, as described above, it may be image data of an image of a subject (second image), or may be a structure in which they are associated.

The authentication unit 308 collates the feature amount (information about the first image) acquired by the feature amount acquisition unit 305 with the feature amount (information about the second image) of the face to be authenticated held and managed by the storage unit 307. and calculate the similarity. If the calculated similarity is equal to or greater than a predetermined threshold, it is determined that the imaged face is similar to the detection target face, and it is recognized that the verification target face (or human body) has been detected. That is, the face detected from the image and the registered face feature information are authenticated. Also, the authentication unit 308 temporarily holds authentication information for each ID of the detected subject.

In the case of a fixed camera, the authentication unit 308 performs authentication using the face pose suitable for authentication set in the camera face pose information and the pose information (information about the second image) of the registered face image. A three-dimensional vector may be used as the orientation with respect to the direction of gravity. An authentication method may be selected based on information that allows orientation with respect to the direction of gravity, such as horizontal orientation or vertical orientation. Also, the information about the posture can be converted into a three-dimensional vector and used for determination. Further, the authentication performed by the authentication unit 308 is not limited to faces. Authentication may be performed based on the body shape of the subject as a whole or characteristic parts (legs, arms, etc.).

In the case of a camera such as a pan-tilt-zoom camera that can change the shooting direction of the camera or a shooting device with a posture sensor, the face posture in relation to the direction of gravity is calculated from the posture of the camera when the authentication image was taken. Any facial pose can also be used for authentication determination. In addition, while the posture and shooting direction of the camera are variable, it is also possible to determine and operate a face posture suitable for authentication like a fixed camera. In this case, authentication is determined based on two conditions, ie, the camera is in a posture suitable for face authentication and the face posture in the direction of gravity is in a posture suitable for authentication.

The authentication result information management unit 309 holds and manages authentication result information of the subject. In association with the ID of the tracked subject, the image of the face area at the time of authentication is held, and the authentication result for each frame image is held and managed. The information to be saved as the authentication result includes, for example, information for specifying the registered subject such as the ID of the registered person, information related to authentication such as the degree of similarity with the face image of the registered person, the authenticated frame image, the camera It contains information about shooting such as. As for the authentication result information, when the tracking of the subject is basically completed, the related information for each tracking target is deleted after the processing required at the time of termination is completed, but it may be permanently stored in the storage device. Alternatively, the usage method may be such that the information is temporarily stored in the memory until an arbitrary timing during the operation of the software of the authentication system according to this embodiment.

Output unit 310 outputs the result of authentication by authentication unit 308 . The authentication result is the result of determining whether or not the information about the detected subject and the information about the subject stored in the storage unit 307 are similar, or the information about the subject determined to be similar to the information about the detected subject. It happens. The authentication result may contain similarity information together with these pieces of information, or may contain information about the first image and information about the second image. Information indicating whether the authentication result to be output is the result of authentication performed by the authentication unit 308 on the image being adjusted can also be included.

The creating unit 311 creates authentication notification information for notifying that the detection target face has been detected. Authentication notification information is created based on the authentication result output from the output unit 310 . The authentication notification information can include information such as camera settings when the imaging device 102 captures the first image, and settings related to tracking and authentication. The output unit 310 and the creation unit 311 may be reversed. That is, the creation unit 311 may create authentication notification information based on the authentication result of the authentication unit 308, and the output unit 310 may output the authentication notification information.

The notification control unit 312 determines whether or not to notify the user of the notification information created by the creation unit 311, and controls notification to the user. Whether or not to notify the user of the notification information is controlled based on the authentication result output from the output unit 310 or the notification information created by the creation unit 311 .

FIG. 4 is a block diagram showing an example of the functional configuration of the operation terminal 105 according to this embodiment. The operation terminal 105 includes an authentication notification reception section 401 , a D 402 , a setting input section 403 , a setting communication section 404 and an authentication notification section 405 . A setting input unit 403 inputs face information including a face image to be detected and authentication notification information. The setting communication unit 404 transmits the input information to the image analysis server 104 via the LAN 101 . The authentication notification reception unit 401 receives authentication notification information from the image analysis server 104 via the LAN 101 . The image acquisition unit 402 acquires from the image recording server 103 the captured image at the alarm occurrence time of the received authentication notification information. The authentication notification unit 405 displays an authentication notification display screen including the received authentication notification information and a captured image related to the authentication notification on the display, and notifies the operator of the authentication. A procedure for using the AE correction function together as an example of the image adjustment function for a captured image will be described.

(Description of operation)
The operation of the imaging system according to this embodiment will be described. FIG. 5 is a flow chart showing an example of the operation of the authentication system according to this embodiment. The operations shown in this flowchart are started by loading a program stored in a ROM or the like in an imaging device or an information processing device into a RAM and reading the loaded program by the CPU.

In step S501, image adjustment of the imaging device 102 is started. The acquisition unit 301 in the image analysis server 104 may be started when the imaging device 102 starts acquiring images. Image adjustment may be started at any timing of the user, such as when an input unit such as a UI displayed on the imaging device 102 , recording server 103 , image analysis server 104 , or operation terminal 105 is operated.

In step S502, the continuation of the image adjustment function is confirmed, and if an instruction to stop the image adjustment function has not been input, the process proceeds to step S503, and if a stop instruction has been input, the operation ends.

In step S503, the acquisition unit 301 acquires an image captured by the imaging device 102, and the process proceeds to step S504. Here, in this embodiment, the image acquisition for the image adjustment function will be described. Captured images may be used so that the implementation procedure of the form is incorporated.

In S<b>504 , the detection unit 302 detects a face or a subject having a correlation with the position of the face on the image acquired by the acquisition unit 301 . Then, for example, it is checked whether or not a subject such as a human body or a wheelchair, which is correlated with the face to be authenticated as an authentication target or the position of the face, exists in the acquired image. Then, it is determined whether or not the area of the face image is within a predetermined luminance value range. The brightness value can be determined, for example, by the average value of the brightness values of each pixel in the face image area. Any method can be used. Here, in the case of a subject that is correlated with a face, the position where the face exists is estimated from the shape and posture of the subject. For example, in the case of a human body, the position and size of the head are estimated using a method such as skeletal estimation, and even an image that cannot be clearly detected as a face is an object to be tried for AE correction.

The authentication system according to this embodiment is not limited to the method of detecting only faces. For example, if a human body is detected, even though the face is unclear, when photographing under strong backlight conditions, the face is present near the head of the human body and can be detected by adjusting the image of the photographing device. In order to finally obtain a desired face image for face authentication, it is also possible to adjust the image by tracking a subject other than the face, such as a human body. In addition, if a wheelchair can be detected in addition to the human body, it is possible to track subjects other than the human body as long as the moving body has a regular positional relationship with the face such that the approximate position of the head or face can be estimated. , image adjustment of face images for face authentication is also possible. If a face or a subject correlated with the face is detected and it is determined that there is a face that requires image adjustment as a subject for face authentication, the process proceeds to step S505.

In step S505, it is determined whether or not the image is being adjusted. If it is not in the image adjusting state, the process proceeds to S506, and after the state is changed to the image adjusting in progress, the process proceeds to S507. If the image is already being adjusted, the process proceeds directly to step S507.

Here, the image adjusting state will be described. When recognizing a subject such as face recognition, for example, image adjustment such as AE correction may be required in a situation where backlight is severe. Image adjustment, such as AE correction, in which correction instructions are repeated until the desired image is obtained, takes time to obtain the desired image. In the photographed images in the image adjustment-in-progress state, it may not be possible to acquire a photographed image of a normal subject such as a face, and some of the images may cause erroneous identification of the subject. When an object is detected from an acquired image, if the image area of the object does not fall within a predetermined value range and image adjustment is required, the photographing device is placed in an image adjusting state in which image adjustment is being performed. For example, if an image adjustment instruction is sent to the imaging apparatus during video shooting by sending an AE correction value as a command to the camera, the images of successively acquired frames change from the middle. If the subject image area falls within a predetermined value range and an image that does not interfere with subject authentication can be obtained, the state transitions to the normal shooting state. In the normal photographing state, an image adjustment instruction is not issued, and a normal subject authentication function can be executed. In image adjustment during moving image shooting, the effect of the image adjustment instruction is confirmed and the next correction instruction is repeated until the desired image is acquired.

In step S507, the adjustment unit 304 instructs the imaging apparatus 102 in the image adjusting state to perform image adjustment. In this embodiment, the AE correction is instructed with a predetermined correction value. The correction value may be any value, but it is desirable that the correction value is such that the image adjustment can be completed more quickly. In the present embodiment, AE correction will be described as an example, but in image adjustment for obtaining a subject area image suitable for subject authentication, an arbitrary value is used according to the characteristics of each image adjustment method. When the image adjustment instruction is transmitted to the imaging apparatus 102 in step S502, the process proceeds to step S502.

If it is determined in step S503 that there is no face or a subject image correlated with the face in the image, or if it is determined that there is no need to adjust the image as a subject for face authentication, the process proceeds to step S508.

In step S508, it is determined whether the end of the image adjusting state is required. If the acquired image does not include a subject for face authentication and the image is not being adjusted, the process advances to S502 to acquire the next image. If it is already in the image adjusting state and in the normal photographing state, it is determined whether or not to end the image adjusting state. For example, if the subject to be adjusted does not exist in a plurality of consecutive frame images and then terminates, the image adjusting state is maintained until it is confirmed that the subject does not exist in a predetermined number of frame images. Without changing, the process proceeds to step S502. If the process ends immediately with one frame image, or if there is no subject for image adjustment among the predetermined number of consecutive frame images, the process proceeds to step S509.

In step S509, the image adjusting state is changed to the normal shooting state, the image adjusting process is terminated, and the process proceeds to step S502.

In step S502, if the user has not input an image adjustment end instruction, the processing from S503 to S509 is continued while changing the state regarding image adjustment in the presence of the subject to be face-recognised in the acquired image.

FIG. 6 is a flow chart showing the operation of the information processing apparatus according to this embodiment. Note that the flowchart of FIG. 6 is based on the premise that the backlight becomes strong during photographing and it is difficult to detect a person. This flowchart starts operation by loading a program stored in a ROM or the like into a RAM and executing the loaded program by the CPU (step S601).

In step S<b>601 , the acquisition unit 301 acquires a captured image (first image) transmitted from the imaging device 102 .

In step S<b>602 , the detection unit 302 detects a subject (eg, human body or face) from the first image acquired by the acquisition unit 301 . If the subject is detected, the process proceeds to step S603, and if the subject cannot be detected, the subject to be authenticated does not exist within the angle of view of the imaging device 102, and the process proceeds to step S605.

In step S603, the authentication unit 308 authenticates the subject detected by the detection unit 302 based on the information regarding the first image and the information regarding the second image. More specifically, the information about the second image stored in the storage unit 307 and the information about the first image are collated. The degree of similarity between the detected subject information and the subject information stored in the storage unit 307 is calculated. If the calculated similarity is equal to or greater than a predetermined threshold, it is determined that the detected subject information is similar to the subject information stored in the storage unit 307 . The authentication result by the authentication unit 308 is a determination result as to whether or not the information about the detected subject and the information about the subject stored in the storage unit 307 are similar. It may be information about the subject. The authentication result may contain similarity information together with these pieces of information, or may contain information about the first image and information about the second image.

In step S604, the determination unit 303 determines whether the first image acquired by the acquisition unit 301 is an image being adjusted (for example, AE is being adjusted). The determination unit 303 determines based on the information output from the adjustment unit 304 that instructs adjustment of the image, the information received from the imaging device 102 that indicates whether the image is being adjusted, and the like. If the first image is being adjusted, the authentication unit 308 has authenticated the image being adjusted, so it is determined that the authentication result is uncertain, and the process advances to step S606.

In step S<b>605 , the output unit 310 outputs the result of authentication by the authentication unit 308 . As described above, in step S605, the creation unit 311 may create authentication notification information based on the authentication result by the authentication unit 308, and the output unit 310 may output it.

In step S606, it is determined whether or not to continue subject authentication. If the authentication of the subject is to be continued, the process returns to step S601 to acquire the image again. If the subject authentication is not to be continued, the operation is terminated.

FIG. 7 shows an example in which the information processing apparatus (authentication system) according to the present embodiment suppresses output of incorrect authentication. Here, the authentication unit 308 erroneously authenticates Mr. A as Mr. B during image adjustment. When the determination unit 303 determines that the image is being adjusted, the authentication result by the authentication unit 308 is not output. Therefore, even if the authentication unit 308 authenticates Mr. B with a degree of similarity above a predetermined value, the authentication result is not output, so output of erroneous authentication can be suppressed. Therefore, the user is not notified of erroneous authentication, and erroneous operations such as opening the gate according to the result of erroneous authentication at the authentication gate can be suppressed.

On the other hand, if the image is not being adjusted, the authentication result is output. In the example of FIG. 7, Mr. A is correctly authenticated in the normal photographing state (the state in which the image is not being adjusted), and the authentication result is also output.

Thus, the authentication result of the authentication unit 308 for the image being adjusted is not output. Therefore, according to the information processing apparatus (authentication system) according to the present embodiment, it is possible to suppress output of erroneous authentication caused by an image being adjusted.

<Embodiment 2>
In the present embodiment, a case will be described in which, in the first embodiment, the subject can only be authenticated using an image being adjusted.

In this embodiment, if the subject can only be authenticated using the image being adjusted, the authentication result cannot be output even if the subject cannot be correctly authenticated using the image being adjusted. According to the present embodiment, it is possible to suppress output omission of the authentication result of the subject. Note that the configuration of the authentication system according to this embodiment and the device configuration and functional configuration of the information processing apparatus according to this embodiment are the same as those of the first embodiment, so descriptions thereof will be omitted.

(Description of operation)
FIG. 8 is a flow chart showing an example of the operation of the information processing apparatus (authentication system) according to this embodiment. Note that the operations in steps S801 to S805 and S809 are the same as those in the first embodiment, and thus description thereof is omitted.

If it is determined in step S804 that the image (first image) captured by the imaging device 102 is being adjusted, the authentication result by the authentication unit 308 is stored in the storage unit 307 in step S806.

In step S806, the first image is compared with the image captured in the previous frame of the first image to determine whether one or more subjects are out of the frame. If it is determined that one or more subjects have gone out of frame, the process proceeds to step S808; otherwise, the process proceeds to step S809. Whether or not one or more subjects are out of the frame can also be determined by comparing the authentication result of the first image and the authentication result of the previous frame accumulated in the storage unit 307, and based on the difference between the authenticated subject IDs.

In step S808, the output unit 310 outputs the authentication result of the frame-out subject (or the authentication notification information created by the creation unit 311). An ID that has not been authenticated is specified by comparing the authentication result of the first image with the authentication result of the image captured in the previous frame of the first image. As a result, it is possible to identify which subject is the frame-out subject. The output unit 310 acquires the authentication result corresponding to the ID of the frame-out subject from the authentication result of the previous frame accumulated in the storage unit 307, and outputs it. It should be noted that it is not necessary to acquire the authentication result corresponding to the subject ID from the authentication result of the previous frame. For example, an authentication result corresponding to the subject's ID can be retrieved from the authentication results accumulated in the storage unit 307, and an arbitrary authentication result can be obtained and output. In particular, it is desirable to output the authentication result with the highest degree of similarity among the plurality of authentication results corresponding to the object ID accumulated in the storage unit 307 .

FIG. 9 shows an example in which the information processing apparatus (authentication system) according to the present embodiment suppresses the output omission of the subject authentication result. Here, it is assumed that the authentication unit 308 has authenticated Mr. C during the adjustment of the image, and has been correctly authenticated. Since the authentication result may be uncertain during image adjustment, the authentication result is accumulated in the storage unit 307 . After that, in the normal photographing state (the state in which the image is not being adjusted), Mr. C may be out of the frame without being authenticated with the same ID that was authenticated in the adjusting state. In this case, the output unit 310 outputs the authentication result in the image adjusting state accumulated in the storage unit 307 . As described above, it is desirable to output the authentication result having the highest degree of similarity among the authentication results stored in the storage unit 307 .

According to the information processing apparatus (authentication system) according to the present embodiment, it is possible to suppress output omission of the authentication result of the subject. Further, the authentication result output from the output unit 310 may include information indicating whether or not the authentication unit 308 has authenticated the image being adjusted. Therefore, when notifying the authentication result by the notification control unit 312, the user can confirm the accuracy of the authentication result by notifying the user that the authentication result was obtained during image adjustment along with the similarity. Further, in the authentication gate, when the authentication result during image adjustment is output, it is possible to set the gate to be opened when the degree of similarity is equal to or higher than a predetermined threshold.

<Other embodiments>
The present invention can be implemented by reading and executing a program that implements one or more functions of the first embodiment described above. This program is supplied to a system or device via a network or storage medium and read and executed by one or more processors in the computer of the system or device. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Each functional component shown in FIG. 3 can be realized by CPU 201 of image analysis server 104 executing a corresponding processing program. However, at least some of the elements shown in FIG. 3 may be configured to be realized by dedicated hardware. In this case, the dedicated hardware operates under the control of the CPU 201 .

100 imaging device 103 image recording server (information processing device)
104 image analysis server (information processing device)
105 operation terminal (information processing device)
301 acquisition unit 302 detection unit 303 determination unit 308 authentication unit 310 output unit

Claims (11)

An information processing device communicable with an imaging device,
an acquisition unit that acquires a first image captured by the imaging device;
a detection unit that detects a subject from the first image;
an authentication unit that authenticates the subject based on information regarding the first image and information regarding the second image stored in a storage unit when the detection unit detects the subject;
a determination unit that determines whether the first image is an image being adjusted; and if the determination unit determines that the first image is an image that is being adjusted, the authentication result of the authentication unit is output. an output unit configured to output the authentication result when the determination unit determines that the first image is not the image being adjusted;
Information processing device having 2. The method according to claim 1, wherein the authentication unit calculates a degree of similarity between information about the first image and information about the second image, and authenticates the subject based on the degree of similarity. Information processing equipment. further comprising the storage unit;
the storage unit stores authentication results of the authentication unit when the determination unit determines that the first image is an image being adjusted;
The output unit, when the determination unit determines that the first image is an image being adjusted, and when one or more subjects are out of the frame from the first image, the 3. The information processing apparatus according to claim 1, wherein the authentication result is output. 4. The information processing apparatus according to any one of claims 1 to 3, further comprising a notification control unit that notifies a user of the authentication result of the authentication unit. An authentication system having an imaging device and an information processing device,
an acquisition unit that acquires a first image captured by the imaging device;
a detection unit that detects a subject from the first image;
an authentication unit that authenticates the subject based on information regarding the first image and information regarding the second image stored in a storage unit when the detection unit detects the subject;
a determination unit that determines whether the first image is an image being adjusted; and if the determination unit determines that the first image is an image that is being adjusted, the authentication result of the authentication unit is output. an output unit configured to output the authentication result when the determination unit determines that the first image is not the image being adjusted;
authentication system. an acquisition step of acquiring a first image captured by the imaging device;
a detection step of detecting a subject from the first image;
an authentication step of authenticating the subject based on the information about the first image and the information about the second image stored in the storing step when the subject is detected in the detecting step;
a determination step of determining whether the first image is an image being adjusted; and outputting an authentication result of the authentication step when the first image is determined to be an image being adjusted in the determination step. an output step of outputting the authentication result if the determination step determines that the first image is not the image being adjusted;
Authentication method with 7. The method according to claim 6, wherein in said authentication step, a degree of similarity between information on said first image and information on said second image is calculated, and said subject is authenticated based on said degree of similarity. Authentication method. further comprising the storing step;
In the storing step, when the determination unit determines that the first image is an image being adjusted, the authentication result of the authentication step is accumulated;
In the output step, if the determination step determines that the first image is an image being adjusted and one or more subjects are out of the frame from the first image, the 8. The authentication method according to claim 6, wherein an authentication result is output. 9. The authentication method according to any one of claims 6 to 8, further comprising a notification control step of notifying a user of the authentication result of said authentication step. A program for causing a computer to execute the authentication method according to any one of claims 6 to 9. A computer-readable storage medium storing the program according to claim 10.

Images