JP2023077195A - Information processing device, authentication system, control method, program, and storage medium - Google Patents

Abstract

To authenticate a subject in a shorter time even when there are multiple subjects in an image.SOLUTION: An information processing device has: an acquisition part 401 for acquiring images; a detection part 402 for detecting a subject area from the image acquired by the acquisition part 401; a determination part 405 for determining the subject area targeted by image adjustment from multiple subject areas when the detection part 402 detects the multiple subject areas from the image; an identification part 404 for identifying the subject area satisfying a prescribed condition from the multiple subject areas detected by the detection part 402 during the image adjustment; an authentication part 406 for authenticating the image in the subject area satisfying the prescribed condition identified by the identification part 404; and an output part 410 for outputting an authentication result by the authentication part 406.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an authentication system, a control method, a program, and a storage medium.

Japanese Patent Application Laid-Open No. 2002-200002 discloses a technique for switching and controlling face detection and face identification. Japanese Patent Application Laid-Open No. 2002-200002 discloses a technique in which an authentication system holds means for adjusting exposure and illumination.

JP 2008-270896 JP 2007-94535

The problem to be solved by the present invention is to perform subject authentication in a shorter time even when there are a plurality of subjects in a captured image.

To solve the above problems, an information processing apparatus according to an aspect of the present invention includes an acquisition unit that acquires an image, a detection unit that detects a subject area from the image acquired by the acquisition unit, and the detection unit that includes the a determination unit that determines, when a plurality of subject regions are detected from an image, a subject region to be subjected to image adjustment from the plurality of subject regions; a specifying unit that specifies a subject area that satisfies a predetermined condition from a subject area; an authentication unit that authenticates an image of the subject area that satisfies the predetermined condition specified by the specifying unit; and an output that outputs an authentication result of the authenticating unit. and

According to the present invention, even when there are multiple subjects in an image, it is possible to perform subject authentication in a shorter time.

The figure which shows an example of the system configuration|structure which concerns on 1st Embodiment. 1 is a diagram showing an example of the device configuration of an imaging device according to the first embodiment; FIG. 1 is a diagram showing an example of the configuration of an information processing apparatus according to a first embodiment; FIG. 1 is a diagram showing an example of a functional configuration of an information processing apparatus according to a first embodiment; FIG. 4 is a flow chart showing an example of the operation of the information processing apparatus according to the first embodiment; FIG. 4 is a diagram showing an example of how authentication is performed by the information processing apparatus according to the first embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions, and the present invention is the following embodiments. is not limited to Also, a part of each embodiment described later may be appropriately combined.

<Embodiment 1>
(System configuration)
FIG. 1 is a diagram showing an example of the configuration of an imaging system including an imaging device according to this embodiment. The imaging system 100 includes an imaging device 101 , a network 102 , an information processing device 103 , a display device 104 (display unit), and an input device (input unit) 105 .

The imaging device 101 can communicate with the information processing device 103 via the network 102 . The imaging device 101 captures an image of a subject, generates an image, and transmits the captured image to the information processing device 103 via the network 102 . A display device 104 and an input device 105 are connected to the information processing device 103 , and an image sent from the imaging device 101 is displayed on the display device 104 . Also, the input device 105 is a keyboard, a mouse, or the like, and is used to operate the information processing device 103 . Note that operations to the information processing apparatus 103 include setting of imaging conditions of the imaging apparatus 101 via the network 102 and operations of PTZ (Pan Tilt Zoom).

In this embodiment, the information processing device 103, the display device 104, and the input device 105 are separately provided, but the information processing device 103, the display device 104, and the input device 105 are integrated like a notebook PC having a touch panel display. may be configured. Also, the imaging apparatus 101 and the information processing apparatus 103 may be directly connected without being connected via the network 102 . Furthermore, like a consumer camera having a touch panel display, the imaging device 101, the information processing device 103, the display device 104, and the input device 105 may all be integrated.

(Device configuration)
FIG. 2 is a diagram showing an example of the configuration of an imaging device according to this embodiment. The imaging device 101 has an imaging unit 201 , an encoder unit 202 , a network I/F 203 , a CPU (Central Processing Unit) 204 , a RAM (Random Access Memory) 205 and a ROM (Read Only Memory) 206 . The imaging optical system 200 is detachably attached to the imaging device 101 .

The imaging optical system 200 is a lens that collects light from an object onto an imaging surface of an imaging element 201a, which will be described later, and is composed of, for example, a zoom lens, a focus lens, and a blur correction lens.

In this embodiment, the imaging device 101 and the imaging optical system 200 are detachably provided as separate units, but the imaging device 101 may have the imaging optical system 200 as in a camera with an integrated lens.

An imaging unit 201 captures an image of a subject by the imaging optical system 200 and generates an image. The imaging unit 201 has an imaging element 201a, an amplifier 201b, and an image processing unit 201c.

The imaging element 201a converts the light from the object condensed on the imaging surface by the imaging optical system 200 into an electric signal for each pixel and outputs the electric signal. The imaging element 201a is an IC chip in which pixels made of photoelectric conversion elements such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor are arranged in a matrix. The image sensor 201a has high sensitivity mainly to visible light, and has high sensitivity to any one of red (R), green (G), and blue (B) for each pixel. It has some sensitivity. Therefore, it is possible to take a clear image of a subject bright with infrared light, such as a time zone when sunlight exists and a place illuminated by infrared light illumination.

The amplifier 201b amplifies and outputs the electrical signal output from the imaging device 201a.

The image processing unit 201c performs A/D conversion of the electrical signal, which is an analog signal output from the amplifier 201b, into a digital signal, and performs image processing including demosaicing processing, white balance processing, gamma processing, and the like to obtain a digital image. to generate

The encoder unit 202 encodes the image data output from the imaging unit 201 (image processing unit 201c) into a predetermined file format such as Motion Jpeg, H264, H265.

A network I/F 203 transmits the encoded image data from the encoder unit 202 to the information processing apparatus 103 via the network 102 . Note that the encoded image data may be stored in an internal storage device such as a RAM 205 or ROM 206, which will be described later, or a detachable storage medium such as an SD card (not shown). Also, these memories may be stored after being output by the image processing unit 201c. In that case, the image data is stored as RAW data before encoding.

A CPU 204 is a central processing unit that controls the imaging apparatus 101 .

A RAM 205 temporarily stores computer programs executed by the CPU 204 . Also, the RAM 205 provides a work area used when the CPU 204 executes processing. Also, the RAM 205 functions as a frame memory or as a buffer memory.

The ROM 206 stores programs and the like for the CPU 204 to control the imaging apparatus 101 .

FIG. 3 is a diagram showing an example of the configuration of an information processing device connected via the Internet to an imaging device according to this embodiment. The information processing device 103 is an information processing device having a CPU 301 , a RAM 302 , a ROM 303 , an input I/F 304 , an output I/F 305 and a network I/F 406 .

A CPU 301 is a central processing unit that controls the information processing apparatus 103 .

A RAM 302 stores programs and the like for the CPU 301 to control the information processing apparatus 103 .

The ROM 303 stores programs and the like for the CPU 301 to control the information processing apparatus 103 .

The input I/F 304 is an interface for receiving an operation for the information processing apparatus 103 that is connected to the input device 105 and input from the user via the input device 105 .

An output I/F 306 is an interface for connecting to the display device 104 and displaying an image output from the imaging device 101 on the display device 104 .

A network I/F 306 is an interface for connecting to the image capturing apparatus 101 via the network 102 and inputting operation information for the image capturing apparatus 101 and receiving images output from the image capturing apparatus 101 .

(Function configuration)
FIG. 4 is a diagram showing an example of the configuration of information processing functions according to the present embodiment. Of the functional blocks shown in FIG. 4, for functions realized by software, a program for providing the functions of each functional block is stored in a memory such as the ROM 303 . Then, the program is read out to the RAM 302 and executed by the CPU 301, thereby realizing this. For the functions realized by hardware, for example, by using a predetermined compiler, a dedicated circuit may be automatically generated on the FPGA from a program for realizing the function of each functional block. FPGA is an abbreviation for Field Programmable Gate Array. Also, a gate array circuit may be formed in the same manner as the FPGA and implemented as hardware. Also, it may be realized by an ASIC (Application Specific Integrated Circuit). Note that the configuration of the functional blocks shown in FIG. 4 is an example, and a plurality of functional blocks may constitute one functional block, or one of the functional blocks may be divided into blocks that perform a plurality of functions. good too.

The information processing apparatus 103 includes an acquisition unit 401, a detection unit 402, an extraction unit 403, an identification unit 404, a determination unit 405, an authentication unit 406, a storage unit 407, a setting unit 408, a creation unit 409, an output unit 410, and a management unit 411. have.

The acquisition unit 401 acquires an image captured by the imaging device 101 .

A detection unit 402 detects a subject area from the image acquired by the acquisition unit 401 . In the present embodiment, the case where the subject area is the area of the face will be described, but the subject area may be the area of the entire person, or may be the area of an animal or an object.

The extraction unit 403 extracts the feature amount of the subject area detected by the detection unit 402 . In the present embodiment, feature amounts relating to facial features such as the orientation and size of the face are extracted.

The specifying unit 404 specifies a subject area to be authenticated by the authentication unit 406, which will be described later. More specifically, among the subject areas detected by the detection unit 402, a subject area that satisfies a predetermined condition is identified. The predetermined condition in this embodiment is whether or not the luminance of the subject area is within a predetermined luminance range. A predetermined condition according to the amount of noise or the contrast value of the subject area may be used.

A determination unit 405 determines a subject region to be subjected to image adjustment from the subject regions detected by the detection unit 402 . In particular, when the detection unit 402 detects a plurality of subject areas, the subject area to be subjected to image adjustment is determined. The subject area to be subjected to image adjustment determined by the determination unit 405 is output to the imaging device 101 . The imaging apparatus 101 performs image adjustment based on the image information of the subject area determined by the determination unit 405 . Image adjustment is control of aperture and exposure time, for example. Image adjustment also includes post-processing such as white balance correction and NR (Noise Reduction) processing. Further, image adjustment is not limited to adjustment by the imaging device 101 and may include image adjustment by the information processing device 103 .

An authentication unit 406 authenticates the image of the subject area specified by the specifying unit 404 . The image of the subject region determined by the determination unit 405 in the adjusted image is also authenticated. The image authentication is performed by comparing the feature amount of the subject area stored in the storage unit 407 (to be described later) and the feature amount of the subject area extracted by the extraction unit 403, thereby matching the feature amount of the subject stored in the storage unit 407. Authentication is performed by determining whether similar objects are detected.

A storage unit 407 stores the feature amount of the image of the subject area. In the present embodiment, facial features are stored. A feature amount stored in advance by a user or a designer, a feature amount of an image of an object region authenticated by the authentication unit 406, and the like are stored as necessary, and referred to when the authentication processing of the authentication unit 406 is executed. database.

A setting unit 408 sets the content of authentication notification according to the authentication result by the authentication unit 408 and the authentication notification of the imaging apparatus 101 and the like. The setting unit 408 registers the authentication notification setting in the management unit 411, which will be described later.

The creation unit 409 creates authentication notification information for notifying the authentication result by the authentication unit 406 based on the authentication notification setting set by the setting unit 408 .

The output unit 410 outputs the authentication notification information created by the creation unit 409 and the authentication result itself. The output authentication result can be used, for example, by being output to the display device 104 or the like via the output I/F 305 and notifying the user of the authentication result.

The management unit 411 comprehensively manages information such as the feature amount of the image of the subject region, the authentication notification setting, and the authentication result stored in the storage unit 407 .

(Description of operation)
FIG. 5 is a flow chart showing an example of the operation of the information processing apparatus according to this embodiment. The operation of this flowchart starts when the CPU 301 in the information processing apparatus 103 reads a program stored in a storage medium such as the RAM 302 or ROM 303 .

In step S501, the acquisition unit 401 acquires an image. An image captured by the imaging device 101 or an image stored in an external storage device is acquired.

In step S<b>502 , the detection unit 402 detects a subject area from the image acquired by the acquisition unit 401 . If the subject area is not detected, the process returns to step S501 assuming that the subject does not exist in the image. If the subject area is detected, the process proceeds to step S503.

In step S503, it is determined whether the image acquired in step S501 is an image being adjusted. That is, it is determined whether or not the imaging apparatus 101 was adjusting the image when the acquired image was captured. In this embodiment, it is determined whether the imaging apparatus 101 is in exposure control. If the image was captured during exposure control, the process proceeds to step S504; otherwise, the process proceeds to step S506.

In step S504, the identifying unit 404 identifies a subject region that satisfies a predetermined condition. A subject area that satisfies a predetermined condition is identified from among the subject areas detected in step S502. The predetermined conditions are conditions such as the luminance range, noise amount, and contrast value of the image of the subject area. For example, a subject area is specified in which the average luminance of the image in the subject area is within a predetermined range. Here, the predetermined range desirably has a brightness that is neither too bright nor too dark to allow authentication by the authentication unit 406 . It should be noted that the extent of the range can be changed by the user or the designer.

In step S<b>505 , the authentication unit 406 authenticates the subject area specified by the specifying unit 404 . More specifically, the extraction unit 403 extracts the feature amount of the subject area, compares it with the feature amount of the subject area stored in the storage unit 407, and calculates the degree of similarity. If the degree of similarity is equal to or greater than a predetermined value, it is determined that the same subject as the subject stored in the storage unit 407 has been detected. Furthermore, in step S505, the output unit 410 outputs the authentication result by the authentication unit 406. FIG.

In step S506, it is determined whether or not there is a subject area to be subjected to image adjustment determined in step S508, which will be described later. Since it is determined in step S503 that the image is not being adjusted, the image acquired by the acquisition unit 401 has been adjusted, or the subject region to be subjected to image adjustment is determined in step S508, which will be described later. It is judged that it has not been done. If it is determined that there is a subject area to be subjected to image adjustment, it is determined that the image adjustment in that subject area has been completed, and the process proceeds to step S507. On the other hand, if it is determined that there is no subject area targeted for image adjustment, the process proceeds to step S508.

In step S507, the authentication unit 406 authenticates the image of the subject area for which image adjustment has been completed. Since the authentication method and the authentication result output method are the same as in step S505, descriptions thereof are omitted here.

In step S508, the determining unit 405 determines a subject area to be subjected to image adjustment. If there is only one subject area detected by the detection unit 402 in step S502, that subject area is determined as the subject area to be subjected to image adjustment. When the detection unit 402 detects a plurality of subject areas, it preferentially determines one subject area to be subjected to image adjustment. A method of determining a subject area to be subjected to image adjustment will be described below. The first is a method of making a decision based on a user's instruction. Images of a plurality of subject regions detected by the detection unit 402 are displayed on the display device 104 , and the user selects a subject region to be subjected to image adjustment via the input device 105 . An operation input to the input device 105 receives information on the selected subject area via the input I/F 305 . A determination unit 405 determines the subject area selected by the user as a subject area to be subjected to image adjustment. The second is a method of determining based on the feature amount of the image of each subject area. In step S508, the extraction unit 403 extracts the feature amount of the image of each subject area. The extracted feature amount includes, for example, the size and orientation of the face. Therefore, a subject area having the largest face size or a subject area facing the front in each subject area is determined as a subject area to be subjected to image adjustment. A third method is to determine based on the size of each subject area. A subject area having the largest subject area among the plurality of detected subject areas is determined as a subject area to be subjected to image adjustment. A fourth method is to determine based on the position of each subject area and the motion vector of the image. For example, when the position of the subject area is near the edge of the image, it is highly likely that the subject area will be out of the angle of view, so that the subject area is preferentially determined as the target of image adjustment. Also, even if it is near the edge of the image, the subject area moving toward the center of the image and the subject area moving toward the edge of the image may deviate from the angle of view. Therefore, the latter is preferentially determined as an image adjustment target. In this way, the subject areas to be subjected to image adjustment are determined based on at least one of the position of each subject area and the motion vector of the image. A fifth method is to determine based on the brightness of each subject area. For example, an average brightness value is calculated for each subject region, and a subject region having the largest difference from the average brightness value of the entire image is determined as an object for image adjustment. For example, a subject area is determined as an image adjustment target. In other words, the subject area with the lowest visibility is determined as the image adjustment target. Information on the subject area determined by the determining unit 404 is sent to the imaging apparatus 101 via the network 102 . The imaging apparatus 101 performs image adjustment (exposure control) based on the information on the subject area (information on the subject area to be subjected to image adjustment) received from the information processing apparatus 103 to improve the visibility of the target subject area. do.

In step S509, it is determined whether or not to continue the authentication operation. For example, when the user operates the input device 105 to stop the authentication operation, the authentication operation is terminated. If there is no such operation, the process returns to step S501.

FIG. 6 is a diagram showing an example of how the information processing apparatus according to the present embodiment performs an authentication operation. 6A, 6B, and 6C are in chronological order, and the images acquired by the acquisition unit 401 are arranged. For simplicity of explanation, only the face is illustrated in the image.

FIG. 6(a) shows the image before image adjustment, and the dashed line indicates the authenticated subject area. Since FIG. 6A is not an image being adjusted, the process proceeds to step S506. Since there is no subject area subject to image adjustment in step S506, the subject area subject to image adjustment is determined in step S508. Here, it is assumed that the object region 601 with the lowest average luminance is determined as the image adjustment target. A frame line indicates that the image is to be adjusted.

FIG. 6B shows an image during image adjustment. Therefore, in step S504, a subject area that satisfies a predetermined condition is authenticated. That is, the image of the subject area whose average brightness is within a predetermined range is authenticated. Here, subject areas 602 to 604 are authenticated.

FIG. 6C shows an image after image adjustment is completed. In step S506, the subject area 601 is authenticated because the subject for image adjustment exists. Once the object area targeted for image adjustment is authenticated, the object area targeted for image adjustment is determined again from the detected object areas.

According to the information processing apparatus according to the present embodiment, it is possible to perform subject authentication in a shorter time even when a plurality of subjects exist in an image.

<Other embodiments>
The present invention can be implemented by reading and executing a program that implements one or more functions of the first embodiment described above. This program is supplied to a system or device via a network or storage medium and read and executed by one or more processors in the computer of the system or device. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

401 acquisition unit 402 detection unit 403 extraction unit 404 identification unit 405 determination unit 406 authentication unit 407 storage unit 408 setting unit 409 creation unit 410 output unit 411 management unit

Claims (17)

an acquisition unit that acquires an image;
a detection unit that detects a subject area from the image acquired by the acquisition unit;
a determining unit that determines a subject region to be subjected to image adjustment from the plurality of subject regions when the detecting unit detects a plurality of subject regions from the image;
a specifying unit that specifies a subject region that satisfies a predetermined condition from the plurality of subject regions detected by the detecting unit during adjustment of the image;
an authentication unit that authenticates an image of the subject area that satisfies the predetermined condition specified by the specifying unit;
an output unit that outputs an authentication result by the authentication unit;
Information processing device having 2. The information processing apparatus according to claim 1, wherein the determining unit determines the subject area to be subjected to the image adjustment based on the brightness of the image in each subject area of the plurality of subject areas. The determination unit selects, from among the plurality of object regions, a subject region having a maximum difference between an average luminance value of the image of the object region and an average value of the image acquired by the acquisition unit, as a target of the image adjustment. 3. The information processing apparatus according to claim 1, wherein the subject area is determined. 3. The determining unit determines, from among the plurality of subject areas, a subject area having a minimum or maximum average luminance value of an image of the subject area as the subject area to be subjected to the image adjustment. 4. The information processing device according to any one of 1 to 3. further comprising an extraction unit for extracting a feature amount of the image of the subject area detected by the detection unit;
5. The determining unit determines the subject region to be subjected to the image adjustment based on the feature quantity extracted by the extracting unit from the plurality of subject regions. The information processing device according to . 6. The information processing apparatus according to any one of claims 1 to 5, wherein the predetermined condition is that an average value of brightness of the image of the subject area is within a predetermined range. 7. The information processing apparatus according to claim 1, wherein said image adjustment includes exposure control. An authentication system comprising the information processing device according to claim 1 and the imaging device. an acquisition step of acquiring an image;
a detection step of detecting a subject area from the image acquired in the acquisition step;
a determining step of determining, from the plurality of subject regions, a subject region to be subjected to image adjustment when a plurality of subject regions are detected from the image in the detecting step;
an identifying step of identifying a subject region that satisfies a predetermined condition from the plurality of subject regions detected in the detecting step during adjustment of the image;
an authentication step of authenticating an image of the subject area that satisfies the predetermined condition specified in the specifying step;
an output step of outputting an authentication result in the authentication step;
A control method for an information processing apparatus having 10. The control of the information processing apparatus according to claim 9, wherein in the determining step, the subject area to be subjected to the image adjustment is determined based on the brightness of the image in each subject area of the plurality of subject areas. Method. In the determining step, among the plurality of subject areas, the subject area having the largest difference between the average luminance value of the image of the subject area and the average value of the image acquired by the acquiring unit is subjected to the image adjustment. 11. The control method for an information processing apparatus according to claim 9, wherein the subject area is determined. 3. The determining step determines, from among the plurality of subject areas, a subject area having a minimum or maximum average luminance value of an image of the subject area as the subject area to be subjected to the image adjustment. 12. The control method for the information processing apparatus according to any one of 9 to 11. further comprising an extraction step of extracting a feature amount of the image of the subject region detected in the detection step;
13. The determining step determines the subject region to be subjected to the image adjustment based on the feature amount extracted from the plurality of subject regions in the extracting step. The control method of the information processing device according to 1. 14. The method of controlling an information processing apparatus according to claim 9, wherein the predetermined condition is that an average value of brightness of the image of the subject area is within a predetermined range. 15. The method of controlling an information processing apparatus according to claim 9, wherein said image adjustment includes exposure control. A program for causing a computer to execute the control method for an information processing apparatus according to any one of claims 9 to 15. A computer-readable storage medium storing the program according to claim 16.

Images