JP2023047757A - Information processing device, imaging apparatus, method, and storage medium - Google Patents

Abstract

To control exposure values of and exposure information on an imaging apparatus more suitably when a plurality of specific areas are detected.SOLUTION: An information processing device has an acquiring unit 403, a detecting unit 405, a calculation unit 406, and a determining unit 407. The acquiring unit acquires an image captured by an imaging apparatus 101. The detecting unit detects a specific area from the image acquired by the acquiring unit 403. The calculation unit 406 calculates an exposure correction value which is the difference between a current exposure value and a target exposure value for imaging apparatus 1010 on the basis of luminance information on the specific area detected by the detecting unit 405. The calculation unit calculates a plurality of exposure correction values corresponding to the luminance information for each of the plurality of specific areas when the detecting unit 405 detects the plurality of specific areas. The determining unit determines an exposure correction value with the minimum absolute value from the plurality of exposure correction values.SELECTED DRAWING: Figure 1

Description

The present invention relates to an information processing device, an imaging device, a method, and a storage medium.

Conventionally, a specific region is detected from an image captured by an imaging device (for example, a human body or face region), and based on information about the specific region (for example, luminance value), the exposure value and exposure information (exposure time, exposure time, gain, aperture value, etc.) is disclosed.

JP 2020-72469

The problem to be solved by the present invention is to more preferably control the exposure value and exposure information of an imaging device when a plurality of specific regions are detected.

To solve the above problems, an imaging device according to an aspect of the present invention includes an acquisition unit that acquires an image captured by the imaging device, and a detection unit that detects a specific region from the image acquired by the acquisition unit. a calculating unit for calculating an exposure correction value, which is a difference between a current exposure value of the imaging device and a target exposure value, based on the luminance information of the specific region detected by the detecting unit, a calculating unit for calculating a plurality of exposure correction values corresponding to luminance information for each of the plurality of specific regions when the unit detects a plurality of specific regions; and a determination unit that determines an exposure correction value.

According to the present invention, when a plurality of specific areas are detected, the exposure correction value and exposure information of the imaging device are more preferably controlled.

1 is a block diagram showing an example configuration of an imaging control system according to a first embodiment; FIG. 1 is a block diagram showing an example of the device configuration of a monitoring camera according to the first embodiment; FIG. 1 is a block diagram showing an example of the configuration of an information processing apparatus according to a first embodiment; FIG. 1 is a block diagram showing an example of a functional configuration of an information processing device according to a first embodiment; FIG. 4 is a flowchart showing an example of exposure determination processing of the information processing apparatus according to the first embodiment; FIG. 7 is a diagram showing an example of a result of counting the number of specific regions corresponding to calculated exposure correction values in the information processing apparatus according to the first embodiment; It is a figure showing an example of functional composition of an information processor concerning a 2nd embodiment. 9 is a flow chart showing an example of exposure determination processing of an information processing apparatus according to a second embodiment; It is a figure which shows an example of the process which excludes the face authenticated person of the information processing apparatus which concerns on 2nd Embodiment. 9 is a flow chart showing an example of face authentication processing of the information processing apparatus according to the second embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions, and the present invention is the following embodiments. is not limited to Also, a part of each embodiment described later may be appropriately combined.

<Embodiment 1>
(System configuration)
FIG. 1 is a block diagram showing an example of the configuration of an imaging control system according to this embodiment. The imaging control system shown in FIG. 1 includes an imaging device 101 , a network 102 , a client device 103 , an input device 104 and a display device 105 . Note that the imaging device 101 is a device capable of imaging a subject and performing image processing for obtaining a moving image. The imaging device 101 and the client device 103 are connected via a network 102 so as to be able to communicate with each other.

(Device configuration)
FIG. 2 is a block diagram showing an example of the device configuration of the monitoring camera according to this embodiment. An imaging optical system 201 is a group of optical members that includes a zoom lens, a focus lens, a blur correction lens, an aperture, a shutter, and the like, and collects optical information of a subject.

The image pickup device 202 is a charge storage type solid-state image pickup device such as a CMOS or CCD that converts the luminous flux condensed by the image pickup optical system 201 into a current value (signal value). An imaging unit that acquires color information. Note that the image pickup device 202 is an image sensor capable of setting an arbitrary exposure time for pixels.

A CPU 203 is a control unit that controls the operation of the imaging apparatus 101 in an integrated manner. The CPU 203 reads instructions stored in a ROM (Read Only Memory) 204 or a RAM (Random Access Memory) 205 and executes processing according to the instructions. In addition, the imaging system control unit 206 controls each unit of the imaging apparatus 101 (based on instructions from the CPU 203) such as focus control, shutter control, and aperture adjustment for the imaging optical system 201. FIG. A communication control unit 207 performs control for transmitting control related to each unit of the imaging apparatus 101 to the CPU 203 through communication with the client apparatus 103 .

An A/D conversion unit 208 converts the amount of light of a subject detected by the image sensor 202 into a digital signal value. The image processing unit 209 is image processing means for performing image processing on the image data of the digital signal output from the image sensor 202 . The encoder unit 210 is conversion means for converting the image data processed by the image processing unit 209 into a file format such as Motion Jpeg, H264, H265, or the like. A network I/F 211 is an interface used for communication with an external device such as the client device 103 via the network 102 and is controlled by the communication control unit 207 .

A network 102 is an IP network that connects the imaging device 101 and the client device 103 . A network is composed of a plurality of routers, switches, cables, etc. that satisfy a communication standard such as Ethernet (registered trademark), for example. In this embodiment, the network 102 may be any network that allows communication between the imaging device 101 and the client device 103, regardless of its communication standard, scale, configuration, and the like. For example, the network 102 may be configured by the Internet, a wired LAN (Local Area Network), a wireless LAN (Wireless LAN), a WAN (Wide Area Network), or the like.

FIG. 3 is a block diagram showing an example of the configuration of the information processing apparatus according to this embodiment. The client device 103 includes a client CPU 301 , main storage device 302 , auxiliary storage device 303 , input I/F 304 , output I/F 305 and network I/F 306 . Each element is communicably connected to each other via a system bus.

The client CPU 301 is a central processing unit that centrally controls the operation of the client device 103 . Note that the client CPU 301 may be configured to execute overall control of the imaging apparatus 101 via the network 102 . The main storage device 302 is a storage device such as a RAM that functions as a temporary storage location for data of the client CPU 301 . The auxiliary storage device 303 is a storage device such as an HDD, ROM, SSD, etc. that stores various programs, various setting data, and the like. The input I/F 304 is an interface used when receiving input from the input device 104 or the like. The output I/F 305 is an interface used to output information to the display device 105 or the like. A network I/F 306 is an interface used for communication with an external device such as the imaging device 101 via the network 102 .

The client CPU 301 executes processing based on the programs stored in the auxiliary storage device 303, thereby realizing the functions and processing of the client device 103 shown in FIG. Details of this will be described later.

As shown in FIG. 1, the input device 104 is an input device including a mouse, keyboard, and the like. A display device 105 is a display device such as a monitor that displays an image output from the client device 103 . In this embodiment, the client device 103, the input device 104, and the display device 105 are configured independently, but the present invention is not limited to this. For example, the client device 103 and display device 105 may be integrated, or the input device 104 and display device 105 may be integrated. Also, the client device 103, the input device 104, and the display device 105 may be integrated. Further, like a consumer camera, the imaging device 101, the client device 103, the input device 104, and the display device 105 may be integrated. In that case, the imaging device 101 has the functional configuration of the client device 103 illustrated in FIG.

FIG. 4 is a block diagram showing an example of the functional configuration of the information processing apparatus according to this embodiment. In other words, each unit illustrated in FIG. 4 is a function/configuration that can be executed by the client CPU 301 , and these units are synonymous with the client CPU 301 . That is, the client CPU 301 of the client device 103 includes an input signal acquisition unit 401, a communication control unit 402, an input image acquisition unit 403, a camera information acquisition unit 404, an object detection unit 405, an exposure calculation unit 406, an exposure determination unit 407, and a display control unit. A portion 408 is included. Note that the client device 103 may have a configuration that includes each unit illustrated in FIG. 4 as a configuration different from that of the client CPU 301 .

The input signal acquisition unit 401 is input means for receiving input from the user via the input device 104 .

A communication control unit 402 executes control for receiving an image transmitted from the imaging device 101 via the network 102 . Also, the communication control unit 402 executes control for transmitting a control command to the imaging apparatus 101 via the network 102 .

An input image acquisition unit (acquisition unit) 403 is an image acquisition unit that acquires an image captured by the imaging device 101 via the communication control unit 402 as an image to be subjected to subject detection processing. Details of the detection process will be described later. A camera information acquisition unit 404 is an acquisition unit that acquires camera information (imaging information) when an image of a subject is captured by the imaging device 101 via the communication control unit 402 . Camera information (imaging information) is various information when capturing an image of a subject to obtain an image, and includes the current exposure value and exposure information (exposure time, gain, aperture value, etc.) set in the imaging device 101 . include.

A subject detection unit (detection unit) 405 detects a specific area from the image acquired by the input image acquisition unit 403 . A specific region is, for example, a face region, a facial organ region, a human body region, or the like. In this embodiment, regions of a face and a human body are described as examples of specific regions, but the configuration may be such that a region of a subject such as an animal or a car is detected.

An exposure calculation unit (calculation unit) 406 calculates a target exposure when capturing an image of a subject and obtaining an image based on the detection result obtained from the subject detection unit 405 . The target exposure is an exposure value according to an exposure control program diagram prerecorded in the client device 103, and is calculated based on luminance information (for example, luminance value) of a specific area detected by the subject detection unit 405. This target exposure is determined for each subject. For example, when there are three subjects, a target exposure is determined for each subject and a total of three target exposures are calculated. The exposure calculation unit 406 also calculates an exposure correction value, which is the difference between the current exposure value acquired by the input image acquisition unit and the target exposure value. Similar to the target exposure, when multiple specific areas are detected, multiple exposure correction values corresponding to each of the multiple specific areas are calculated. The target exposure or exposure correction value is temporarily stored in the auxiliary storage device 303 or the like.

An exposure determining unit (determining unit) 407 determines exposure according to predetermined conditions from the stored target exposure (exposure correction value). Detailed processing related to the operations of the exposure calculation unit 406 and the exposure determination unit 407 will be described later with reference to the flowchart of FIG. A display control unit 408 is display control means for outputting an image reflecting the exposure determined by the exposure determination unit 407 to the display device 105 according to an instruction from the client CPU 301 .

(Subject detection processing/exposure determination processing)
Subject detection processing and exposure determination processing according to the present embodiment will be described below with reference to the flowchart shown in FIG. FIG. 5 is a flowchart showing an example of exposure determination processing of the information processing apparatus according to this embodiment. In the imaging system illustrated in FIG. 1, it is assumed that each device is powered on and the connection (communication) between the imaging device 101 and the client device 103 is established. In this state, it is assumed that the imaging system repeats imaging of a subject, transmission of image data, and image display on a display device at a predetermined update cycle. 5 is started in response to input of an image obtained by imaging a subject from the imaging apparatus 101 to the client CPU 301 of the client apparatus 103 via the network 102. do.

First, in step S<b>501 , the camera information (imaging information) obtained when the imaging device 101 captures an image of a subject and obtains an image is obtained from the camera information obtaining unit 404 .

In step S502, the subject detection unit 405 detects a subject (specific area). The detection method applied to the image differs between the face detection area and the human body detection area. For example, in the face detection region and the human body detection region, respective patterns corresponding to the characteristic portion of the face and the characteristic portion of the human body are stored in the client device 103 in advance. Detect regions. When the face area is detected, the face can be detected with high precision, and the face area and subjects other than the face can be clearly distinguished. However, if the direction of the face, the size of the face, the brightness of the face, etc. are not suitable conditions for face detection, the face region cannot be detected accurately. On the other hand, when detecting a human body, it is possible to detect a region in which a person exists regardless of the direction of the face, the size of the face, the brightness of the face, and the like.

As a method for detecting an object, a pattern (discriminator) created using statistical learning may be used as the pattern matching method described above, or a luminance gradient in a local region may be used as a method other than pattern matching. The configuration may be such that the subject is detected. That is, the detection method is not limited, and various methods such as detection based on machine learning and detection based on distance information can be employed.

Next, in step S<b>503 , the exposure calculation unit 406 calculates the average luminance value Iface of each face area and the average luminance value of each human body area Ibody based on the detection result obtained from the subject detection unit 405 . Specifically, based on the detection result obtained from the subject detection unit 405, the exposure calculation unit 406 calculates the detection position at which the face and the human body are detected and the information regarding the detection size using the following formulas (1) and (2). apply to In this embodiment, the unit of luminance value is calculated as a BV value in APEX (ADDITIVE SYSTEM OF PHOTOGRAPHIC EXPOSURE) unit.

Here, I(x, y) represents the luminance value of the two-dimensional coordinate position (x, y) in the horizontal direction (x-axis direction) and vertical direction (y-axis direction) in the image. Also, (v, h) represent the center coordinates at which the face and human body are detected, and k and l represent the detected size of the subject in the horizontal and vertical directions, respectively.

Next, in step S504, the exposure calculation unit 406 calculates a difference value ΔDiffface between a predetermined target luminance value Ifacetarget of the face region and Iface, as shown in Equation (3). Also, as shown in equation (4), a difference value ΔDiffbody between a predetermined target luminance value Ibodytarget of the human body region and Ibody is calculated.

Here, the target luminance value Ifacetarget of the face region and the target luminance value Ibodytarget of the human body region may be target values preset by the user or may be fixed values preset on hardware.

Next, in step S505, the exposure calculation unit 406 corrects the exposure based on the difference values ΔDiffface and ΔDiffbody, which are luminance information between the human body region and the face region, a predetermined threshold value Th, and the exposure value EVcurrent related to the current exposure. Calculate the amount (target exposure) EVcorrection. For example, the correction amount EVcorrection is calculated as in Equation (5). Note that EVcurrent is an APEX-converted EV value based on the subject brightness value (BV value), and is set based on a program diagram relating to exposure control, which is stored in advance in the client device 103 .

where the parameters β and γ are exposure correction values that are the difference between the current exposure value EVcurrent and the target exposure EVcorrection. The exposure correction value is a coefficient that affects the degree of correction (speed) when correcting the exposure to the underexposure side or the overexposure side centering on the current exposure value EVcurrent. β and γ are positive values and have a relationship of β<γ. The parameters β and γ are set as exposure correction values for the current exposure value EVcurrent when the difference ΔDiff is equal to or greater than the set threshold 2Th or Th. The larger the absolute value of ΔDiff, the larger the absolute value of the exposure correction value.

Next, in step S506, the exposure determination unit 407 counts the exposure correction values calculated for each face region and each human body region. FIG. 6 is a diagram showing an example of the result of counting the number of specific regions (the number of face regions and human body regions) corresponding to the calculated exposure correction value in the information processing apparatus according to the present embodiment. The number of regions is the sum of the number of face regions and the number of human body regions for which the exposure correction values have been calculated. In the case of FIG. 6, the sum of the face region and the human body region where the exposure correction value is -γ is 2, the sum of the face region and the human body region where the exposure correction value is +β is 2, and the face region where the exposure correction value is +γ. and the human body region is 1.

Next, in step S507, an exposure correction value to be set in the exposure determination unit 407 imaging device is determined. In the example of FIG. 6, the minimum absolute values of the exposure correction values are -β and +β. Here, the number of regions with the exposure correction value −β is set to 0 for convenience, but it is a value not calculated by the exposure calculation unit 406 . Therefore, the exposure correction value with the minimum absolute value may be determined before counting the number of regions in step S506. That is, since −β is not calculated by the exposure calculation unit 406, the exposure correction value to be set in the imaging apparatus 101 is determined to be +β, which is the exposure correction value with the minimum absolute value. In FIG. 6, if there are one or more areas with an exposure correction value of −β, the number of areas with −β and the number of areas with +β are compared to determine the exposure correction value with the largest exposure correction value. For example, when the number of -β regions is 1, the number of +β regions is larger, so the exposure correction value to be set in the imaging device 101 is determined to be +β. If the number of regions is the same, the exposure correction value to be set in the imaging apparatus 101 is determined based on the distribution of the calculated exposure correction value. For example, if the number of regions of −β is 2, the exposure determination unit 407 calculates the number of regions with positive exposure correction values and the number of regions with negative exposure correction values based on the exposure correction value of 0. That is, since there are 3 regions with positive exposure correction values and 4 regions with negative exposure correction values, the exposure correction value to be set in the imaging apparatus 101 is determined to be -β. If the distribution of the exposure compensation values is also equal between the positive exposure compensation value and the negative exposure compensation value, the conditions set by the user or the designer (for example, determining the negative exposure compensation value, etc.) Based on this, an exposure correction value to be set in the imaging apparatus 101 is determined.

In step S508, the exposure correction value determined by the exposure determination unit 407 is output to the imaging apparatus 101 via the network 102 by an output unit (not shown) and set. The imaging apparatus 101 determines exposure information such as exposure time and gain from the output exposure correction value, and executes exposure control. In this embodiment, the client apparatus 103 determines the exposure correction value, and the imaging apparatus 101 determines the exposure information. However, the exposure determination unit 407 may further determine exposure information including at least one of the exposure time, gain, and aperture value to be set in the imaging device 101 based on the exposure correction value.

As described above, the imaging system of this embodiment detects the face area and the human body area in the captured image, calculates the target exposure for each area, determines the exposure correction value from the target exposure, and sends the exposure correction to the imaging device. set the value. Therefore, when a plurality of specific areas (face area and human body area) are detected, the exposure value and exposure information of the imaging device can be more preferably controlled. To efficiently perform exposure correction of a face area when there are a plurality of faces of subjects, thereby efficiently making the face area suitable for face authentication. As a result, face authentication can be performed efficiently.

<Embodiment 2>
In this embodiment, a configuration will be described in which the face authentication unit 701 performs face authentication processing on a detected face area, and excludes the face area subjected to the face authentication process from subsequent exposure correction targets. Note that the configurations of the imaging device 101, the network 102, the input device 104, and the display device 105, which constitute the imaging system according to the present embodiment, are the same as those of the first embodiment described above, so description thereof will be omitted. As shown in FIG. 7, the client device 103 has a face authentication unit 701 added to the configuration of the information processing device according to the first embodiment.

Subject detection processing and exposure determination processing according to the present embodiment will be described below with reference to the flowchart shown in FIG. FIG. 8 is a flowchart showing an example of exposure determination processing of the information processing apparatus according to this embodiment. Note that the processing start timing is the same as in the first embodiment, so the description is omitted. Also, since each step other than steps S803 and S804 is the same as in the first embodiment, description thereof is omitted.

In step S803, information on the face area of the person authenticated by the face authentication unit 701 is obtained. Face authentication processing in the face authentication unit 701 will be described later.

In step S804, the face area and human body area of the authenticated person acquired in step S803 are excluded from the subject's face area detected in step S802. FIG. 9 is a diagram illustrating an example of processing for excluding a face-authenticated person of the information processing apparatus according to the present embodiment. Nine subject persons, A through I, are imaged in the acquired image of frame 50 . According to the information acquired from the face authentication unit 701, faces of persons B, D, F, and H have been authenticated. Therefore, in subsequent processing, persons B, D, F, and H are excluded from processing targets. Similarly, in the next frame 51, nine subjects A to I are captured in the acquired image. Face areas presumed to be persons B, D, F, and H authenticated in the previous frame are excluded from processing targets. A method of estimating that the person is the same person in frames 50 and 51 uses image person tracking or the like. The method is not particularly limited. Persons B, D, F, and H do not need to be exposed because they have already been authenticated. Subsequent processing is performed only for areas that have not been excluded.

Next, face authentication processing will be described with reference to the flowchart shown in FIG. FIG. 10 is a flowchart showing an example of face authentication processing of the information processing apparatus according to this embodiment. It should be noted that the start timing of the processing is started in response to input of an image obtained by imaging a subject, as in the flowchart of FIG. 8 .

In step S1001, the subject detection unit 405 detects a face area. Each pattern corresponding to the characteristic portion of the face is stored in advance on the client device 103 side, and the face area is detected by pattern matching based on this pattern. As a result, the face can be detected with high accuracy, and the face area and subjects other than the face can be clearly distinguished. However, if the direction of the face, the size of the face, the brightness of the face, etc. are not suitable conditions for face detection, the face region cannot be detected accurately.

Next, the face authentication unit 701 performs face authentication processing on the face area detected in step S1002.

Next, in step S1003, the face area and the authenticated person are linked and stored as data.

As described above, exposure control suitable for more efficient face authentication can be performed by excluding the face area for which face authentication has been completed from subsequent processing.

<Other embodiments>
The present invention can be implemented by reading and executing a program that implements one or more functions of the first embodiment described above. This program is supplied to a system or device via a network or storage medium and read and executed by one or more processors in the computer of the system or device. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

101 imaging device 102 network 103 client device 403 input image acquisition unit 405 subject detection unit 406 exposure calculation unit 407 exposure determination unit

Claims (17)

an acquisition unit that acquires an image captured by the imaging device;
a detection unit that detects a specific area from the image acquired by the acquisition unit;
a calculation unit that calculates an exposure correction value that is a difference between a current exposure value of the imaging device and a target exposure value based on the luminance information of the specific area detected by the detection unit, the detection unit comprising: A calculation unit that calculates a plurality of exposure correction values corresponding to luminance information for each of the plurality of specific regions when the detects a plurality of specific regions;
a determination unit that determines an exposure correction value having a minimum absolute value from the plurality of exposure correction values;
An information processing device comprising: The determination unit counts the number of the specific regions corresponding to the exposure correction value with the minimum absolute value when there are a plurality of exposure correction values with the minimum absolute value, and the number of the specific regions is the largest. 2. The information processing apparatus according to claim 1, wherein the exposure correction value having the minimum absolute value is determined by: When there are a plurality of exposure correction values having the largest number of the specific regions and the smallest absolute value, the determining unit determines based on the number of the specific regions corresponding to the exposure correction values larger than the absolute value. 3. The information processing apparatus according to claim 2, wherein the exposure correction value is determined. 4. The information processing apparatus according to any one of claims 1 to 3, further comprising an output unit that outputs the exposure correction value having the minimum absolute value determined by the determination unit to the imaging device. 5. The determining unit further determines exposure information including at least one of exposure time, gain and aperture value of the image based on the exposure correction value determined by the determining unit. The information processing device according to any one of . 6. The information processing apparatus according to claim 5, further comprising an output section that outputs the exposure information determined by the determination section to the imaging device. 7. The information processing apparatus according to any one of claims 1 to 6, wherein the specific area is an area of a face or a human body. An imaging device comprising the information processing device according to any one of claims 1 to 7. an acquisition step of acquiring an image captured by the imaging device;
a detection step of detecting a specific region from the image acquired in the acquisition step;
a calculating step of calculating an exposure correction value, which is a difference between a current exposure value of the imaging device and a target exposure value, based on the luminance information of the specific region detected in the detecting step, the detecting step a calculation step of calculating a plurality of exposure correction values corresponding to luminance information for each of the plurality of specific regions when detecting a plurality of specific regions in
a determination step of determining an exposure correction value having a minimum absolute value from the plurality of exposure correction values;
A method comprising: In the determining step, when there are a plurality of exposure correction values with the smallest absolute value, the number of the specific regions corresponding to the exposure correction value with the smallest absolute value is counted, and the number of the specific regions is the largest. 10. The method of claim 9, wherein the minimum absolute value of the exposure correction value is determined by . In the determining step, when there are a plurality of exposure correction values having the largest number of the specific regions and the smallest absolute value, based on the number of the specific regions corresponding to the exposure correction value larger than the absolute value 11. The method of claim 10, further comprising determining the exposure correction value. 12. The method according to any one of claims 9 to 11, further comprising an output step of outputting the minimum absolute exposure correction value determined in the determination step to the imaging device. 13. The determining step further determines exposure information including at least one of an exposure time, a gain and an aperture value of the image based on the exposure correction value determined in the determining step. A method according to any one of 14. A method according to claim 13, further comprising an outputting step of outputting said exposure information determined in said determining step to said imaging device. 15. A method according to any one of claims 9 to 14, wherein said specific region is a face or a human body region. A program for causing a computer to execute the method according to any one of claims 9 to 15. A computer-readable storage medium storing the program according to claim 16.

Images