JP2023075857A - Imaging apparatus, control method, program, and storage medium - Google Patents

Abstract

To suppress reduction in image quality of areas of high luminance in an image.SOLUTION: An imaging apparatus has an imaging unit 201 capable of setting exposure parameters for each pixel group composed of a plurality of pixels on an imaging surface, a determination unit 402 for determining, from an image captured by the imaging unit 201, a second image area that includes a first image area that exhibits a luminance of a predetermined value or higher and is larger than the first image area, and an exposure control unit 403 for uniformly controlling the image exposure parameters of a group of pixels corresponding to the second image area determined by the determination unit 402.SELECTED DRAWING: Figure 4

Description

The present invention relates to an imaging device, control method, program, and storage medium.

In Patent Document 1, an image with a wide dynamic range is captured by shortening the exposure time of a pixel group corresponding to a high-brightness region in an image and increasing the exposure time of a pixel group corresponding to a low-brightness region in the image. A technique for doing so is disclosed.

JP 2019-4521

A problem to be solved by the present invention is to suppress deterioration of image quality in a high-brightness region in an image.

To solve the above problems, an imaging apparatus according to an aspect of the present invention includes an imaging unit capable of setting an exposure parameter for each pixel group consisting of a plurality of pixels on an imaging surface, and an image captured by the imaging unit. a determination unit that determines a second image area that includes a first image area exhibiting luminance equal to or greater than a predetermined value and is larger than the first image area; and the exposure of the pixel group corresponding to the second image area. and an exposure controller that uniformly controls the parameters.

ADVANTAGE OF THE INVENTION According to this invention, the deterioration of the image quality of the area|region where the brightness|luminance is high in an image can be suppressed.

1 is a diagram showing an example of an imaging system according to a first embodiment; FIG. 1 is a diagram showing an example of the device configuration of an imaging device according to the first embodiment; FIG. FIG. 2 is a diagram showing an example of a device configuration of a client device according to the first embodiment; FIG. 1 is a diagram showing an example of a functional configuration of an imaging device according to a first embodiment; FIG. 4 is a flowchart showing an example of the operation of the imaging device according to the first embodiment; 4A and 4B are diagrams showing an example of an image, a first image area, and a second image area captured by the imaging device according to the first embodiment; FIG. The figure which shows an example of the functional structure of the imaging device which concerns on 2nd Embodiment. 9 is a flow chart showing an example of the operation of the imaging device according to the second embodiment; FIG. 7 is a diagram showing an example of data stored in a storage unit of an imaging device according to the second embodiment;

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. The embodiments described below are examples of means for realizing the present invention, and should be appropriately modified or changed according to the configuration of the apparatus to which the present invention is applied and various conditions, and the present invention is the following embodiments. is not limited to Also, a part of each embodiment described later may be appropriately combined.

<Embodiment 1>
(System configuration)
FIG. 1 is a diagram showing an example of the configuration of an imaging system including an imaging device according to this embodiment. The imaging system 100 has an imaging device 101 , a network 102 , a client device 103 , a display device 104 and an input device 105 .

The imaging device 101 can communicate with the client device 103 via the network 102 . The imaging device 101 captures an image of a subject, generates an image, and transmits the captured image to the client device 103 via the network 102 . A display device 104 and an input device 105 are connected to the client device 103 , and an image sent from the imaging device 101 is displayed on the display device 104 . An input device 105 is a keyboard, a mouse, or the like, and is used to operate the client device 103 . Note that operations to the client apparatus 103 include setting of imaging conditions of the imaging apparatus 101 via the network 102 and operations of PTZ (Pan Tilt Zoom).

In the present embodiment, the client device 103, the display device 104, and the input device 105 are separate units. good. In addition, the imaging apparatus 101 and the client apparatus 103 may be directly connected without needing to be connected via the network 102 . Furthermore, like a consumer camera having a touch panel display, the imaging device 101, the client device 103, the display device 104, and the input device 105 may all be integrated.

(Device configuration)
FIG. 2 is a diagram showing an example of the configuration of an imaging device according to this embodiment. The imaging device 101 has an imaging unit 201 , an encoder unit 202 , a network I/F 203 , a CPU (Central Processing Unit) 204 , a RAM (Random Access Memory) 205 and a ROM (Read Only Memory) 206 . The imaging optical system 200 is detachably attached to the imaging device 101 .

The imaging optical system 200 is a lens that collects light from an object onto an imaging surface of an imaging element 201a, which will be described later, and is composed of, for example, a zoom lens, a focus lens, and a blur correction lens.

In this embodiment, the imaging device 101 and the imaging optical system 200 are detachably provided as separate units, but the imaging device 101 may have the imaging optical system 200 as in a camera with an integrated lens.

An imaging unit 201 captures an image of a subject by the imaging optical system 200 and generates an image. The imaging unit 201 has an imaging element 201a, an amplifier 201b, and an image processing unit 201c. An exposure parameter can be set/changed for each pixel group consisting of a plurality of pixels (for example, 128×128 pixels) on the imaging plane. Exposure parameters are parameters related to exposure, and include exposure time, analog gain, and exposure value.

The imaging element 201a converts the light from the object condensed on the imaging surface by the imaging optical system 200 into an electric signal for each pixel and outputs the electric signal. The imaging device 201a captures an image of a subject with an exposure time for each pixel group according to the set exposure parameters. The imaging element 201a is an IC chip in which pixels made of photoelectric conversion elements such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor are arranged in a matrix. The image sensor 201a has high sensitivity mainly to visible light, and has high sensitivity to any one of red (R), green (G), and blue (B) for each pixel. It has some sensitivity. Therefore, it is possible to take a clear image of a subject bright with infrared light, such as a time period when sunlight exists and a place illuminated by infrared light illumination.

The amplifier 201b amplifies and outputs the electrical signal output from the imaging element 201a. The amplifier 201b is provided for each pixel or pixel group, and its signal amplification factor (analog gain) is set according to the exposure parameter for each pixel group.

The image processing unit 201c performs A/D conversion of the electrical signal, which is an analog signal output from the amplifier 201b, into a digital signal, and performs image processing including demosaicing processing, white balance processing, gamma processing, and the like to obtain a digital image. to generate The image processing unit 201c corrects the brightness of the image by amplifying or reducing the digital value of the image signal output from each pixel or pixel group for each pixel or pixel group.

The encoder unit 202 encodes the image data output from the imaging unit 201 (image processing unit 201c) into a predetermined file format such as Motion Jpeg, H264, H265.

A network I/F 203 transmits the encoded image data from the encoder unit 202 to the client device 103 via the network 102 . Note that the encoded image data may be stored in an internal storage device such as a RAM 205 or ROM 206, which will be described later, or a detachable storage medium such as an SD card (not shown). Also, these memories may be stored after being output by the image processing unit 201c. In that case, the image data is stored as RAW data before encoding.

A CPU 204 is a central processing unit that controls the imaging apparatus 101 .

A RAM 205 temporarily stores computer programs executed by the CPU 204 . Also, the RAM 205 provides a work area used when the CPU 204 executes processing. Also, the RAM 205 functions as a frame memory or as a buffer memory.

The ROM 206 stores programs and the like for the CPU 204 to control the imaging apparatus 101 .

FIG. 3 is a diagram showing an example of the configuration of a client device connected via the Internet to the imaging device according to this embodiment. The client device 103 is an information processing device having a CPU 301 , RAM 302 , ROM 303 , input I/F 304 , output I/F 305 and network I/F 406 .

A CPU 301 is a central processing unit that controls the client device 103 .

A RAM 302 stores programs and the like for the CPU 301 to control the client device 103 .

The ROM 303 stores programs and the like for the CPU 301 to control the client device 103 .

The input I/F 304 is an interface that is connected to the input device 105 and receives an operation for the client device 103 input from the user via the input device 105 .

An output I/F 306 is an interface for connecting to the display device 104 and displaying an image output from the imaging device 101 on the display device 104 .

A network I/F 306 is an interface for connecting to the image capturing apparatus 101 via the network 102 and inputting operation information for the image capturing apparatus 101 and receiving images output from the image capturing apparatus 101 .

(Functional configuration)
FIG. 4 is a diagram showing an example of the functional configuration of the imaging device according to the present embodiment. Of the functional blocks shown in FIG. 4, for functions realized by software, a program for providing the function of each functional block is stored in a memory such as the ROM 206 or the like. Then, it is realized by reading the program to the RAM 205 and executing it by the CPU 204 . For the functions realized by hardware, for example, by using a predetermined compiler, a dedicated circuit may be automatically generated on the FPGA from a program for realizing the function of each functional block. FPGA is an abbreviation for Field Programmable Gate Array. Also, a gate array circuit may be formed in the same manner as the FPGA and implemented as hardware. Also, it may be realized by an ASIC (Application Specific Integrated Circuit). Note that the configuration of the functional blocks shown in FIG. 1 is an example, and a plurality of functional blocks may constitute one functional block, or one of the functional blocks may be divided into blocks that perform a plurality of functions. good too.

The imaging device 101 has an imaging unit 201 , a specifying unit 401 , a determining unit 402 and an exposure control unit 403 .

The specifying unit 401 specifies a first image region exhibiting luminance equal to or higher than a predetermined value from the image captured by the imaging unit 201 . The first image region may be specified in pixel units, or the first pixel region may be specified in pixel group units.

A determination unit 402 determines an exposure area (second image area). The exposure area includes the first image area specified by the specifying unit 401 and is determined to be larger than the first image area. An exposure area is an image area corresponding to one or more pixel groups, and a uniform exposure parameter is applied to the pixel groups in the exposure area by the exposure control unit 403, which will be described later. The second image area may be the entire image, but preferably the ratio between the size of the first image area and the size of the second image area is a predetermined value, or a predetermined range. It is desirable to keep it inside. The predetermined value and predetermined range may be set by the user or the designer, or may be set based on the ratio of the first image area to the entire image. For example, if the ratio of the size of the first image area to the size of the entire image is 0.1, the ratio of the size of the first image area to the size of the second image area is set to 0.1. The second image area is determined so that the number is 5 or more and less than 1. Also, the size of the second image area may be equal to or less than a predetermined upper limit value and equal to or more than a predetermined lower limit value. The predetermined upper limit value is preferably the size of the entire image, and the predetermined lower limit value is preferably the size of the first image region, but can be set by the user or the designer.

The exposure control unit 403 uniformly controls the exposure parameters of the pixel groups corresponding to the exposure area (second image area). The exposure parameters of the pixel groups corresponding to the image regions other than the second image region determined by the determining unit 402 may be individually controlled or uniformly controlled.

(Description of operation)
FIG. 5 is a flow chart showing an example of the operation of the imaging device according to this embodiment. This operation is started by reading a program stored in a storage medium such as the ROM 206 in the imaging apparatus 101, and reading and executing the program by the CPU 204 using a memory such as the RAM 205 as a work area.

In step S501, the exposure control unit 403 performs control so that all pixel groups on the imaging surface of the imaging unit 101 have uniform exposure parameters, and an image of the subject is captured.

In step S502, the identifying unit 401 identifies the first image area from the image captured in step S501. The first image area is an area exhibiting luminance equal to or higher than a predetermined threshold (predetermined value), and the predetermined threshold can be set by a user or a designer. Alternatively, it may be calculated from the maximum luminance value that can be displayed by the display device 104, the maximum number of bits that can be transmitted over the network 102, the bandwidth of the network 102, and the like. The predetermined threshold is, for example, a high luminance value that is approximately 90% of the maximum luminance value.

In step S503, the determining unit 402 determines an exposure area (second image area) that includes the first image area and is larger than the first image area.

In step S<b>504 , the exposure control unit 403 uniformly controls the exposure parameters of the pixel group corresponding to the second image area determined by the determination unit 402 . In other words, the same exposure time and gain are set for all pixel groups corresponding to the second image area determined by the determining unit 402 . Further, the exposure time and gain to be controlled are calculated based on the luminance in the second image area. Further, the exposure parameters of pixel groups corresponding to image areas other than the second image area may be individually controlled based on the luminance output from each pixel group, or may be uniformly controlled in the same way as the second image area. may be controlled.

FIG. 6 is a diagram showing an example of an image, a first image area, and a second image area captured by the imaging device according to the present embodiment. In this embodiment, an image 601 includes a subject 602 and a light source 603 . A dashed line indicates a boundary of an image area 604 corresponding to each 6×4 pixel group arranged on the imaging surface of the imaging unit 201 . Image area 604 indicates the minimum area for which exposure parameters can be set. Therefore, exposure parameters cannot be set in an area smaller than the image area 604 in this embodiment. In this embodiment, it is assumed that the first area and the second image area are specified (determined) in units of corresponding pixel groups. FIG. 6A shows an image captured in step S501. FIG. 6B shows how the identification unit 401 identifies the first image area 606 in step S502. The first image area is a hatched area, and the average luminance value of the image in this area is higher than a predetermined threshold. FIG. 6C shows how the determining unit 402 determines the exposure area (second image area 609) in step S503. A second image area 609 is determined from the first image area 606 by the determination unit 402. In this embodiment, the ratio of the size of the first image area to the size of the second image area is 1/9. , and the second image region is determined so that the first image region corresponds to a 1×1 pixel group, while the second image region corresponds to a 3×3 pixel group. It is

Assume that the operation shown in the flowchart of FIG. 5 is not performed. In this case, the exposure parameter for each pixel group is calculated and set based on the luminance for each pixel group. Therefore, a pixel group corresponding to the first image area in which the light source 603 dominates is set with a higher gain than a pixel group corresponding to an image area in which the light source 603 does not exist. However, in the first image area, even an area having a luminance lower than that of the light source 603, such as a part of the subject 602, is imaged with a high gain set. As a result, an image is generated in which the area other than the light source 603 contains noise in the image within the first image area 606 .

On the other hand, according to the imaging apparatus according to the present embodiment, the light source 603 (that is, high luminance area) is reduced. As a result, the gain of the pixel group corresponding to the second image area is lowered, and noise in the first image area can be suppressed.

<Embodiment 2>
An imaging system including the imaging device according to the present embodiment and the device configuration of the imaging device according to the present embodiment are the same as those in the first embodiment, so description thereof will be omitted.

In this embodiment, images are captured while changing the ratio of the first image area and the second image area, and the images captured each time are analyzed until a predetermined number of times is reached. The analysis score is stored in the storage unit together with information about the pixel group and the value of the ratio between the first image area and the second image area. After the predetermined number of analyses, the determination unit reads the analysis score, the ratio value thereof, and the information on the pixel group from the storage unit, and determines the second image region.

(Functional configuration)
FIG. 7 is a diagram showing an example of the functional configuration of the imaging device 101 according to this embodiment. The imaging device 101 further has an analysis unit 701 and a storage unit 702, as compared with the first embodiment.

The analysis unit 701 analyzes the image captured by the imaging unit 201, detects subjects, and calculates the number of subjects. The calculated result is sent to the client device 103 . The analysis result by the analysis unit 701 is scored and stored in the storage unit 702 .

The storage unit 702 records the analysis score calculated by the analysis unit 701 . In this embodiment, the content of the analysis is assumed to include, for example, counting the number of people present in the image and authentication, but other image analysis may be used.

(Description of operation)
FIG. 8 is a flowchart showing an example of the operation of the imaging device according to this embodiment. Since each operation of steps S801 to S804 and S808 is the same as that of the first embodiment, description thereof is omitted.

In step S805, exposure control is performed by the exposure control unit 403 in step S804, and the analysis unit 701 performs image analysis on the image captured by the imaging unit 201. FIG. The analysis results are scored and stored in the storage unit 702 in association with image-related information such as the value of the ratio between the first image area and the second image area.

FIG. 9 shows an example of data stored in the storage unit 702 in the imaging apparatus according to this embodiment. As shown in FIG. 9, items such as the number of times, the value of the ratio, the exposure parameter, the analysis score, and the pixel group are linked and stored as one structure in the storage unit 702 . The example of FIG. 9 shows a case where a plurality of first image regions and a plurality of second image regions are respectively specified and determined. Each may be one.

The number of times indicates how many times the analysis by the analysis unit 701 is performed. The ratio value indicates the ratio value of each exposure area (ratio value between the size of the first image area and the size of the second image area). The exposure parameter indicates an exposure parameter (for example, shutter speed or gain) uniformly controlled for the pixel group corresponding to each exposure area. The analysis score indicates, for example, the number of subjects, the evaluation value of the image quality of the subjects, and the like. A pixel group indicates position information of each exposure area on the imaging surface of the imaging unit 201 . In this embodiment, the positions of the pixel groups corresponding to each exposure area are stored at the upper left position and the lower right position of the pixel group. In this embodiment, since it is assumed that the exposure area is rectangular as shown in FIG. 6, the position information is as shown in FIG. On the other hand, since the shape of the exposure area is not limited to this, position information corresponding to the shape of the exposure area is required.

In step S806, it is determined whether the image analysis in step S805 has been performed a predetermined number of times. The predetermined number of times is a value that can be set by the user or the designer. If the image analysis has not been performed the predetermined number of times, the process returns to step S803. In step S803, the determination unit 402 refers to the past ratio values stored in the storage unit 702, and redetermines the second image region so that the ratio values are not the same. If the image analysis has been performed the predetermined number of times, the process proceeds to step S807.

In step S807, the analysis score stored in the storage unit 702 is referred to, and data with an analysis score equal to or greater than a predetermined value is extracted. The predetermined value can be set by the user or designer. If there are multiple pieces of data with analysis scores equal to or higher than a predetermined value, the data with the largest analysis score is preferably extracted. In step S807, the determining unit 402 determines the exposure area based on the extracted data.

According to the imaging apparatus of the present embodiment, it is possible to set pixel groups and control exposure so as to further improve the image analysis score and image quality.

<Other embodiments>
The present invention can be implemented by reading and executing a program that implements one or more functions of the first embodiment described above. This program is supplied to a system or device via a network or storage medium and read and executed by one or more processors in the computer of the system or device. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

101 imaging device 201 imaging unit 402 determination unit 403 exposure control unit

Claims (20)

an imaging unit capable of setting exposure parameters for each pixel group consisting of a plurality of pixels on an imaging surface;
a determining unit configured to determine a second image area including a first image area exhibiting luminance equal to or higher than a predetermined value and larger than the first image area from the image captured by the imaging unit;
an exposure control unit that uniformly controls the exposure parameters of the pixel group corresponding to the second image area;
An imaging device having 2. The imaging apparatus according to claim 1, further comprising a specifying unit that specifies the first image region exhibiting luminance equal to or higher than the predetermined value from the image captured by the imaging unit. The determination unit determines the second image area such that a ratio of the size of the first image area to the size of the second image area is equal to or less than a predetermined upper limit value and equal to or more than a predetermined lower limit value. 3. The image pickup apparatus according to claim 1, wherein: 4. The method according to any one of claims 1 to 3, wherein the predetermined upper limit is the size of the first image area, and the predetermined lower limit is the size of the entire image. imaging device. 5. The determining unit determines the second image area based on a ratio between the size of the first image area and the size of the entire image. 1. The imaging device according to item 1. 6. The specifying unit specifies the first image region from an image captured by applying a uniform exposure parameter to pixel groups corresponding to the entire image. 10. The image pickup device according to claim 1. an analysis unit that analyzes the image captured by the imaging unit;
a storage unit for storing analysis results analyzed by the analysis unit;
further having
7. The method according to any one of claims 1 to 6, wherein the determination unit determines the second image region in which the analysis score of the analysis result stored in the storage unit is equal to or greater than a predetermined value. Imaging device. The imaging apparatus according to any one of claims 1 to 7, wherein the determination unit determines the second image region having the maximum analysis score stored in the storage unit. The storage unit stores the value of the ratio between the size of the first image area and the size of the second image area together with the analysis score of the analysis result, and the second image area controlled by the exposure control unit. and position information of the pixel group corresponding to the second image area in the image. The imaging device described. an imaging process in which an exposure parameter can be set for each pixel group consisting of a plurality of pixels on an imaging surface;
a determination step of determining a second image region that includes a first image region exhibiting luminance equal to or greater than a predetermined value and is larger than the first image region from the image captured by the imaging step;
an exposure control step of uniformly controlling the exposure parameters of the pixel groups corresponding to the second image area;
A control method for an imaging device having 11. The method of controlling an imaging apparatus according to claim 10, further comprising a specifying step of specifying the first image region exhibiting luminance equal to or higher than the predetermined value from the image captured in the imaging step. In the determining step, the second image area is determined such that the ratio of the size of the first image area to the size of the second image area is equal to or less than a predetermined upper limit and not less than a predetermined lower limit. 12. The method of controlling an imaging apparatus according to claim 10, wherein: 13. The imaging apparatus according to claim 10, wherein the upper limit is the size of the first image area, and the lower limit is the size of the entire image. control method. 14. The second image area according to any one of claims 10 to 13, wherein in the determining step, the second image area is determined based on a ratio of the size of the first image area to the size of the entire image. 2. The control method of the imaging device according to item 1. 15. The first image area according to any one of claims 10 to 14, wherein in the identifying step, a uniform exposure parameter is applied to pixel groups corresponding to the entire image to identify the first image region from an image captured. 11. A control method for the imaging device according to claim 1. an analysis step of analyzing the image captured in the imaging step;
a storage step of storing the analysis result analyzed in the analysis step;
further having
16. The method according to any one of claims 10 to 15, wherein in said determining step, said second image region in which an analysis score of the analysis result stored in said storing step is equal to or greater than a predetermined value is determined. A control method for an imaging device. 17. The method of controlling an imaging apparatus according to any one of claims 10 to 16, wherein in the determining step, the second image region having the maximum analysis score stored in the storing step is determined. . In the storing step, the value of the ratio between the size of the first image region and the size of the second image region is uniformly distributed to the second image region controlled in the exposure control step together with the analysis score of the analysis result. and position information of the pixel group corresponding to the second image area in the image. A control method for the described imaging device. A program for causing a computer to execute the imaging apparatus control method according to any one of claims 10 to 18. A computer-readable storage medium storing the program according to claim 19.

Images