JP2021164101A - Imaging apparatus, imaging system, control method, and program - Google Patents

Abstract

To provide an imaging apparatus, an imaging system, a control method, and a program that can set an appropriate strobe main flash amount even in the presence of a highly reflective object when firing a strobe.SOLUTION: An imaging apparatus 100 includes a polarization imaging element 102, which outputs image signals at four different polarization angles of 0°, 45°, 90°, and 135°, a light-emitting device 109, and a polarization calculation unit 104, which calculates polarization information from the image signals of the four polarization angles output from the polarization imaging element 102 while the light-emitting device 109 is pre-luminous, and an exposure control unit 106, which determines the amount of light emitted by the main light emission of the light-emitting device 109 on the basis of the polarization information calculated by the polarization calculation unit 104.SELECTED DRAWING: Figure 1

Description

The present invention relates to an image pickup device and an image pickup system, a control method, and a program, and more particularly to an image pickup device and an image pickup system having an image pickup device and a light emitting device having polarized pixels, a control method, and a program.

In strobe photography in which a conventional light emitting device is used for shooting, if a high-reflecting object such as a gold folding screen is present on the shooting screen, the weighting coefficient of the photometric area where the high-reflecting object is present is increased. The amount of light emitted at the time is calculated. For this reason, there is a problem that the amount of light emitted is insufficient for the main subject intended by the photographer, resulting in underexposure.

In order to solve such a problem, for example, in Patent Document 1, a region in the photometric area that corresponds to specific color information is determined to be a region with a gold folding screen, and the photometric value in that region is excluded from the calculation of the amount of light emitted.

On the other hand, as a technique for generating another image from an image acquired by imaging, a polarization imaging method is known in which information on various polarization components is acquired from an image obtained by imaging an object and various images are obtained from the information. Has been done. By using this polarized light imaging method, for example, by extracting a non-polarized component from an image obtained by imaging a vehicle, a clear image of the inside of the vehicle from which light from the windshield has been removed can be obtained. ..

Further, Patent Document 2 discloses a method of calculating the polarization intensity (intensity of reflected light) of a subject by obtaining polarization luminance information in three directions with respect to the subject. Further, Patent Document 3 discloses a method of estimating a surface normal (polarization angle having the strongest reflection) by obtaining polarization luminance information in three directions with respect to a subject.

Japanese Unexamined Patent Publication No. 2012-242676 Japanese Unexamined Patent Publication No. 2017-228910 JP-A-2017-72499

However, since it is determined that there is a high-reflecting object only in a region corresponding to specific color information as in Patent Document 1, a high-luminance subject that does not correspond to the specific color information is not determined as a high-reflecting object. As a result, when determining the amount of light emitted, such a highly reflective object is used in the calculation of photometry, and the amount of light emitted is insufficient for the main subject, resulting in underexposure.

On the other hand, when the main subject has a high-brightness subject that applies to the specific color information, the high-brightness subject information that should be used for calculating the photometric value is excluded, and the high-brightness subject is white. There was a problem of flying. A similar problem occurs when a region with a highly reflective object is determined based on the polarization intensity calculated by the method of Patent Document 2 and the surface normal estimated by the method of Patent Document 3.

Therefore, an object of the present invention is to provide an image pickup apparatus and an image pickup system, a control method, and a program capable of setting an appropriate amount of the main light emission of the strobe even if there is a highly reflective object at the time of strobe light emission.

The image pickup apparatus according to claim 1 of the present invention includes an image pickup element capable of outputting image signals having three or more polarization angles different from each other, a light emitting device, and output from the image pickup device during pre-emission of the light emitting device. It is characterized by including a calculation means for calculating polarization information from the image signals having three or more polarization angles, and a determination means for determining the amount of main light emission of the light emitting device based on the calculated polarization information. And.

The imaging system according to claim 8 of the present invention is an imaging system including an imaging device and a light emitting device externally connected to the imaging device, and the imaging device outputs image signals having three or more polarization angles different from each other. Calculation to calculate polarization information from an image pickup element to be output, a control means for controlling the light emitting device, and an image signal having three or more polarization angles output from the image pickup element during pre-emission of the light emitting device. The light emitting device includes means and a determining means for determining the amount of main light emitted by the light emitting device based on the calculated polarization information, and the light emitting device responds to an instruction of the pre light emission from the control means. It is characterized by including a pre-light emitting means for emitting light and a main light emitting means for performing the main light emission in response to an instruction of the main light emission at the determined light emitting amount from the control means.

According to the present invention, it is possible to set an appropriate amount of main light emission of the strobe even if there is a highly reflective object at the time of light emission of the strobe.

It is a block diagram which shows the hardware structure of the image pickup apparatus which concerns on Example 1. FIG. It is a figure which shows the example of the arrangement of four kinds of polarization filters provided on the pixel of the polarization image pickup device in FIG. It is a figure explaining the calculation method of the polarization angle dependent component by the polarization calculation part in FIG. It is a figure explaining the method of obtaining surface normal information which concerns on Example 1. FIG. It is a flowchart of the strobe photography process which concerns on Example 1. FIG. It is a figure which shows the example which there is a small high-reflecting object inside a photometric frame. It is a figure which shows the example which there is a gold folding screen as a large high-reflecting object inside a photometric frame. It is the figure which extracted the gold folding screen of FIG. 6B. It is a flowchart of the strobe photography process which concerns on Example 2. It is a block diagram which shows the hardware structure of the image pickup apparatus which concerns on Example 3. FIG.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Example 1)
Hereinafter, embodiments of the present invention will be described.

FIG. 1 is a block diagram showing a hardware configuration of the image pickup apparatus 100 according to the present embodiment.

As shown in FIG. 1, the image pickup apparatus 100 includes an optical lens 101, a polarization image pickup element 102, an image acquisition unit 103, a polarization calculation unit 104, a photometric value calculation unit 105, an exposure control unit 106, an image processing unit 107, and a signal output unit. It includes 108 and a light emitting device 109.

The optical lens 101 is an optical lens for taking in the light of a subject, forming an image of the light on the polarized light image sensor 102, and entering the light.

The polarized light imaging device 102 includes a plurality of pixels that receive incident light from the optical lens 101, convert it into an electric signal, and output it. Further, the polarizing image sensor 102 includes four types of polarizing filters provided on the plurality of pixels. The type of the polarizing filter included in the polarizing image sensor 102 may be three or more.

Here, FIG. 2 shows an example of an arrangement of four types of polarizing filters included in the polarizing image sensor 102.

As shown in FIG. 2A, the polarized light image sensor 102 is composed of a plurality of unit regions 201. Each unit region 201 includes four pixels of the polarizing image sensor 102, and each of the four pixels is covered with a different type of polarizing filter. Specifically, the pixels located at the upper left, lower left, lower right, and upper right of each unit region 201 are covered with polarizing filters having polarization angles of 0 °, 45 °, 90 °, and 135 °, respectively. In this way, the polarizing image sensor 102 having four pixels (hereinafter, simply referred to as “polarizing pixels”) covered with different types of polarizing filters for each unit region 201 has images of four polarization angles having different polarization states from each other. It is possible to output a signal (hereinafter, simply referred to as "polarization angle image"). In this embodiment, various polarization information of the subject is acquired from the images of these four polarization angles.

As shown in FIG. 2B, the polarized light imaging device 102 may further include a Bayer-arranged color filter.

In the example of FIG. 2B, the polarized light imaging device 102 is composed of a unit region 202 of a plurality of Bayer arrays. Each unit area 201 includes four unit areas 201, and different color filters cover each of the four unit areas 201. Specifically, the color filters R, Gb, B, and Gr cover the unit areas 201 located at the upper left, lower left, lower right, and upper right of each unit area 202, respectively.

The arrangement of the polarizing filter and the color filter in this embodiment is not limited to the example shown in FIG. Further, the polarized light image sensor 102 may have a configuration in which the polarizing filter can be inserted and removed with respect to the optical path of the light flux incident on the image sensor.

The image acquisition unit 103 captures the video signal output from the polarization image sensor 102, and outputs an image of each polarization angle to the polarization calculation unit 104. Further, the image acquisition unit 103 outputs a video signal for recording to the image processing unit 107.

Based on the video signal output from each polarized pixel of the polarized light image sensor 102, the polarized light calculation unit 104 sets four polarized pixels having polarization angles of 0 °, 45 °, 90 °, and 135 ° in each unit region 201. Pixel outputs corresponding to the above four polarization angles are obtained as a group. Further, the polarization calculation unit 104 obtains images of four polarization angles having different polarization states from the pixel outputs corresponding to the obtained four polarization angles, and calculates various polarization information of the subject. Here, the polarization information includes a pixel-level difference value output from each polarization pixel, a polarization angle-dependent component, surface normal information, and the like.

FIG. 3 is a diagram illustrating a method of calculating the polarization angle-dependent component Ip−p by the polarization calculation unit 104.

As described with reference to FIG. 2A, four polarized pixels having different polarization angles are arranged in each unit region 201 in the polarized light image sensor 102. Therefore, four different polarization angles of 0 °, 45 °, 90 °, and 135 ° are output from the polarization image sensor 102 to the polarization calculation unit 104 via the image acquisition unit 103 by imaging.

When the brightness values of one pixel (polarized pixel included in the same unit region 201) corresponding to each other in these four input images are plotted, they can be represented as shown in FIG.

Therefore, first, the polarization calculation unit 104 uses an optimization technique such as the least squares method for the function I (θ) capable of expressing a sine function or a cosine function having a period of 180 ° to obtain the brightness values of the four polarization angles θ in FIG. Let it fit.

Next, the polarization calculation unit 104 obtains the maximum brightness value Imax and the minimum brightness value Imin of the function I (θ) after fitting, and calculates Ip−p = Imax-Imin, which is a polarization angle-dependent component of the subject. do. This polarization angle-dependent component is a numerical value that serves as an index for determining how much the subject is reflected by the light source.

In this embodiment, the luminance values of the four polarization angles θ are used, but if the luminance values of three or more polarization angles θ are known, Ip-p can be similarly calculated by using an optimization technique such as the least squares method. Can be calculated.

Next, with reference to FIG. 4, a method for obtaining surface normal information according to this embodiment will be described.

FIG. 4A is a diagram showing the relationship between the surface normal and the angle formed by the incident light on the image pickup apparatus 100. The light incident on the subject at the incident angle θ0 receives the reflectance of the subject and is incident on the image pickup apparatus 100 at the same reflection angle θ1. At this time, in the case of oblique specular reflection as shown in FIG. 4A, the amplitude Rp of P-polarized light, which is a polarization component for vibration in the incident surface, and S-polarized light, which is a polarization component for light vibrating perpendicular to the incident surface. It is known that the amplitude Rs of No. 4 exhibits an incident angle dependence as shown in FIG. 4 (b).

Therefore, the polarization calculation unit 104 estimates the rough surface normal information by calculating the ratio of the amplitudes Rp and Rs. This surface normal information is a numerical value that serves as an index for determining the polarization angle with the strongest reflection from the subject.

The photometric value calculation unit 105 calculates the photometric value from the polarization information obtained by the polarization calculation unit 104. The operation of calculating the metering value using the polarization information will be described later.

The exposure control unit 106 calculates the brightness of the subject based on the photometric value calculated by the photometric value calculation unit 105. The exposure control unit 106 further controls the iris of the optical lens 101, the gain of the polarized light image sensor 102, the shutter speed, and the like based on the calculated brightness of the subject so that the subject to be photographed has an appropriate exposure state. Take control. If the subject is dark or backlit and the strobe light is emitted during shooting using the light emitting device 109, the amount of light emitted is calculated from the brightness of the subject based on the photometric value calculated by the photometric value calculation unit 105 at the time of pre-flash. And also controls the light emitting device 109.

The image processing unit 107 performs arbitrary composition processing on the images of each polarization angle, and also performs the same development processing as a normal video signal such as debayer processing and gamma correction. For example, the polarization calculation unit 104 may determine which polarization angle image has the smallest reflection, and may perform a synthesis process of selecting and outputting an image of any one polarization angle from the result. Further, by weighted averaging the images of each polarization angle, a compositing process may be performed to generate an image for one frame in which the reflection state of the subject is arbitrarily controlled. After performing the compositing process, predetermined image processing such as debayer processing, gamma correction, knee correction, and noise reduction is performed as described above. In this embodiment, it is not necessary to use a particularly fixed image processing of the video output to the signal output unit 108, and the present embodiment is not limited to this example. Further, the image processing unit 107 may output the image of each polarization angle as it is without performing the above-mentioned processing.

The signal output unit 108 records the video signal received from the image processing unit 107 in a storage device or storage medium (not shown in FIG. 1) of the image pickup device 100, or outputs the video signal to a monitor (not shown) in the same figure. Or something.

The light emitting device 109 is a strobe mounted on a digital camera or the like. Also called flash or speedlight.

The light emitting device 109 performs a light emitting operation (main light emission) at the timing of the operation of capturing an image with the amount of light emitted calculated by the exposure control unit 106.

Next, the strobe photography process according to this embodiment will be described with reference to the flowchart of FIG.

In this process, the exposure control unit 106 reads and executes a program held in a ROM (not shown) which is not shown.

In step S501, the user waits for a strobe shooting instruction to the image pickup apparatus 100, receives the instruction, and proceeds to step S502. Examples of the strobe shooting instruction here include full pressing of the release button in the strobe shooting mode.

In steps S502 and S503, an autofocus operation for adjusting the position of the focus lens in the optical lens 101 is performed by a known method. In the case of manual focus, the process may be performed directly from step S501 to step S504.

In step S504, in order to determine the amount of light emitted by the light emitting device 109 at the time of main light emission, the light emitting device 109 is instructed to perform pre-light emission. During the pre-emission started by the light emitting device 109 according to this instruction, the image of the subject is imaged and acquired by the polarization image sensor 102 in step S505.

In step S506, it is determined whether or not the main subject is detected by the image processing unit 107 from the image captured in step S505. Various methods such as face detection and detection of a subject located substantially in the center can be applied to the detection of the main subject, but the method is not limited in this embodiment.

When the main subject is detected (YES in step S506), the polarization information (polarization angle-dependent component) of the region where the main subject is detected in step S507 (hereinafter referred to as the main subject region) is acquired from the polarization calculation unit 104. After that, the process proceeds to step S509. Here, the case where the polarization angle-dependent component is large is synonymous with the case where the light reflected from the light emitting device 109 is large.

On the other hand, when the main subject is not detected (NO in step S506), the process proceeds to step S508, and the polarization information of the entire image captured in step S505 is acquired from the polarization calculation unit 104. In this case, the amount of reflective objects on the subject is detected based on the polarization angle-dependent component of the entire image captured in step S505.

In step S509, the amount of light emitted by the light emitting device 109 at the time of the main light emission and the exposure of the optical lens 101 and the polarized light image sensor 102 are determined. At this time, if the photometric frame for determining the exposure includes a region in which the value of the polarization angle-dependent component acquired in step S507 or step S508 is larger than a predetermined value, that region is excluded from the calculation of the photometric value. Thereby, it is possible to determine an appropriate exposure for a subject including a highly reflective object.

In step S510, the exposure state of the optical lens 101 and the polarized light image sensor 102 is controlled so that the exposure is determined in step S509, and the light emitting device 109 is subjected to the main light emission with the amount of light emitted determined in step S509. Instruct. During the main light emission that the light emitting device 109 starts according to this instruction, the polarized light imaging element 102 captures and acquires an image of the subject in step S511, and the main process ends.

FIG. 6A is a diagram showing a case where the light measuring frame 601 is set from the face to the chest of a person wearing a small high-reflecting object 602 on the neck.

When the main emission amount and exposure of the light emitting device 109 are determined based on all the pixel values in the photometric frame 601 of the image acquired at the time of pre-emission as in the conventional case, the pixel level is affected by the high reflector 602. It becomes large and the subject is recognized as bright. Therefore, the determined main light emission amount and exposure are reduced, and the main subject in the image acquired at the time of main light emission becomes dark.

On the other hand, in this embodiment, in step S509 of FIG. 5, the region having a large polarization angle-dependent component in the photometric frame 601 is excluded from the calculation of the photometric value as the region where the highly reflective object 602 exists, and the main light emission of the light emitting device 109 Determine the amount and exposure. As a result, the main subject can be photographed with an appropriate brightness at the time of the main light emission.

As described above, the image pickup apparatus 100 according to the present embodiment calculates the polarization angle-dependent components from the images of the four polarization angles output from the polarization image pickup element 102 at the time of pre-emission when performing strobe photography, and the photometric frame. The photometric value is calculated by excluding the region where the polarization angle-dependent component is large. As a result, even if the photometric frame contains a highly reflective object, strobe photography can be performed under appropriate exposure conditions and light emission amount.

In this embodiment, the method of arranging the polarizing filters of the polarizing image sensor 102 and the presence or absence of the color filter may be arbitrary. Further, in this embodiment, the region of the polarization angle-dependent component larger than the predetermined value included in the photometric frame is excluded from the calculation of the photometric value, but such a region is weighted and used in the calculation of the photometric value. May be good.

Further, in this embodiment, the region excluded from the photometric frame when calculating the photometric value is a region having a large polarization angle-dependent component, but a region having a large plane normal information may also be used.

(Example 2)
In the first embodiment, the region where the polarization angle-dependent component is larger than the predetermined value is uniformly excluded from the calculation of the photometric value as the region where the high reflector exists, but in this embodiment, when the subject has a gold folding screen, the gold folding screen is excluded. Exclude the area from the calculation of the metering value.

Therefore, in the strobe photography process of this embodiment, a step of determining whether or not the highly reflective object is a gold folding screen using the polarization information is added.

Therefore, since the hardware configuration of the image pickup apparatus 100 of this embodiment and the basic operation of the strobe photography process are the same as those of the first embodiment, the same numbering is used for the same configuration and steps, and duplicate explanations are given. Omit.

In Example 1, it was described in Example 1 that surface normal information of a subject can be acquired as polarization information.

As shown in FIG. 6B, when the subject included in the photometric frame 603 has a large gold folding screen 604, which is a high-reflecting object, in addition to the person, in order to determine whether the high-reflecting object is the gold folding screen 604. Obtain the surface normal information of the image captured in step S505.

FIG. 6C is an extracted view of the gold folding screen 604 of FIG. 6B. As shown in regions 605 and 606 of FIG. 6C, in general, the folding screen surfaces are alternately bent at a certain angle in order to make the folding screen self-supporting. Therefore, the surface normal information of the region of the gold folding screen 604 included in the image captured in step S505 is periodically replaced as shown by the arrow shown in FIG. 6C.

Therefore, in this embodiment, the region having the color information peculiar to the gold folding screen is extracted as the gold folding screen region (specific region) from the entire captured image, and the surface normal information of the gold folding screen region is acquired. When the acquired surface normal information is replaced with a certain period, it is determined that the gold folding screen region is the region of the gold folding screen 604. The method for extracting the gold folding screen region is not limited to the above method, and any known method may be used.

By excluding the area determined to be the gold folding screen 604 from the calculation of the photometric frame 603, it is possible to shoot the subject so as to have an appropriate brightness.

Next, the strobe photography process according to this embodiment will be described with reference to the flowchart of FIG. 7.

This process differs from the strobe photography process of Example 1 in that steps S701 to S708, which will be described later, are provided instead of steps S506 to S509 of FIG.

In step S701, it is determined whether or not the gold folding screen region is tentatively determined (extracted) from the entire image captured during the pre-emission by the method described above using FIGS. 6B and 6C.

When the gold folding screen region is tentatively determined (YES in step S701), the polarization information (plane normal information) of the gold folding screen region is acquired from the polarization calculation unit 104 in step S702. On the other hand, when the gold folding screen region is not tentatively determined (NO in step S701), the process proceeds to step S706, the polarization information is obtained in the same manner as in the first embodiment, and the metering value is calculated.

When it is determined in step S703 that the surface normal information is periodically exchanged in the gold folding screen region, it is determined in step S704 that the subject in the gold folding screen region is a gold folding screen. On the other hand, if the surface normal information in the gold folding screen area is uniform (the gold folding screen area is a plane), or if the surface normal information in the gold folding screen area is a random value without periodic replacement, gold It is determined that the subject in the folding screen area is not a gold folding screen. In this case, the process proceeds to step S706, the polarization information is obtained in the same manner as in the first embodiment, and the metering value is calculated.

In step S705, the gold folding screen region is set as a region excluded from the calculation in the photometric frame 603, and the polarization information excluding the set region from the photometric frame 603 in step S706 is acquired from the polarization calculation unit 104. As a result, even if there is a high-reflecting object such as a gold folding screen, it is possible to take a picture in which the main subject is accurately exposed. In addition to excluding it from the calculation in the metering frame 603, the gold folding screen area may be weighted and used in the calculation of the metering value.

In step S707, after determining the amount of light emitted by the light emitting device 109 at the time of main light emission and the exposure of the optical lens 101 and the polarized light imaging element 102 based on the polarization information acquired in step S706, the process proceeds to step S510.

As described above, the image pickup apparatus 100 according to the present embodiment extracts the gold folding screen region from the entire image output from the polarized image pickup element 102 when performing strobe photography, and the surface normal line calculated from the gold folding screen region. Based on the information, it is determined whether the subject in the gold folding screen area is a gold folding screen. As a result, it is possible to accurately determine whether or not there is a gold folding screen in the photometric frame, and it is possible to perform strobe photography under appropriate exposure conditions and light emission amount based on the determination result.

In this embodiment as well, as in the first embodiment, the method of arranging the polarizing filters of the polarizing image sensor 102 and the presence or absence of the color filter may be arbitrary. Further, the polarization angle-dependent component may not only be excluded from the calculation of the photometric value, but may be weighted and used in the calculation, and is not limited to this embodiment.

(Example 3)
The image pickup device 100a (FIG. 8) according to the present embodiment includes an optical lens 801 and an image pickup element 802, an image acquisition section 803, an image processing section 804, and a signal output section 805, and includes an image processing section 107 and a signal output section 108. It is different from the image pickup apparatus 100 according to the first and second embodiments in that it does not have.

That is, in Examples 1 and 2, the output of the polarized light image sensor 102 was used not only for calculating the photometric value, but also for displaying on a monitor and recording on a storage medium. On the other hand, in this embodiment, the output of the polarized image sensor 102 is used only for the purpose of calculating the photometric value, and the output of the image sensor 802 is used for other purposes.

Therefore, since the hardware configuration of the image pickup apparatus 100a of this embodiment and the basic operation of the strobe photography process are the same as those of the first embodiment, the same numbering is used for the same configuration and steps, and duplicate explanations are given. Omit.

FIG. 8 is a block diagram showing a hardware configuration of the image pickup apparatus 100a according to the present embodiment.

The optical lens 801 is an optical lens for taking in the light of a subject, forming an image of the light on the image pickup device 802, and entering the light. The configurations of the two optical lenses 101 and 801 may be different, and the focal length and the aperture need not be the same.

The image pickup device 802 is an image pickup device that does not have a polarizing filter. Generally, it is equipped with a color filter, but this is not the case depending on the application.

The image acquisition unit 803 outputs the output from the image sensor 802 to the image processing unit 804.

The image processing unit 804 performs the same development processing as the image processing unit 107 described in the first embodiment on the output from the image sensor 802 from the image acquisition unit 803, and the obtained video signal is transmitted by the signal output unit 805. Record on a storage medium or output to a monitor.

Similar to the cases of Examples 1 and 2, the polarization calculation unit 104 calculates the polarization information, and the photometric value calculation unit 105 calculates the photometric value. However, when it is determined that the polarization information is not necessary, such as when it is determined that the reflection of the subject is small, the photometric value calculation unit 105 may calculate the photometric value using the output from the image sensor 802. ..

The light emitting device 109 performs the main light emission at the time of shooting with the light emitting amount calculated by the exposure control unit 106.

As described above, the polarizing image sensor 102 and the ordinary image sensor 802 having no polarizing filter are provided. As a result, when removing the region of the polarization angle-dependent component larger than the predetermined value (the region including the highly reflective object) included in the photometric frame from the calculation of the photometric value, the photometric value is calculated using the polarized light image sensor 102. Perform strobe photography under appropriate exposure conditions. On the other hand, in the case of the conventional measurement value calculation, the measurement value can be calculated more accurately by using the output of the normal image sensor 802 without a polarizing filter for the calculation of the measurement value.

Although the image pickup device 100a of the present embodiment has a configuration having a polarized image sensor 102 and an image sensor 802, two or more image pickup elements including the polarized image sensor 102 and another image sensor having no polarizing filter. The configuration is not limited to this as long as it has. Further, in the actual configuration, the image acquisition units 103 and 803, the image processing unit 804, and the like can be covered by one IC.

Further, the light emitting device 109 is built in the main body of the image pickup device 100 or the image pickup device 100a in Examples 1 to 3, but is not limited thereto. For example, the light emitting device 109 may be a device that can be externally connected to the image pickup device 100 or the image pickup device 100a. In this case, the imaging system according to the present invention is configured with the imaging device 100 and the light emitting device 109 connected, and the pre-light emission instruction from the exposure control unit 106 and the amount of light emitted determined in steps S509 and S707 are used. The light emitting device 109 emits light in response to the main light emitting instruction of.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiment is supplied to the system or device via a network or various storage media, and the computer (or CPU, MPU, etc.) of the system or device reads and executes the program code. It is a process to do. In this case, the program and the recording medium that stores (stores) the program constitute the present invention.

100 Image sensor 101 Optical lens 102 Polarized image sensor 103 Image acquisition unit 104 Polarization calculation unit 105 Photometric value calculation unit 106 Exposure control unit 107 Image processing unit 108 Signal output unit 109 Light emitting device

Claims (10)

An image sensor capable of outputting image signals with three or more polarization angles that are different from each other,
Light emitting device and
A calculation means for calculating polarization information from image signals having three or more polarization angles output from the image sensor during pre-emission of the light emitting device.
An imaging device including a determining means for determining the amount of main light emitted by the light emitting device based on the calculated polarization information. The calculation means calculates a polarization angle-dependent component as the polarization information, and obtains the polarization angle-dependent component.
The imaging device according to claim 1, wherein the determination means calculates the amount of light emitted by using the polarization angle-dependent component. The calculation means calculates surface normal information as the polarization information, and
The imaging device according to claim 1, wherein the determination means calculates the amount of light emitted by using the surface normal information. The determination means is
The imaging device according to claim 3, wherein when the surface normal information is periodically replaced in a specific region, the specific region is weighted when the calculation of the amount of light emission is performed. The determination means is
The imaging apparatus according to claim 3, wherein when the surface normal information is periodically replaced in a specific region, the specific region is excluded when calculating the amount of light emitted. The image pickup apparatus according to any one of claims 1 to 5, wherein the determination means further determines the exposure of the image pickup device at the time of the main light emission based on the calculated polarization information. A second image sensor different from the image sensor is further provided.
The determination means according to any one of claims 1 to 5, wherein the determination means further determines the exposure of the second image pickup device at the time of the main light emission based on the calculated polarization information. Image sensor. An imaging system consisting of an imaging device and a light emitting device externally connected to the imaging device.
The image pickup device
An image sensor capable of outputting image signals with three or more polarization angles that are different from each other,
A control means for controlling the light emitting device and
A calculation means for calculating polarization information from image signals having three or more polarization angles output from the image sensor during pre-emission of the light emitting device.
A determination means for determining the amount of main emission of the light emitting device based on the calculated polarization information is provided.
The light emitting device is
A pre-emission means that performs the pre-emission in response to an instruction of the pre-emission from the control means,
An imaging system comprising: a main light emitting means for performing the main light emission in response to an instruction of the main light emission at the determined light emission amount from the control means. It is a control method of an image pickup device including an image pickup device that outputs image signals having three or more polarization angles different from each other and a light emitting device.
A calculation step of calculating polarization information from the image signals of the three or more polarization angles output from the image sensor during the pre-emission of the light emitting device, and
A control method comprising a determination step of determining the amount of main emission of the light emitting device based on the calculated polarization information. A program comprising executing the control method according to claim 9.

Images