WO2021251260A1 - Dispositif d'imagerie et procédé d'imagerie - Google Patents

Dispositif d'imagerie et procédé d'imagerie Download PDF

Info

Publication number
WO2021251260A1
WO2021251260A1 PCT/JP2021/021165 JP2021021165W WO2021251260A1 WO 2021251260 A1 WO2021251260 A1 WO 2021251260A1 JP 2021021165 W JP2021021165 W JP 2021021165W WO 2021251260 A1 WO2021251260 A1 WO 2021251260A1
Authority
WO
WIPO (PCT)
Prior art keywords
transmittance
image
exposure period
exposure
image pickup
Prior art date
Application number
PCT/JP2021/021165
Other languages
English (en)
Japanese (ja)
Inventor
賢二 和嶋
Original Assignee
ソニーグループ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ソニーグループ株式会社 filed Critical ソニーグループ株式会社
Publication of WO2021251260A1 publication Critical patent/WO2021251260A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70—Circuitry for compensating brightness variation in the scene
    • H04N23/75—Circuitry for compensating brightness variation in the scene by influencing optical camera components

Definitions

  • This technology relates to an image pickup device and an image pickup method.
  • an HDR (High Dynamic Range) mode and a flicker reduction mode are executed.
  • the HDR mode long exposure and short exposure are repeated every two lines (pixel rows).
  • the HDR image is acquired by synthesizing the two types of images obtained at each exposure time.
  • the flicker reduction mode the total of the long and short exposure times is set to the same time as the flicker cycle.
  • the long-exposure image and the short-exposure image are combined in continuous frames. As a result, an image in which the flicker phenomenon is canceled is acquired (paragraphs [0008] [0035] [0036], etc. in the specification of Patent Document 1).
  • an object of the present technology is to provide an image pickup device and an image pickup method capable of acquiring an image having a wider dynamic range while reducing the flicker phenomenon.
  • the image pickup apparatus includes a transmittance adjusting unit, an image pickup element, an exposure period setting unit, a transmittance setting unit, and a compositing unit.
  • the transmittance adjusting unit adjusts the transmittance of light.
  • the image pickup device performs imaging by exposing the light transmitted through the transmittance adjusting unit.
  • the exposure period setting unit sets a first exposure period and a second exposure period shorter than the first exposure period based on the flicker period of the light transmitted through the transmittance adjusting unit.
  • the transmittance setting unit determines the first transmittance when the image pickup is executed in the first exposure period and the second transmittance when the image pickup is executed in the second exposure period.
  • the compositing unit is acquired by the first image acquired by the first exposure period and the image pickup in the first transmittance, and by the image pickup in the second exposure period and the second transmittance. Is combined with the second image.
  • the first exposure period and the second exposure period shorter than the first exposure period are set based on the flicker cycle of the light to be imaged. Further, the first transmittance and the second transmittance are set as the transmittance corresponding to each of the first exposure period and the second exposure period. This makes it possible to acquire an image having a wider dynamic range while reducing the flicker phenomenon by synthesizing the first image and the second image.
  • the exposure period setting unit may set the second exposure period based on the flicker cycle.
  • the synthesizing unit may generate an HDR (High Dynamic Range) image by synthesizing the first image and the second image.
  • HDR High Dynamic Range
  • the exposure period setting unit may set the second exposure period to an integral multiple of 1/2 of the flicker cycle.
  • the exposure period setting unit may set the first exposure period to an integral multiple of 1/2 of the flicker cycle.
  • the transmittance setting unit may set the first transmittance to a value larger than the second transmittance.
  • the transmittance setting unit may set the first transmittance to 100%.
  • the transmittance setting unit may set the first transmittance to the same value as the second transmittance.
  • the image pickup apparatus further sets an exposure amount setting unit for setting a first exposure amount for acquiring the first image and setting a second exposure amount for acquiring the second image. May be provided.
  • the transmittance setting unit may set the second transmittance based on the set second exposure amount and the set second exposure period.
  • the transmittance setting unit may set the first transmittance based on the set first exposure amount and the set first exposure period.
  • the transmittance adjusting unit may adjust the transmittance according to the applied voltage.
  • the transmittance adjusting unit may be a liquid crystal ND (Neutral Density) filter.
  • the image pickup device may further include a detection unit that detects the flicker cycle of the light transmitted through the transmittance adjusting unit.
  • the imaging method is an imaging method executed by an imaging apparatus, and is a first exposure based on a flicker cycle of light transmitted through a transmittance adjusting unit capable of adjusting the transmittance of light. It includes setting a period and a second exposure period shorter than the first exposure period.
  • the first transmittance when the image pickup is performed in the first exposure period is set by the image pickup device that executes the image pickup by exposing the light transmitted through the transmittance adjusting unit.
  • the image sensor sets a second transmittance when imaging is performed in the second exposure period.
  • a first image acquired by imaging with the first exposure period and the first transmittance, and a second image acquired by imaging with the second exposure period and the second transmittance. Is synthesized.
  • FIG. 1 is a block diagram showing a functional configuration example of the image pickup apparatus 100 according to an embodiment of the present technology.
  • the image pickup device 100 shown in FIG. 1 is a digital camera (digital still camera, digital video camera) capable of capturing still images and moving images. Further, the image pickup apparatus 100 is capable of reproducing the recorded image data.
  • the specific configuration of the image pickup apparatus 100 is not limited, and may be arbitrarily designed.
  • the image pickup apparatus 100 includes a lens system 10, a liquid crystal ND (Neutral Density) filter 11, an image pickup element 12, an analog signal processing circuit 13, an A / D conversion circuit 14, and digital signal processing. It has a circuit 15. Further, the image pickup apparatus 100 includes a lens driver 16, a liquid crystal ND driver 17, a timing generator (TG: Timing Generator) 18, a recording unit 19, and an operation unit 20. Further, the image pickup apparatus 100 includes a control unit 30, a data storage unit 21, and a display processing unit 22.
  • TG Timing Generator
  • Subject light which is light from the subject to be imaged, is incident on the lens system 10.
  • the lens system 10 includes various lenses such as an image pickup lens and a focus lens, and an aperture.
  • the subject light passes through various lenses and a diaphragm in the lens system 10 and is imaged on the exposed surface of the image pickup device 12.
  • the aperture is configured to be able to mechanically adjust the amount of subject light by adjusting its opening degree.
  • the diaphragm may be arranged between the image pickup lens and the focus lens, or may be arranged behind the focus lens.
  • the liquid crystal ND filter 11 is arranged on the optical path of the subject light.
  • the liquid crystal ND filter 11 is configured so that the light transmittance can be adjusted according to the applied voltage.
  • the transmittance is a ratio of the amount of light incident on the liquid crystal ND filter 11 to the amount of light emitted from the liquid crystal ND filter 11.
  • the transmittance can also be expressed as a concentration.
  • the transmittance can be adjusted accurately. Therefore, it is possible to accurately adjust the amount of subject light imaged on the image sensor 12.
  • the liquid crystal ND filter 11 functions as an embodiment of the transmittance adjusting unit.
  • a device different from the liquid crystal ND filter 11 may be used as the transmittance adjusting unit.
  • a method for adjusting the transmittance a method different from the application of voltage may be adopted.
  • the present technology can be applied to a configuration in which a plurality of ND filters having different transmittances (concentrations) are arranged so as to be switchable on the optical path.
  • the image pickup device 12 executes imaging by exposing the light transmitted through the liquid crystal ND filter 11.
  • the image pickup device 12 has a plurality of pixels (R pixel, G pixel, B pixel).
  • the subject light transmitted through the liquid crystal ND filter 11 and incident on the exposed surface is converted into an electronic signal by photoelectric conversion for each pixel.
  • the three primary color signals (R, G, B), which are electronic signals converted by each RGB pixel, are output to the analog signal processing circuit 13 as analog image data.
  • the image pickup device 12 is composed of, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device) sensor, or the like. In addition, any configuration may be used as the image pickup device 12.
  • CMOS Complementary Metal Oxide Semiconductor
  • CCD Charge Coupled Device
  • the analog signal processing circuit 13 executes, for example, CDS (Correlated Double Sampling) processing, gain processing, or the like on the image signal output from the image pickup device 12.
  • the signal processing method is not limited, and any method may be adopted.
  • the A / D conversion circuit 14 converts the analog image data output from the analog signal processing circuit 13 into digital image data.
  • the specific A / D conversion method is not limited, and any method may be adopted.
  • the digital signal processing circuit 15 has, for example, various digital such as noise removal processing, white balance adjustment processing, color correction processing, edge enhancement processing, and gamma correction processing for the digital image data output from the A / D conversion circuit 14. Perform signal processing.
  • the processing for the digital image data is not limited, and any processing may be executed.
  • the digital image data processed by the digital signal processing circuit 15 is output to the control unit 30. Further, in the HDR mode described later, the digital signal processing circuit 15 detects the digital image data output from the A / D conversion circuit 14 and outputs the result to the control unit 30.
  • the lens driver 16 controls the lens system 10 in response to an instruction from the control unit 30. Specifically, the positions of the image pickup lens, the focus lens and the diaphragm are controlled, and the opening degree of the diaphragm is controlled.
  • the liquid crystal ND driver 17 controls the transmittance (concentration) of the liquid crystal ND filter 11 by controlling the voltage applied to the liquid crystal ND filter 11 in response to an instruction from the control unit 30.
  • the timing generator 18 generates a drive pulse necessary for driving the image pickup device 12 in response to an instruction from the control unit 30, and supplies the drive pulse to the image pickup device 12.
  • the subject image is imaged (electronic shutter), and the image of the subject image is acquired.
  • the timing generator 18 adjusts the shutter speed of the image pickup device 12 to control the exposure period at the time of imaging.
  • the exposure period is the time width during which light is incident on the image pickup device 12 when an image is acquired by imaging.
  • the exposure period can also be said to be the shutter time.
  • the timing generator 18 causes the image pickup device 12 to take an image at a frame rate of, for example, 30 fps. Of course, it is not limited to this frame rate.
  • the recording unit 19 records the image data output from the control unit 30 and the metadata related to the image data (for example, the date and time when the image data was acquired).
  • the recording unit 19 records data on, for example, a semiconductor memory such as a memory card, an optical disk, or a recording medium such as an HD (hard disc).
  • the recording medium may be fixed inside the image pickup apparatus 100, or may be configured to be detachable from the image pickup apparatus 100.
  • the operation unit 20 includes, for example, a power switch, a shutter button, a recording button, a setting button, a mode switching button, and the like.
  • the power switch is used to switch ON / OFF of the power of the image pickup apparatus 100.
  • the shutter button is used to record image data as still image data in a still image capturing mode for capturing a still image.
  • the record button is used to record image data as moving image data in a moving image imaging mode for capturing a moving image.
  • the setting button is used, for example, to adjust the position of the zoom lens, the focus lens, and the aperture, and to adjust the opening degree of the aperture.
  • the setting button is used to adjust the electronic shutter, change the gain value in the gain processing of the analog signal processing circuit 13, and change the setting value of various processing by the digital signal processing circuit 15.
  • the mode switching button is used to switch between the still image imaging mode and the moving image imaging mode. It is also possible to select the continuous shooting mode with the mode switching button. Further, in the present embodiment, the mode switching button is used to switch to the reproduction mode in which the image recorded by the recording unit 19 is reproduced.
  • the normal mode is a mode in which imaging is performed by normal operation. Specifically, image pickup is performed by the image pickup device 12 at a predetermined frame rate. Then, the digital image data output via the image pickup element 12, the analog signal processing circuit 13, the A / D conversion circuit 14, and the digital signal processing circuit 15 is recorded as the data of the captured image.
  • the HDR mode is a mode in which an HDR image is generated by taking a plurality of images having different exposure periods and combining them as one image. By selecting the HDR mode, it is possible to generate an HDR image with a wider dynamic range and rich gradation expression. The operation in the HDR mode will be described later.
  • the data storage unit 21 is, for example, a non-volatile memory (for example, EEPROM (Electrically Erasable Programmable ROM), a program ROM (Read Only Memory)) in which various programs and various data are fixedly stored, and a temporary storage area. It is composed of a memory device such as a RAM (Radom Access Memory) used as a memory device.
  • the data storage unit 21 is also used as a frame memory for storing the combined image in the HDR mode. Further, the data storage unit 21 stores a transmittance vs. voltage table.
  • the transmittance vs. voltage table is a table showing the relationship between the transmittance of the liquid crystal ND filter 11 and the applied voltage.
  • the image data output from the control unit 30 and the metadata related to the image data may be stored in the data storage unit 21.
  • the display processing unit 22 executes display processing for the display device. For example, a captured image, a GUI (Graphical User Interface) for user operation, and the like are displayed.
  • the display processing unit 22 is realized including the display device. That is, a display device including a liquid crystal display, an organic EL (Electroluminescence) display, or the like is provided in the image pickup apparatus 100.
  • display processing such as an image or GUI may be executed on an external display device connected to the image pickup apparatus 100.
  • the display processing unit 22 displays a through image by displaying the image data output from the digital signal processing circuit 15 in real time. The through image is displayed to allow the user to adjust the angle of view when capturing a still image or a moving image.
  • the display processing unit 22 can also reproduce the image recorded by the recording unit 19.
  • the control unit 30 has hardware necessary for configuring a computer, such as a processor such as a CPU, GPU, and DSP, a memory such as ROM and RAM, and a storage device such as an HDD.
  • a computer such as a processor such as a CPU, GPU, and DSP, a memory such as ROM and RAM, and a storage device such as an HDD.
  • the image pickup method according to the present technology is executed by the CPU or the like loading and executing the program according to the present technology recorded in advance in the ROM or the like into the RAM.
  • the configuration of the control unit 30 is not limited, and any hardware and software may be used. Of course, hardware such as FPGA and ASIC may be used.
  • an exposure control processing unit 31, an image composition processing unit 32, and a flicker detection unit 33 are configured as functional blocks.
  • the program is installed in the image pickup apparatus 100, for example, via various recording media. Alternatively, the program may be installed via the Internet or the like.
  • the type of recording medium on which the program is recorded is not limited, and any computer-readable recording medium may be used. For example, any non-transient storage medium readable by a computer may be used.
  • the flicker detection unit 33 can detect whether or not flicker is generated in the subject light to be imaged. Further, the flicker detection unit 33 can detect the flicker cycle when flicker is generated. As described above, in the present embodiment, the image pickup device 12 exposes the light transmitted through the liquid crystal ND filter 11 to perform imaging. The flicker detection unit 33 can detect the presence or absence of flicker of light transmitted through the liquid crystal ND filter 11 and the flicker cycle. The flicker detection unit 33 functions as an embodiment of the detection unit according to the present technology.
  • the image composition processing unit 32 generates an HDR image by synthesizing two images (a long-exposure image and a short-exposure image described later) captured by different exposure amounts when the HDR mode is set. ..
  • the image composition processing unit 32 functions as an embodiment of the composition unit according to the present technology.
  • FIG. 2 is a block diagram showing a functional configuration example of the exposure control processing unit 31.
  • the exposure control processing unit 31 can execute various processes for controlling the exposure amount when imaging is executed, and the exposure amount setting unit 34, the exposure period setting unit 35, and the transmittance setting. It has a portion 36.
  • the exposure amount setting unit 34 sets the exposure amount when imaging is executed.
  • the exposure period setting unit 35 sets the exposure period when the imaging is executed.
  • the transmittance setting unit 36 sets the transmittance of the liquid crystal ND filter 11 when imaging is executed.
  • the exposure amount, the exposure period, and the transmittance are set according to the following equations.
  • Log (exposure amount) Log (exposure period) + Log (transmittance) ...
  • the exposure amount setting unit 34 sets the exposure amount required when imaging is executed.
  • the exposure period and the transmittance are set by the exposure period setting unit 35 and the transmittance setting unit 36 according to the equation (1) so that the exposure amount is realized.
  • the non-fixed parameter may be adjusted so that the required exposure amount is obtained.
  • FIG. 3 is a flowchart showing the processing of the image pickup apparatus 100 according to the present embodiment.
  • the control unit 30 determines whether or not the imaging setting is the HDR mode (step 101).
  • the switching between the normal mode and the HDR mode can be executed by, for example, the mode switching button of the operation unit 20.
  • the image pickup apparatus 100 performs a normal image pickup process (step 102).
  • FIG. 4 is a flowchart showing an example of the normal imaging process.
  • the exposure amount setting unit 34 sets the exposure amount for acquiring an image (step 201).
  • the exposure amount is set based on, for example, a set value input by the user or the amount of ambient light that illuminates the subject.
  • the method of setting the exposure amount is not limited, and any method may be adopted.
  • the exposure period setting unit 35 sets the exposure period when imaging is executed.
  • the transmittance setting unit 36 sets the transmittance of the liquid crystal ND filter 11 when imaging is executed (step 202). For example, according to the above equation (1), it is possible to calculate the exposure period and the transmittance so that the exposure amount set in step 201 is realized.
  • step 203 image pickup is executed by the image pickup device 12 and an image is acquired (step 203).
  • the control unit 30 controls the transmittance of the liquid crystal ND filter 11 so that the transmittance is calculated in step 202 according to the timing at which the image data of the next frame is captured.
  • the control unit 30 refers to the transmittance vs. voltage table, and reads out the applied voltage corresponding to the transmittance calculated in step 202.
  • the control unit 30 instructs the liquid crystal ND driver 17 to apply the read applied voltage.
  • the voltage may be calculated in real time according to the program, not limited to the case where the table is referred to by the control unit 30.
  • the liquid crystal ND driver 17 applies a voltage to the liquid crystal ND filter 11 based on an instruction from the control unit 30.
  • the liquid crystal ND filter 11 adjusts the transmittance according to the applied voltage. By using the liquid crystal ND filter 11, the transmittance can be adjusted with high accuracy.
  • the control unit 30 controls the timing generator 18 and adjusts the shutter speed of the image sensor 12 so that the exposure period calculated in step 202 is reached.
  • the subject image transmitted through the liquid crystal ND filter 11 is coupled to the exposed surface of the image pickup device 12, and the image pickup device 12 generates image data.
  • This image data is subjected to various processing by the analog signal processing circuit 13, and then converted into a digital signal by the A / D conversion circuit 14.
  • the image data converted into a digital signal is acquired by the control unit 30 and stored in the recording unit 19 after various processes are executed in the digital signal processing circuit 15.
  • any other method may be adopted as a specific method for normal imaging.
  • the image pickup apparatus 100 executes HDR image generation by the processing of steps 103 to 108.
  • the HDR image is an image having a wider dynamic range in which overexposure and underexposure are sufficiently suppressed. For example, a plurality of photographs are taken while changing the exposure amount, and the captured images are combined to obtain an HDR image.
  • the long-exposure image P1 and the short-time exposure image P2 are generated in continuous frames.
  • the HDR image P3 is generated by combining the long-exposure image P1 and the short-time exposure image P2.
  • the long-exposure image P1 is generated by performing imaging by long-time exposure.
  • the exposure period and the transmittance are appropriately set, and imaging by long-time exposure is performed.
  • the exposure period for capturing the long-exposure image P1 is referred to as the first exposure period T1.
  • the transmittance for capturing the long-exposure image P1 is defined as the first transmittance.
  • the first transmittance can also be said to be the transmittance when imaging is performed in the first exposure period T1.
  • the long-exposure image P1 is acquired by imaging with the first exposure period T1 and the first transmittance.
  • the short-time exposure image P2 is generated by performing imaging by short-time exposure.
  • an exposure period shorter than the first exposure period T1 and a transmittance are appropriately set, and imaging by short exposure is executed.
  • the exposure period for capturing the short-time exposure image P2 is referred to as the second exposure period T2.
  • the second exposure period T2 is shorter than the first exposure period T1.
  • the transmittance for capturing the short-exposure image P2 is defined as the second transmittance.
  • the second transmittance can also be said to be the transmittance when imaging is performed in the second exposure period T2.
  • the short exposure image P2 is acquired by imaging with the second exposure period T2 and the second transmittance.
  • imaging by long exposure is executed at the nth frame. Therefore, imaging is performed in the first exposure period T1.
  • Imaging by short exposure is performed at the n + 1th frame. Therefore, imaging is performed in the second exposure period T2.
  • the image data of the long-exposure image P1 captured in the nth frame is output and stored in the frame memory.
  • the image data of the short exposure image P2 captured at the n + 1th frame is output.
  • the image data of the long-exposure image P1 and the image data of the short-exposure image P2 are combined to generate the HDR image P3.
  • n + 2nd frame imaging by long exposure is executed to generate an HDR image of the next frame. Therefore, assuming that the frame rate at which each frame of n, n + 1, and n + 2 shown in FIG. 5 is repeated is M (fps), the frame rate at which the HDR image P3 is generated is 1/2 times M / 2 (fps). ..
  • the long-exposure image P1 in the long-exposure image P1, overexposure occurs in a portion corresponding to the front side of the subject. That is, in this example, a portion having a large exposure amount appears white and the gradation is lost.
  • the long-exposure image P1 is set with a sufficiently long exposure period so that blackout does not occur.
  • an image in which overexposure is allowed may be captured.
  • blackout occurs in a portion corresponding to the back side of the subject. That is, in this example, the portion where the exposure amount is small appears black, and the gradation is lost.
  • the short exposure period of the short exposure image P2 is set so that overexposure does not occur.
  • an image in which blackout is allowed may be captured.
  • the HDR image P3 is generated by HDR combining the long exposure image P1 and the short exposure image P2. This makes it possible to generate an image in which the gradation is not lost in either the portion corresponding to the front side of the subject and the portion corresponding to the back side of the subject. That is, it is possible to acquire an HDR image P3 having a wider dynamic range and rich gradation expression.
  • a threshold value is set for a gradation value (pixel value).
  • the gradation value of the long-exposure image P1 or the gradation value of the short-time exposure image P2 is appropriately allocated to each pixel based on the threshold value and output as a composite image.
  • the method of image composition is not limited, and any method such as an image composition method including gain processing or the like may be adopted.
  • the long-exposure image P1 corresponds to one embodiment of the first image.
  • the short exposure image P2 corresponds to one embodiment of the second image.
  • the double speed drive setting is executed (step 103).
  • the drive frequency is set so that the frame rate in which each frame of n, n + 1, and n + 2 shown in FIG. 5 is repeated is twice the frame rate of the normal imaging process. This makes it possible to make the frame rate at which the normal image is generated in the normal mode equal to the frame rate at which the HDR image is generated in the HDR mode. This improves usability for the user who uses the image pickup apparatus 100.
  • the frame rate at which the HDR image P3 is generated is not limited.
  • FIG. 7 is a flowchart showing an example of the imaging process of the long-exposure image P1.
  • the exposure amount setting unit 34 sets a first exposure amount for acquiring the long-exposure image P1 (step 301).
  • the target first exposure amount is set based on the set value from the user, the amount of ambient light, and the like.
  • the exposure period setting unit 35 sets the first exposure period T1 when the long-exposure image P1 is captured.
  • the transmittance setting unit 36 sets the first transmittance when the imaging of the long-exposure image P1 is executed (step 302).
  • the first exposure amount, the first exposure period T1, and the first transmittance are set according to the following equations.
  • Log (first exposure amount) Log (first exposure period) + Log (first transmittance) ...
  • a target first exposure amount is set, and first, a first exposure period T1 is set.
  • the first transmittance is set according to the equation (2) based on the first exposure amount and the first exposure period T1.
  • image pickup is executed by the image pickup device 12, and the long-exposure image P1 is acquired (step 303).
  • the long-exposure image P1 acquired by imaging is stored in the frame memory of the data storage unit 21.
  • the flicker detection unit 33 determines whether or not flicker is generated in the subject light to be imaged, that is, the subject light transmitted through the liquid crystal ND filter 11 (step 105). For example, when the light source that illuminates the subject is a fluorescent lamp, flicker may occur in the subject light.
  • the fluorescent lamp repeatedly blinks at a frequency (100 Hz, 120 Hz) that is twice the power frequency (50 Hz, 60 Hz). This blinking causes flicker of the subject light.
  • flicker occurs in the subject light, a flicker phenomenon may occur in the captured image, and the quality of the image may deteriorate. For example, in a moving image taken at a predetermined frame rate, inter-frame flicker may occur. When the flicker phenomenon occurs, the displayed image appears to be flickering, which makes the user uncomfortable.
  • FIG. 8 is a flowchart showing an example of the flicker detection process.
  • the flicker detection unit 33 detects the presence or absence of flicker and the flicker cycle (step 401).
  • the digital signal processing circuit 15 detects the digital image data input from the A / D conversion circuit 14, and the detection result is output to the control unit 30.
  • the flicker detection unit 33 determines whether or not flicker has occurred based on the detection result. For example, the flicker detection unit 33 acquires the detection result of image data for a predetermined frame (about 5 frames).
  • the presence or absence of flicker and the flicker cycle are detected based on the degree of variation in the brightness (gradation) of the entire image data.
  • flicker detection may be performed using an image acquired in a normal imaging process.
  • flicker detection may be executed based on the short exposure image P2.
  • imaging for flicker detection may be performed.
  • the method of detecting flicker is not limited. For example, an arbitrary method such as a method of specifying the presence or absence of flicker and a flicker period by using a discrete Fourier transform may be adopted.
  • the flicker detection unit 33 determines the presence or absence of flicker based on the result of step 401 (step 402).
  • the state of the subject is set to the flicker state (step 403).
  • the state of the subject is set to the normal state (step 404).
  • step 105 shown in FIG. 3 it is determined which state the subject state is set to.
  • the state of the subject is the normal state, it is determined that no flicker has occurred in the subject light.
  • the state of the subject is a flicker state, it is determined that flicker is generated in the subject light (step 105).
  • the imaging process of the short-time exposure image P2 when there is no flicker is executed (step 106). If flicker is generated in the subject light (Yes in step 105), the imaging process of the short-time exposure image P2 in the case of flicker is executed (step 107).
  • FIG. 9 is a flowchart showing an example of the imaging process of the short-exposure image P2 when flicker does not occur.
  • the exposure amount setting unit 34 sets a second exposure amount for acquiring the short-time exposure image P2 (step 501).
  • a target second exposure amount is set based on a set value from the user, the amount of ambient light, and the like.
  • the exposure period setting unit 35 sets a second exposure period T2 when the short-exposure image P2 is captured.
  • the transmittance setting unit 36 sets a second transmittance when the short-exposure image P2 is captured (step 502).
  • the second exposure amount, the second exposure period T2, and the second transmittance are set according to the following equations.
  • Log (second exposure amount) Log (second exposure period) + Log (second transmittance) ... (3)
  • a target second exposure amount is set, and first, a second exposure period T2 is set.
  • the second transmittance is set according to the formula (3) based on the second exposure amount and the second exposure period T2.
  • an image pickup is executed by the image pickup device 12, and a short-time exposure image P2 is acquired (step 503).
  • the short-time exposure image P2 acquired by imaging is stored in the frame memory of the data storage unit 21.
  • FIG. 10 is a flowchart showing an example of the imaging process of the short-exposure image P2 when flicker is generated.
  • FIG. 11 is a schematic diagram for explaining the principle of reducing the flicker phenomenon.
  • FIG. 11 shows a graph in which the horizontal axis represents time and the vertical axis represents the amount of light from a flicker light source.
  • the exposure amount setting unit 34 sets a second exposure amount for acquiring the short-time exposure image P2 (step 601).
  • a target second exposure amount is set based on a set value from the user, the amount of ambient light, and the like.
  • the exposure period setting unit 35 sets a second exposure period T2 when the short-exposure image P2 is captured (step 602).
  • the second exposure period T2 is set to an integral multiple (multiplication) of 1/2 of the flicker period.
  • the area formed by the graph and the horizontal axis in the second exposure period T2 indicated by the arrow on the left side is the area formed by the graph and the horizontal axis in the second exposure period T2 indicated by the arrow on the right side. Consistent with the area of. Therefore, it is possible to keep the amount of subject light incident on the liquid crystal ND filter 11 constant during the second exposure period T2, regardless of the time when the imaging of the short-exposure image P2 is started. This makes it possible to sufficiently reduce the occurrence of the flicker phenomenon in the short-exposure image P2 due to the flicker of the subject light. That is, it is possible to sufficiently reduce that the exposure amount of the short-exposure images P2 taken in different frames varies due to the flicker of the subject light.
  • the second exposure period T2 is not limited to the 1/2 flicker cycle, and any value that is an integral multiple of 1/2 of the flicker cycle may be set.
  • FIG. 11 also shows a first exposure period T1 when the long-exposure image P1 is captured.
  • the first exposure period T1 is longer than the second exposure period T2.
  • the influence of the flicker of the subject light is small and often sufficiently acceptable. Therefore, for example, by setting the maximum exposure period as the first exposure period T1, it is possible to capture a high-precision long-exposure image P1 without blackout.
  • the first exposure period T1 may be set to be an integral multiple of 1/2 of the flicker period.
  • the exposure period may be set to be an integral multiple of 1/2 of the flicker period even when the normal imaging process shown in FIG. 4 is executed. This makes it possible to sufficiently suppress interframe flicker in the normal imaging process.
  • the exposure amount setting unit 34 resets the exposure amount (step 603).
  • the resetting of the exposure amount is set according to the following formula.
  • Log (second exposure amount) -Log (second exposure period) Log (exposure amount_reset) ...
  • the transmittance setting unit 36 sets the second transmittance according to the following formula based on the reset exposure amount and the set second exposure period T2 (step 604).
  • Log (exposure amount_reset) Log (second transmittance) ... (5) Therefore, in the present embodiment, the target second exposure amount is first set. Then, the second exposure period T2 is set so as to be an integral multiple of 1/2 of the flicker period.
  • the second transmittance is finally set based on the target second exposure amount and the second exposure period T2.
  • the liquid crystal ND filter 11 By using the liquid crystal ND filter 11, it is possible to perform such a second transmittance setting with high accuracy.
  • Other methods may be adopted for setting the second exposure amount, the second exposure period T2, and the second transmittance.
  • image pickup is executed by the image pickup device 12, and the short-time exposure image P2 is acquired (step 605).
  • the short-time exposure image P2 acquired by imaging is stored in the frame memory of the data storage unit 21.
  • the image composition processing unit 32 combines the long-exposure image P1 and the short-time exposure image P2 stored in the frame memory to generate the HDR image P3 (step 108).
  • the short exposure image P2 does not have to be stored in the frame memory. That is, the HDR image P3 may be generated by synthesizing the long-exposure image P1 stored in the frame memory and the short-exposure image P2 generated in steps 503 and 605 (step 108). In the present embodiment, even when flicker is generated in the subject light, it is possible to generate an HDR image in which the flicker phenomenon is reduced with high accuracy.
  • the display processing unit 22 executes display processing of the generated HDR image P3.
  • the first exposure period T1 and the second exposure period T2 shorter than the first exposure period T1 are based on the flicker cycle of the subject light to be imaged.
  • a first transmittance and a second transmittance are set as the transmittances corresponding to each of the first exposure period T1 and the second exposure period T2. This makes it possible to acquire the HDR image P3 having a wider dynamic range while reducing the flicker phenomenon by synthesizing the long-exposure image P1 and the short-time exposure image P2.
  • the exposure period T2 of the short-time exposure image P2 is set based on the flicker cycle. As a result, the flicker phenomenon of the short-time exposure image P2 is reduced. Further, the transmittance of the liquid crystal ND filter 11 is adjusted in order to realize the target second exposure amount. This makes it possible to generate a high quality HDR image P3. For example, it is conceivable to adjust the aperture opening in order to realize the target second exposure amount. However, in the method of adjusting the aperture opening, there is a high possibility that the aperture blur occurs between the long-exposure image P1 and the short-time exposure image P2, or the depth of field shifts. As a result, the image quality of the HDR image P3 acquired by compositing deteriorates.
  • the transmittance of the liquid crystal ND filter 11 by adjusting the transmittance of the liquid crystal ND filter 11, it is possible to acquire a high-quality short-exposure image P2, and it is possible to improve the image quality of the HDR image P3 acquired by compositing. It becomes. That is, in the present embodiment, a high-quality HDR image in which the flicker phenomenon is reduced by a simple method of setting the exposure period T2 of the short-time exposure image P2 to the flicker cycle by adjusting the transmittance of the liquid crystal ND filter 11. It becomes possible to generate P3.
  • the transmittance for example, when the long-exposure image P1 is captured, the transmittance (first transmittance) is set high (the density is set thin) in order to obtain information on the brighter side. Then, when the short-exposure image P2 is photographed, the transmittance (second transmittance) is lowered (the density is increased). For example, the first transmittance is set to 100% (a value close to 100%), and the second transmittance is set to a low value. By setting the first transmittance to a value larger than the second transmittance in this way, it is possible to generate a higher quality HDR image 3. Setting the first transmittance to 100% (a value close to 100%) is equivalent to not using the liquid crystal ND filter 11.
  • the control of setting the first transmittance to 100% (a value close to 100%) and setting the second transmittance to a low value uses the liquid crystal ND filter 11 when capturing the long-exposure image P1. It can be said that the control uses the liquid crystal ND filter 11 only when the short-exposure image P2 is taken.
  • the first transmittance is also possible to set the first transmittance to the same value as the second transmittance. This eliminates the need to frequently control the liquid crystal ND filter 11, so that the load can be reduced in terms of control / equipment. As a result, it becomes possible to generate an HDR image while reducing the load.
  • the transmittance of the liquid crystal ND filter 11 may be affected by the temperature environment. For example, when the liquid crystal ND filter 11 becomes hot, there may be a difference between the transmittance set by the transmittance setting unit 36 and the transmittance when performing imaging. In order to suppress this difference, for example, feedback control may be executed. Specifically, the control unit 30 compares the transmittance when the imaging is executed with the set transmittance, sets the voltage applied to the liquid crystal ND filter 11 so that the difference is suppressed, and sets the liquid crystal. You may give an instruction to the ND driver 17. Further, the analog signal processing circuit 13 may execute gain processing or the like based on the difference.
  • expressions using "more” such as “greater than A” and “less than A” include both the concept including the case equivalent to A and the concept not including the case equivalent to A. It is an expression that includes the concept. For example, “greater than A” is not limited to the case where the equivalent of A is not included, and “greater than or equal to A” is also included. Further, “less than A” is not limited to “less than A” and includes “less than or equal to A”. When implementing this technique, specific settings and the like may be appropriately adopted from the concepts included in “greater than A” and “less than A” so that the effects described above can be exhibited.
  • a transmittance adjustment unit that can adjust the transmittance of light
  • An image sensor that performs imaging by exposing the light transmitted through the transmittance adjusting unit
  • An exposure period setting unit that sets a first exposure period and a second exposure period shorter than the first exposure period based on the flicker period of light transmitted through the transmittance adjusting unit.
  • a transmittance setting unit that sets a first transmittance when imaging is executed in the first exposure period and a second transmittance when imaging is executed in the second exposure period.
  • An image pickup device including a compositing unit that synthesizes and.
  • the exposure period setting unit is an image pickup device that sets the second exposure period based on the flicker cycle.
  • the synthesizing unit is an image pickup device that generates an HDR (High Dynamic Range) image by synthesizing the first image and the second image.
  • the exposure period setting unit is an image pickup device that sets the second exposure period to an integral multiple of 1/2 of the flicker cycle.
  • the exposure period setting unit is an image pickup device that sets the first exposure period to an integral multiple of 1/2 of the flicker cycle.
  • the image pickup apparatus is an image pickup device that sets the first transmittance to a value larger than the second transmittance.
  • the transmittance setting unit is an image pickup device that sets the first transmittance to 100%.
  • the image pickup apparatus is an image pickup device that sets the first transmittance to the same value as the second transmittance.
  • the image pickup apparatus according to any one of (1) to (8), and further. It is provided with an exposure amount setting unit for executing a first exposure amount setting for acquiring the first image and a second exposure amount setting for acquiring the second image.
  • the transmittance setting unit is an image pickup device that sets the second transmittance based on the set second exposure amount and the set second exposure period. (10) The image pickup apparatus according to (9). The transmittance setting unit is an image pickup device that sets the first transmittance based on the set first exposure amount and the set first exposure period. (11) The image pickup apparatus according to any one of (1) to (10). The transmittance adjusting unit is an image pickup device capable of adjusting the transmittance according to an applied voltage. (12) The image pickup apparatus according to (11). The transmittance adjusting unit is an image pickup device that is a liquid crystal display ND (Neutral Density) filter. (13) The image pickup apparatus according to any one of (1) to (12), and further.
  • An image pickup apparatus including a detection unit that detects a flicker cycle of light transmitted through the transmittance adjusting unit. (14) A first exposure period and a second exposure period shorter than the first exposure period are set based on the flicker period of the light transmitted through the transmittance adjusting unit capable of adjusting the transmittance of the light.
  • the first transmittance when the image pickup is executed in the first exposure period is set by the image pickup element that executes the image pickup by exposing the light transmitted through the transmittance adjusting unit.
  • the second transmittance when the image pickup is performed by the image pickup element in the second exposure period is set.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)
  • Blocking Light For Cameras (AREA)

Abstract

Ce dispositif d'imagerie comprend une unité d'ajustement de transmittance ; un élément d'imagerie ; une unité de réglage de période d'exposition ; une unité de réglage de transmittance ; et une unité de synthèse. L'unité d'ajustement de transmittance ajuste la transmittance de lumière. L'élément d'imagerie capture une image par exposition à la lumière transmise par l'unité d'ajustement de transmittance. L'unité de réglage de période d'exposition règle une première période d'exposition et une seconde période d'exposition, qui est plus courte que la première période d'exposition, sur la base de la période de papillotement de la lumière transmise par l'unité d'ajustement de transmittance. L'unité de réglage de transmittance règle une première transmittance pour l'imagerie pendant la première période d'exposition, et une seconde transmittance pour l'imagerie pendant la seconde période d'exposition. L'unité de synthèse synthétise une première image, acquise par imagerie avec la première transmittance dans la première période d'exposition, et une seconde image, acquise par imagerie avec la seconde transmittance dans la seconde période d'exposition.
PCT/JP2021/021165 2020-06-12 2021-06-03 Dispositif d'imagerie et procédé d'imagerie WO2021251260A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020102320A JP2021197613A (ja) 2020-06-12 2020-06-12 撮像装置、及び撮像方法
JP2020-102320 2020-06-12

Publications (1)

Publication Number Publication Date
WO2021251260A1 true WO2021251260A1 (fr) 2021-12-16

Family

ID=78846116

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/021165 WO2021251260A1 (fr) 2020-06-12 2021-06-03 Dispositif d'imagerie et procédé d'imagerie

Country Status (2)

Country Link
JP (1) JP2021197613A (fr)
WO (1) WO2021251260A1 (fr)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023248589A1 (fr) * 2022-06-23 2023-12-28 株式会社ジャパンディスプレイ Module de caméra

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014179924A (ja) * 2013-03-15 2014-09-25 Canon Inc 撮像装置及びその制御方法、プログラム、並びに記憶媒体
JP2015198292A (ja) * 2014-03-31 2015-11-09 ソニー株式会社 撮像装置、フリッカの補正方法及びプログラム
JP2017011367A (ja) * 2015-06-17 2017-01-12 キヤノン株式会社 撮像装置及びその制御方法

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014179924A (ja) * 2013-03-15 2014-09-25 Canon Inc 撮像装置及びその制御方法、プログラム、並びに記憶媒体
JP2015198292A (ja) * 2014-03-31 2015-11-09 ソニー株式会社 撮像装置、フリッカの補正方法及びプログラム
JP2017011367A (ja) * 2015-06-17 2017-01-12 キヤノン株式会社 撮像装置及びその制御方法

Also Published As

Publication number Publication date
JP2021197613A (ja) 2021-12-27

Similar Documents

Publication Publication Date Title
JP6680377B2 (ja) 撮像装置、撮像方法及びプログラム
US10194091B2 (en) Image capturing apparatus, control method therefor, program, and recording medium
TWI390971B (zh) 成像裝置，成像處理方法及成像控制程式
US9843735B2 (en) Image processing apparatus, imaging apparatus comprising the same, and image processing method
JP5123137B2 (ja) 撮像装置および撮像方法
US9681060B2 (en) Imaging apparatus, method of correcting flicker, and program
US9277135B2 (en) Image-pickup apparatus, control method for the same, and non-transitory computer-readable storage medium
US9277134B2 (en) Image pickup apparatus and image pickup method
JP2016082510A (ja) 画像処理装置及び画像処理方法
JP2018006827A (ja) 撮像装置、撮像プログラム、撮像方法
US11128799B2 (en) Image processing apparatus and image processing method
US11006050B2 (en) Imaging apparatus and imaging method using first and second images for generating high dynamic range image
JP5984975B2 (ja) 撮像装置、撮像方法、およびプログラム
WO2021251260A1 (fr) Dispositif d'imagerie et procédé d'imagerie
JPWO2015146471A1 (ja) 撮影装置
JP2018056941A (ja) 撮像装置、表示装置、電子機器、撮像方法、プログラム、撮像システム、表示システムおよび画像処理装置。
JP2010103700A (ja) 撮像装置および撮像方法
JP6242129B2 (ja) 撮像装置、その制御方法、及び制御プログラム
JP2014146969A (ja) デジタルカメラ
JP6075829B2 (ja) 撮像装置、カメラシステム、撮像装置の制御方法、プログラム、記憶媒体
JP5268823B2 (ja) 撮像装置及び撮像方法
JP3206826U (ja) 画像処理装置
JP2009225252A (ja) 撮像装置および撮像方法
JP2016140057A (ja) 撮像装置
JP2013106228A (ja) 撮像装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21821780

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21821780

Country of ref document: EP

Kind code of ref document: A1