JP2010278728A - Image pickup device, integrated circuit, image processing method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To suppress white-band-like interference due to external flash light, such as photoflash in an image pickup device, and to obtain a bright photographed image available by the photoflash which is required per se. <P>SOLUTION: The image pickup device 100 includes an imaging part 101, which acquires imaging signal from a subject focused with a lens part 110; a timing generation part 102 which controls shutter speed of the imaging part 101 for outputting shutter speed information representing the shutter speed; a flash detecting part 104 which detects flash light from the imaging signal, based on the shutter speed information from the timing generation part 102; a camera process part 103, which applies a camera process to the imaging signal for outputting image data; and a buffer 105 in which the image data is temporarily stored. The camera process part 103, according to flash detection result of the flash detecting part 104, removes image data of a plurality of continuous frames from the buffer 105 and adds the image data of a plurality of frames and outputs the result. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a technique for reducing an adverse effect on the quality (image quality) of a picked-up image by a flash light such as a flash in an image pickup apparatus such as a digital camera or a video camera that picks up continuous images.

In recent years, imaging devices using CMOS (Complementary Metal Oxide Semiconductor, hereinafter referred to as CMOS) image sensors with features such as small size, low power consumption, and high-speed imaging have made great strides in the field of consumer video cameras and professional video cameras. Yes.
CMOS has various features compared to a conventional CCD (Charge Coupled Device, hereinafter referred to as CCD). In CMOS, charge readout of PD (Photo Diode, hereinafter referred to as PD) is not read at the same timing (so-called global shutter) in all pixels as in CCD, but the readout timing shifts in line (pixel) units. (So-called rolling shutter), the timing of the accumulation period of each pixel is shifted. In CMOS, problems due to this occur.
One of the problems is that when a subject in which flash light is blown, such as a still camera, is captured with a video camera of a CMOS image sensor, white-band interference occurs in the image.

The phenomenon will be described below with reference to FIGS.
FIG. 2 is a diagram schematically illustrating the accumulation period, readout, and scanning period of the CMOS image sensor. FIG. 2 is a diagram when the frame period TF for driving the image sensor and the accumulation period TS for accumulating charges in the image sensor are the same, that is, when the shutter operation is not performed (shutter off).
Here, the accumulation period corresponds to an exposure period in which a shutter is opened, which is referred to as a still camera, and is also referred to as a shutter period.
In the rolling shutter CMOS image sensor, the timing of the accumulation period TS is shifted for each line as shown in the figure.
Therefore, for example, when the flash is turned on at the timing shown in FIG. 2, bright flash light affects the exposure period in the second half of frame 1 and the exposure period in the first half of frame 2. In the figure, the flash light straddles the accumulation periods of different frames from Line 3 to Line 5.

In other words, in frame 1, the lines before Line 2 are not affected by the flash light, and Lines 3 to 5 are gradually affected by the flash light. And after Line 6, it is affected by the total amount of flash light. Similarly, frame 2 is affected by the total amount of flash light before line 2, line 3 to line 5 gradually decrease, and line 6 and later are not affected by the flash light.
As a result of the influence of the flash light, as shown in FIG. 2, a white belt-like disturbance appears in the camera output image, where the lower side is bright in frame 1 and the upper side is bright in frame 2. As is clear from the principle of white band generation, the time width of the white band is substantially equal to the accumulation period, and the time width during which the signal level changes at the edge of the white band is approximately equal to the time width during which flash light is shining. .
Next, a case where the shutter is applied when the CMOS image sensor is driven (shutter ON) will be described with reference to FIG.

When the shutter is applied, the accumulation period TS is shortened as shown in the figure, so that the number of lines affected by the flash light is reduced. For this reason, the width of the white belt-like interference appearing in the camera output image is narrower than in the case of shutter-off. In this case, the time width of the white belt is almost equal to the accumulation period.
As described above, if the CCD is used for a global shutter, the entire screen will be brightened uniformly when the flash is used. However, a white-banded interference occurs on the screen for a CMOS image sensor for a rolling shutter. there were.
In order to solve this problem, the following imaging device is conventionally known (for example, see Patent Document 1). FIG. 6 is a block diagram illustrating a configuration example of a conventional imaging apparatus. 6 includes a lens unit 12, an imaging unit 13, an image processing unit 14, a buffer unit 15, a recording display processing unit 16, a buffer unit 17, a recording unit 18, and a display unit 19. And an evaluation unit 20 and a control unit 23.

The operation of the conventional imaging apparatus 10 configured as described above will be described below.
First, light from a subject incident from the lens unit 12 is converted into an imaging signal by the imaging unit 13 and supplied to the image processing unit 14.
The image processing unit 14 performs predetermined image processing on the imaging signal and supplies the image signal to the recording / display processing unit 16 and the evaluation unit 20.
The buffer unit 15 stores data that needs to be temporarily stored when the image processing unit 14 performs image processing.
The recording display processing unit 16 is supplied with image data subjected to image processing from the image processing unit 14, and performs recording control for recording the image data in the recording unit 18 and display control for displaying the image data on the display unit 19.

The buffer unit 17 stores data that needs to be temporarily stored when the recording / display processing unit 16 performs recording control or display control.
The evaluation unit 20 supplies information obtained from the image supplied from the image processing unit 14, for example, information indicating the brightness of a predetermined portion of the image to the control unit 23 as an evaluation value.
For example, if the control unit 23 determines that the image has been adversely affected by the external flash light based on the evaluation value, the control unit 23 excludes the image and records the image that has not been adversely affected by the external flash light. To record.

JP 2007-306225 A

However, in the above conventional imaging device, white band interference caused by external flash such as flash light can be removed, but since the image affected by the flash light is excluded, a bright captured image by flash light is obtained. I can't.
SUMMARY OF THE INVENTION The present invention solves the above-mentioned conventional problems. An imaging apparatus and an integrated circuit capable of suppressing white-band interference caused by external flash light such as flash light and obtaining a bright captured image by the originally required flash light An object is to provide an image processing method and a program.

A first invention is an imaging apparatus including an imaging unit, a timing generation unit, a flash detection unit, a camera processing unit, and a buffer.
The imaging unit acquires an imaging signal from the subject imaged by the lens unit. The timing generation unit controls the shutter time of the imaging unit and outputs shutter time information indicating the shutter time. The flash detection unit detects the flash from the imaging signal based on the shutter time information from the timing generation unit. The camera processing unit performs camera processing on the imaging signal and outputs image data. The buffer temporarily stores image data.
Further, the camera processing unit takes out image data of a plurality of continuous frames from the buffer according to the flash detection result of the flash detection unit, adds the image data of the plurality of frames, and outputs them.

Here, when the flash detection unit detects the flash, for example, the time width when the level of the imaging signal from the imaging unit exceeds a predetermined level is compared with the shutter time. Including doing. Further, adding image data of a plurality of frames includes addition averaging as well as addition.
In this imaging device, in order to add and output the image data of a plurality of frames by the camera processing unit in accordance with the flash detection result of the flash detection unit, a white belt-like shape caused by an external flash such as flash light over the plurality of frames is output. It is possible to suppress the interference and obtain a bright captured image by the flash light originally required.
2nd invention is 1st invention, Comprising: When a flash detection part detects a flash, a timing production | generation part controls to turn off the shutter operation | movement of an imaging part.

In this imaging apparatus, when the flash is detected by the flash detection unit, the timing generation unit controls the shutter operation of the imaging unit to be off, so that a bright captured image by the originally required flash light can be obtained with certainty.
3rd invention is 1st invention, Comprising: A timing production | generation part controls to turn off the shutter operation | movement of an imaging part, when a flash detection part detects a flash more than predetermined times within a predetermined period. .
In this imaging apparatus, the flash detection unit can detect the flash more reliably.
4th invention is 2nd or 3rd invention, Comprising: When a timing production | generation part controls the shutter operation | movement of an imaging part to OFF, a camera process part is with respect to an imaging signal so that an output signal may become a predetermined level. Adjust the gain.

In this imaging apparatus, when the shutter operation is switched off by the timing generation unit, the gain of the imaging signal is adjusted by the camera processing unit 103, whereby a difference in the brightness of the image obtained between when the shutter operation is on and when the shutter operation is off. Can be suppressed, and the image quality can be guaranteed.
5th invention is 1st invention, Comprising: Only when the shutter operation | movement of an imaging part is OFF, a camera process part takes out the image data of the continuous flame | frame from a buffer, and acquires image data of a flame | frame of several frames Add and output.
In this imaging apparatus, only when the shutter operation of the imaging unit is off, image data is added and output by the camera processing unit according to the flash detection result, so that a bright captured image by the originally required flash light is more reliably obtained. Obtainable.

6th invention is 1st invention, Comprising: When a flash detection part does not detect a flash within a predetermined | prescribed period, a timing generation part controls to turn on the shutter operation | movement of an imaging part.
This imaging apparatus can prevent the shutter-off state from continuing unnecessarily.
A seventh invention is an imaging apparatus including an imaging unit, a timing generation unit, a flash detection unit, a camera processing unit, and a buffer.
The imaging unit acquires an imaging signal from the subject imaged by the lens unit. The timing generation unit controls the shutter time of the imaging unit and outputs shutter time information indicating the shutter time. The flash detection unit detects the flash from the imaging signal based on the shutter time information from the timing generation unit. The camera processing unit performs camera processing on the imaging signal and outputs image data. The buffer temporarily stores image data.

Further, when the flash detection unit detects the flash more than a predetermined number of times within a predetermined period, the timing generation unit controls to turn off the shutter operation of the imaging unit. The camera processing unit adjusts the gain for the imaging signal so that the output signal becomes a predetermined level when the timing generation unit controls the shutter operation of the imaging unit to be off. The camera processing unit further extracts image data of a plurality of continuous frames from the buffer when the shutter operation of the imaging unit is off and the flash detection unit detects a flash, and extracts the image data of the plurality of frames. Add and output.
In this imaging apparatus, it is possible to suppress white-band interference caused by external flash light such as flash light that spans a plurality of frames, and to obtain a bright captured image using flash light that is originally required.

An eighth invention is an integrated circuit used in an imaging apparatus including an imaging unit that acquires an imaging signal from a subject imaged by a lens unit, and includes a timing generation unit, a flash detection unit, a camera processing unit, Is provided.
The timing generation unit controls the shutter time of the imaging unit and outputs shutter time information indicating the shutter time. The flash detection unit detects the flash from the imaging signal based on the shutter time information from the timing generation unit. The camera processing unit performs camera processing on the imaging signal, and stores or outputs the image data in a buffer.
Further, the camera processing unit takes out the image data of a plurality of consecutive frames stored in the buffer according to the flash detection result of the flash detection unit, adds the image data of the plurality of frames, and outputs them.

In this integrated circuit, the image pickup apparatus can suppress white-band interference caused by external flash light such as flash light extending over a plurality of frames, and can obtain a bright captured image by flash light originally required.
A ninth invention is an image processing method used in an imaging apparatus including an imaging unit that acquires an imaging signal from a subject imaged by a lens unit, and a buffer that temporarily stores image data. A generation step, a flash detection step, a shutter operation control step, a camera processing step, a gain adjustment step, and an image data output step.
In the timing generation step, the shutter time of the imaging unit is controlled, and shutter time information indicating the shutter time is output. In the flash detection step, the flash is detected from the imaging signal based on the shutter time information. In the shutter operation control step, when the flash is detected a predetermined number of times or more within a predetermined period, the shutter operation of the imaging unit is turned off. In the camera processing step, camera processing is performed on the imaging signal and image data is output. In the gain adjustment step, when the shutter operation is controlled to be turned off in the shutter operation control step, the gain for the imaging signal is adjusted so that the output signal becomes a predetermined level. In the image data output step, when the shutter operation of the imaging unit is off and when the flash is detected in the flash detection step, the image data of a plurality of frames is extracted from the buffer, and the image data of the plurality of frames is extracted. Are added and output.

In this image processing method, the imaging apparatus can suppress white-band interference caused by external flash light such as flash light over a plurality of frames, and can obtain a bright captured image by flash light that is originally required.
According to a tenth aspect of the invention, a computer executes an image processing method used in an imaging apparatus that includes an imaging unit that acquires an imaging signal from a subject imaged by a lens unit, and a buffer that temporarily stores image data. A program comprising a timing generation step, a flash detection step, a shutter operation control step, a camera processing step, a gain adjustment step, and an image data output step.
In the timing generation step, the shutter time of the imaging unit is controlled, and shutter time information indicating the shutter time is output. In the flash detection step, the flash is detected from the imaging signal based on the shutter time information. In the shutter operation control step, when the flash is detected a predetermined number of times or more within a predetermined period, the shutter operation of the imaging unit is turned off. In the camera processing step, camera processing is performed on the imaging signal and image data is output. In the gain adjustment step, when the shutter operation is controlled to be turned off in the shutter operation control step, the gain for the imaging signal is adjusted so that the output signal becomes a predetermined level. In the image data output step, when the shutter operation of the imaging unit is off and when the flash is detected in the flash detection step, the image data of a plurality of frames is extracted from the buffer, and the image data of the plurality of frames is extracted. Are added and output.

  In this program, the imaging apparatus can suppress white-band interference caused by external flash light such as flash light over a plurality of frames, and can obtain a bright captured image by flash light that is originally required.

  As described above, the present invention provides an excellent effect of suppressing white band interference caused by external flash light such as flash light and obtaining a bright captured image by flash light that is originally required.

1 is a block diagram illustrating a configuration of an imaging device according to an embodiment of the present invention. Timing chart for explaining the operation of the image pickup apparatus when the shutter is off in the embodiment Timing chart for explaining the operation of the image pickup apparatus when the shutter is on in the embodiment Timing chart for explaining the operation of the image pickup apparatus when the shutter is on in the embodiment Timing chart of operation of image pickup apparatus in embodiment Block diagram showing the configuration of a conventional imaging device

Hereinafter, embodiments of the present invention will be described with reference to FIGS.
(Embodiment)
<1.1. Configuration of Imaging Device 100>
FIG. 1 is a block diagram showing a configuration of an imaging apparatus 100 according to an embodiment of the present invention.
In FIG. 1, an imaging apparatus 100 includes a lens unit 110, an imaging unit 101, a timing generation unit (TG) 102, a camera processing unit 103, a flash detection unit 104, a frame buffer (buffer) 105, and a recording unit. 106 and a display unit 107. The camera processing unit 103 and the flash detection unit 104 are configured by an integrated circuit such as an LSI. The integrated circuit may be configured to include the frame buffer 105.

The lens unit 110 includes an optical system.
The imaging unit 101 includes a CMOS imaging device and the like, acquires an imaging signal from a subject imaged by the lens unit 110, and outputs it to the camera processing unit 103 and the flash detection unit 104. The imaging unit 101 further receives a shutter operation control signal output from the timing generation unit 102 and controls the shutter time of the imaging unit 101 based on the signal. Note that the shutter time is a charge accumulation period by the image sensor adjusted by a so-called electronic shutter. Further, changing the shutter time includes switching between shutter-on and shutter-off as described later.
The timing generation unit 102 outputs a shutter operation control signal to the imaging unit 101 to control the shutter operation of the imaging unit 101, and outputs shutter time information indicating the shutter time to the flash detection unit 104. Here, the shutter time information is the shutter time itself in the imaging unit 101 or expressed as a ratio to the imaging interval in the imaging unit 101. The timing generation unit 102 also receives a shutter control signal output from the flash detection unit 104 described below, and controls the shutter time of the imaging unit 101 according to the signal.

The flash detection unit 104 receives an imaging signal output from the imaging unit 101 and shutter time information output from the timing generation unit 102. The flash detection unit 104 determines whether or not the flash has been burned based on the length of the period in which the imaging signal exceeds a predetermined signal level (the width in which the white belt lines shown in FIG. 2 are continuous) and the shutter time information.
As will be described later, the flash detection unit 104 further outputs a shutter control signal to the timing generation unit 102 when the imaging unit 101 is in the shutter-on state when a flash satisfying a predetermined condition is detected. The shutter operation 101 is turned off. The flash detection unit 104 outputs an image processing control signal (gain adjustment) to the camera processing unit 103 at a predetermined timing after switching the shutter operation to OFF, and the gain of the imaging signal is set according to the shutter time as described later. Let them adjust.

The flash detection unit 104 also outputs an image processing control signal (frame addition) to the camera processing unit 103 at a predetermined timing when it is determined that the flash is fired when the shutter is turned off as will be described later. Addition processing of frame image data is executed.
The camera processing unit 103 receives the imaging signal output from the imaging unit 101 and performs predetermined processing of the camera, such as black balance, white balance, gain adjustment, and gamma, on the imaging signal. The camera processing unit 103 also receives the image processing control signal (gain adjustment / frame addition) output from the flash detection unit 104, and adjusts the gain of the imaging signal in accordance with the control signal, or the frame buffer 105. A plurality of read image data of a plurality of frames are simultaneously read and added. Note that image data of a plurality of frames may be only added or averaged.

The frame buffer 105 temporarily stores image data of a plurality of continuous frames generated based on the imaging signal processed by the camera processing unit 103.
The recording unit 106 includes a recording medium such as a magnetic tape, an optical disk, a magnetic disk, or a semiconductor memory that records the image data processed by the camera processing unit 103. The display unit 107 includes a display device such as a CRT or LCD that displays the image data processed by the camera processing unit 103.
<1.2. Operation of Imaging Device 100>
The operation of the imaging apparatus 100 configured as described above will be described with reference to FIGS.
<1.2.1. Operation when the shutter is off>
First, a case will be described in which the flash detection unit 104 detects a flashed light when the shutter operation is off. Note that shutter-off refers to a state in which the charge accumulation period and the frame period of the image sensor are set to be the same.

FIG. 2 is a diagram schematically illustrating the accumulation period, readout, and scanning period of the CMOS image sensor. In FIG. 2, the frame period TF for driving the image sensor and the accumulation period TS for accumulating charges in the image sensor, that is, the shutter time TS, are the same, and the image sensing apparatus 100 is not performing a shutter operation (shutter off). The timing chart in the case is shown. In a so-called rolling shutter CMOS image sensor, the timing of the accumulation period TS is shifted for each line as shown in FIG. 2, and when the accumulation period TS ends for each line, the charge is read and an image signal is output during the scanning period. Is done. For this reason, when the flash is turned on at the timing shown in FIG. 2, the flash light affects the accumulation period TS of the second half line of frame 1 and the accumulation period TS of the first half line of frame 2. In the figure, the flash light spans different frame accumulation periods TS from Line 3 to Line 5.
In other words, in frame 1, the lines before Line 2 are not affected by the flash light, and Lines 3 to 5 are gradually affected by the flash light. After Line 6, it becomes brighter under the influence of the total amount of flash light.

Frame 2 is brighter than the total amount of flash light before Line 2, and the amount of light gradually decreases from Line 3 to Line 5, and is not affected by the flash light after Line 6.
As described above, as a result of the influence of the flash light, a white belt-like disturbance appears in the camera output image. The frame 1 has a bright lower side and the frame 2 has a bright upper side.
As is clear from the principle of white band generation, the time width of the white band is equal to the accumulation period TS, and the time width during which the signal level changes at the edge of the white band is equal to the flash emission time. Note that the flash emission time is sufficiently short with respect to the frame period, and the change in the signal level at the edge of the white band is steep.
In other words, when the shutter is off, it is possible to detect that the flash has been burned by comparing the time width of the white band in which bright lines exceeding a predetermined level are continuously output to the imaging signal and the frame period. . In other words, in consecutive frames, if the line position where the white band starts in the front frame and the line position where the white band ends in the rear frame are at the same line position, it can be seen that the flash was imaged.

Note that the line position where the white band starts and the line position where the white band ends can be specified as follows, for example. For example, the flash detection unit 104 can detect a predetermined steep level change as shown in the camera output level of FIG. 2 and can identify these line positions by detecting timings t1 and t2 between the level changes. .
On the other hand, even if a bright line exceeding a predetermined level exists in the camera output, if the start position and the end position are not in the same line position, it does not mean that the subject where the flash has been shot is taken. It can be distinguished as a photograph of a bright subject.
Since the flash is spread randomly over time, the white belt often spans two consecutive frames, but only a single frame can be bright. However, in any case, when the shutter operation is off, the time width of the white band coincides with the length of one frame, that is, the frame period.

When the white band extends over two frames of the image pickup signal as shown in FIG. 2, the camera processing unit 103 reads the image data of continuous frames 1 and 2 over which the white band extends over the frame buffer 105. Then, by changing the processing so that the camera processing unit 103 adds the two frames, an image in which the entire frame is uniformly brightened can be obtained. If the white band does not extend over the two frames of the image signal and only a single frame becomes bright due to the flashing timing, it is not necessary to add it to either the previous or next frame. There is no harm.
<1.2.2. Operation when shutter is on>
Next, the case where the shutter is applied in driving the CMOS image sensor (shutter ON) will be described with reference to the timing charts of FIGS. Note that shutter-on refers to a state in which the charge accumulation period of the image sensor is set shorter than the frame period.

When the shutter is on, the accumulation period, that is, the shutter time TS is shorter than the frame period as shown in the figure, so that the number of lines affected by the flash light is less than one frame. For this reason, the width of the white belt-like interference appearing in the camera output image is narrower than that in the case of shutter-off. However, also in this case, the time width of the white belt coincides with the accumulation period TS.
That is, even when the shutter is on, it is possible to detect that the flash has been burned by determining whether or not the time width of the white band in which bright lines exceeding a predetermined level continue matches the accumulation period TS.
However, when the shutter is on, adding the frame 1 and the frame 2 shown in FIG. 4 does not result in an image in which the entire frame is brightened.
Therefore, in the imaging apparatus 100 according to the present embodiment, when the flash is frequently applied, that is, when the flash detection unit 104 detects flash emission more than a predetermined number of times within a predetermined period, the timing generation unit 102 A shutter control signal is outputted to control the shutter operation of the imaging unit 101 to be turned off. Thereby, as shown in FIG. 2, it is possible to control the time width of the white belt to be one frame. In addition, after the shutter operation is switched off, an image processing control signal (gain adjustment) is output from the flash detection unit 104 to the camera processing unit 103 at a predetermined timing, and gain adjustment of the imaging signal is performed. Details of the gain adjustment will be described later.

When the flash is fired after the shutter is turned off, an image processing control signal (frame addition) is output from the flash detection unit 104 to the camera processing unit 103, and the camera processing unit 103 continues over the white belt. By adding the two frames, an image in which the entire frame is uniformly brightened is obtained.
<1.2.3. Operation Timing of Imaging Device 100>
FIG. 5 is a timing chart of the operation of the imaging apparatus 100 according to this embodiment.
5, (A) is an imaging signal output from the imaging unit 101, (B) is a gain for gain adjustment applied to the imaging signal in the camera processing unit 103, and (C) is a camera processing unit for the imaging signal. An imaging signal after gain adjustment is performed at 103, and (D) is a camera output signal that is finally output from the camera processing unit 103.

Hereinafter, the operation of the apparatus will be described along with changes in signals output from the respective units of the illustrated imaging apparatus 100.
First, when the imaging unit 101 is operating in the shutter-on state, the accumulation period TS is input from the timing generation unit 102 to the flash detection unit 104 as shutter time information. When a bright line exceeding a predetermined level is continuously output from the time (a) to (a ′) in the image pickup signal (A) from the image pickup unit 101, the flash detection unit 104 determines the length and the accumulation period TS. If they match, it is determined that the subject in which the flash has been shot is captured. When it is determined that a flash has been burned more than a predetermined number of times within a predetermined period, that is, when it is detected that a flash has been burned more than a certain frequency, the timing generation unit 102 The shutter operation is turned off, and the gain of the camera processing unit 103 is changed as shown in (B).

At this time, if the shutter operation of the imaging unit 101 is turned off from on, the signal level of the imaging signal output from the imaging unit 101 increases in proportion to the length of the accumulation period. To change. Specifically, gain adjustment corresponding to the shutter time is performed based on an image processing control signal (gain adjustment) input from the flash detection unit 104 to the camera processing unit 103. In gain adjustment, the reference is (1.0) when the shutter is on, and the gain is (TS / TF) when the shutter is off. Thereby, the level of the imaging signal can be maintained as shown in (C) before and after the shutter operation is turned off.
Next, when the imaging unit 101 operates in the shutter-off state, the accumulation period TS output as the shutter time information from the timing generation unit 102 is equal to the frame period TF. At this time, when a bright line exceeding a predetermined level is continuously output from the time (c) to (c ′) in the image pickup signal (A) from the image pickup unit 101, the flash detection unit 104 The storage period TS is compared, and if they match, it is determined that the subject in which the flash has been shot is captured. When the flash detection unit 104 detects a flash, the camera processing unit 103 takes out the frames 4 and 5 including the white belt-like interference caused by the flash from the frame buffer 105 and outputs the result as new frames 4 ′ and 5 ′. . In the shutter-off state, the line position of frame 4 where the white belt-like disturbance due to the flash begins coincides with the line position of frame 5 where the white belt-like disturbance ends. It is possible to obtain an image that is uniformly bright as a whole.

In FIG. 5, the signal level of the white band-like interference caused by the flash has been described as being binary, but the frame is added even if the signal level has a gradient of the flash emission time width at the edge of the white band. The result is similar.
<1.3. Effects of the present embodiment>
As described above, according to the present embodiment, the flash detection unit 104 detects the flash, and when the flash is detected, the camera processing unit 103 performs addition processing of a plurality of frames, thereby capturing the flash. In addition to suppressing white belt-like interference, it is possible to obtain an image in which the entire surface is uniformly brightened using a conventional CCD image sensor.
In addition, according to the above embodiment, the flash detection unit 104 detects frequent flashes and switches the shutter operation off, so that it can reliably detect the flash and is originally necessary. It is possible to more reliably obtain a bright captured image using flash light.

In addition, according to the above-described embodiment, the camera processing unit 103 performs addition processing of a plurality of frames only when a flash is detected when the shutter operation is off. Can be obtained more reliably.
Furthermore, according to the above embodiment, when the shutter operation is switched off by flash detection, the camera processing unit 103 adjusts the gain of the imaging signal. Thereby, it is possible to suppress a difference in brightness of the image obtained between when the shutter operation is on and when the shutter operation is off, and it is possible to guarantee the quality of the image.
<2. Modification>
<2.1. Modification 1>
In the embodiment described above, the flash detection unit 104 detects that the flash has been blown frequently and switches to shutter-off to suppress white band interference. In addition, the flash detection unit 104 may control the imaging unit 101 to return to the original shutter-on state when no flash is detected for a predetermined period in the shutter-off state. .

Thereby, it is possible to prevent the shutter-off state from continuing unnecessarily.
<2.2. Modification 2>
In the above embodiment, when the shutter operation is switched off by the flash detection, the camera processing unit 103 adjusts the gain of the imaging signal, but it may not be performed.
(Other embodiments)
In the imaging apparatus 100 described in the above embodiment, each block may be individually made into one chip by a semiconductor device such as an LSI, or may be made into one chip so as to include a part or the whole.
Note that the name used here is LSI, but it may also be called IC, system LSI, super LSI, or ultra LSI depending on the degree of integration.

Further, the method of circuit integration is not limited to LSI, and implementation with a dedicated circuit or a general-purpose processor is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI, or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used.
Further, if integrated circuit technology comes out to replace LSI's as a result of the advancement of semiconductor technology or a derivative other technology, it is naturally also possible to carry out function block integration using this technology. Biotechnology can be applied as a possibility.
Moreover, each process of the said embodiment may be implement | achieved by hardware, and may be implement | achieved by software. Further, it may be realized by mixed processing of software and hardware. Needless to say, when the imaging apparatus 100 according to the above-described embodiment is realized by hardware, it is necessary to perform timing adjustment for performing each process. In the above embodiment, for convenience of explanation, details of timing adjustment of various signals generated in actual hardware design are omitted.

  The specific configuration of the present invention is not limited to the above-described embodiment, and various changes and modifications can be made without departing from the scope of the invention.

  The present invention can be widely applied as an imaging apparatus such as a video camera using a CMOS imaging device, an integrated circuit, an image processing method, and a program.

DESCRIPTION OF SYMBOLS 10 Imaging device 100 Imaging device 101 Imaging part 102 Timing generation part 103 Camera processing part 104 Flash detection part 105 Frame buffer 110 Lens part

Claims (10)

An imaging unit that acquires an imaging signal from a subject imaged by the lens unit;
A timing generator that controls shutter time of the imaging unit and outputs shutter time information indicating the shutter time;
A flash detection unit that detects a flash from an imaging signal based on shutter time information from the timing generation unit;
A camera processing unit that performs camera processing on the imaging signal and outputs image data;
A buffer for temporarily storing image data;
With
The camera processing unit takes out image data of a plurality of continuous frames from the buffer according to a flash detection result of the flash detection unit, adds the image data of the plurality of frames, and outputs,
Imaging device.
When the flash detection unit detects a flash, the timing generation unit controls to turn off the shutter operation of the imaging unit,
The imaging device according to claim 1.
When the flash detection unit detects flash more than a predetermined number of times within a predetermined period, the timing generation unit controls to turn off the shutter operation of the imaging unit;
The imaging device according to claim 1.
When the timing generation unit controls the shutter operation of the imaging unit to be off, the camera processing unit adjusts the gain for the imaging signal so that the output signal becomes a predetermined level.
The imaging device according to claim 2 or 3.
The camera processing unit takes out image data of a plurality of consecutive frames from the buffer only when the shutter operation of the imaging unit is off, adds the image data of the plurality of frames, and outputs,
The imaging device according to claim 1.
When the flash detector does not detect a flash within a predetermined period, the timing generator controls to turn on the shutter operation of the imaging unit;
The imaging device according to claim 1.
An imaging unit that acquires an imaging signal from a subject imaged by the lens unit;
A timing generator that controls shutter time of the imaging unit and outputs shutter time information indicating the shutter time;
A flash detection unit that detects a flash from an imaging signal based on shutter time information from the timing generation unit;
A camera processing unit that performs camera processing on the imaging signal and outputs image data;
A buffer for temporarily storing image data;
With
When the flash detection unit detects the flash more than a predetermined number of times within a predetermined period, the timing generation unit controls to turn off the shutter operation of the imaging unit,
The camera processing unit
When the timing generation unit controls the shutter operation of the imaging unit to be off, the gain for the imaging signal is adjusted so that the output signal becomes a predetermined level,
When the shutter operation of the imaging unit is off and the flash detection unit detects a flash, the image data of a plurality of consecutive frames is extracted from the buffer, and the image data of the plurality of frames is added. ,Output,
Imaging device.
An integrated circuit used in an imaging device including an imaging unit that acquires an imaging signal from a subject imaged by a lens unit,
A timing generator that controls shutter time of the imaging unit and outputs shutter time information indicating the shutter time;
A flash detection unit that detects a flash from an imaging signal based on shutter time information from the timing generation unit;
A camera processing unit that performs camera processing on an imaging signal and stores or outputs image data in a buffer;
With
The camera processing unit takes out the image data of a plurality of consecutive frames stored in the buffer according to the flash detection result of the flash detection unit, adds the image data of the plurality of frames, and outputs,
Integrated circuit.
An image processing method used in an imaging apparatus comprising: an imaging unit that acquires an imaging signal from a subject imaged by a lens unit; and a buffer that temporarily stores image data,
A timing generation step of controlling shutter time of the imaging unit and outputting shutter time information indicating the shutter time;
A flash detection step for detecting a flash from the imaging signal based on the shutter time information;
A shutter operation control step for turning off the shutter operation of the imaging unit when a flash is detected a predetermined number of times or more within a predetermined period; and
A camera processing step of performing camera processing on the imaging signal and outputting image data;
A gain adjustment step of adjusting the gain for the imaging signal so that the output signal becomes a predetermined level when the shutter operation is controlled to be off in the shutter operation control step;
When the shutter operation of the imaging unit is off and a flash is detected in the flash detection step, image data of a plurality of consecutive frames is extracted from the buffer, and the image data of the plurality of frames is added. Output image data output step;
An image processing method comprising:
A program that causes a computer to execute an image processing method used in an imaging apparatus including an imaging unit that acquires an imaging signal from a subject imaged by a lens unit, and a buffer that temporarily stores image data.
A timing generation step of controlling shutter time of the imaging unit and outputting shutter time information indicating the shutter time;
A flash detection step for detecting a flash from the imaging signal based on the shutter time information;
A shutter operation control step for turning off the shutter operation of the imaging unit when a flash is detected a predetermined number of times or more within a predetermined period; and
A camera processing step of performing camera processing on the imaging signal and outputting image data;
A gain adjustment step of adjusting the gain for the imaging signal so that the output signal becomes a predetermined level when the shutter operation is controlled to be off in the shutter operation control step;
When the shutter operation of the imaging unit is off and a flash is detected in the flash detection step, image data of a plurality of consecutive frames is extracted from the buffer, and the image data of the plurality of frames is added. Output image data output step;
A program for causing a computer to execute an image processing method.

Images