JP5132125B2 - IMAGING DEVICE, ITS CONTROL METHOD, PROGRAM, AND STORAGE MEDIUM - Google Patents

Description

 The present invention relates to a technique for performing an imaging operation using both a mechanical shutter and an electronic shutter in an imaging apparatus.

  A CCD or CMOS image sensor is generally used as an imaging device used in an imaging apparatus such as a digital camera. An image pickup apparatus using these image pickup elements has a narrow dynamic range as compared with a silver halide photographic system, particularly a negative film, and a signal is crushed in a high-luminance part and a low-luminance part, and gradation expression is insufficient. In order to ensure a wide dynamic range, the sensitivity of the image sensor with respect to the subject brightness may be reduced. However, this reduces the S / N and degrades the image quality.

  Therefore, as a means for obtaining an image with a wide dynamic range, a method of performing a photographing operation a plurality of times with different exposure amounts and synthesizing the obtained images has been considered. However, it is not suitable for shooting a moving subject as a still image because the shooting times of the images to be combined are different.

  In an image pickup apparatus using a CCD, the exposure state of image data obtained from the output of the image pickup device is controlled by controlling the charge accumulation time from when the charge is discharged until the accumulated charge is transferred. be able to. This operation is usually called an electronic shutter. Patent Document 1 discloses a method for obtaining an image having a wide dynamic range by taking advantage of this feature. According to the contents, after opening the mechanical shutter, the charge of all the pixels of the CCD is first discharged, and the exposure state is controlled by transferring the accumulated charge in the odd field when exposed for a predetermined time. In the even field, the exposure state is controlled by closing the mechanical shutter after further accumulation. Then, the dynamic range of the image is expanded by combining the obtained two field images.

This method is considered to solve the above problem because only one photographing operation is required. However, while the exposure start time due to charge discharge is the same for all pixels, the exposure end time due to the mechanical shutter varies depending on the shutter running direction position. Therefore, there is a problem that the exposure state varies depending on the pixels in the even field. However, it is considered that this problem can be avoided by performing exposure control of the even field by transferring the stored charge as in the odd field.
JP-A-9-214829 Japanese Patent Laid-Open No. 11-41523

  On the other hand, a CMOS image sensor, which is an XY address type image pickup device, has advantages such as the occurrence of smear is so small that it can be ignored compared to a CCD, and the development of large-format technology is progressing. However, in the electronic shutter operation of the CMOS image sensor, charge accumulation is performed at a different timing for each row as a so-called rolling shutter, and the accumulation operation cannot be completed simultaneously for all pixels. If exposure control is performed by controlling the charge accumulation time of the CMOS image sensor, the accumulation period is shifted by about one frame between the first and last rows of the scanning line, so that a moving subject is imaged as a still image. Is not considered suitable. Therefore, the exposure end control of the CMOS image sensor needs to be performed by a mechanical shutter, and the configuration as in Patent Document 1 cannot be taken.

  By the way, in the electronic shutter operation of the CMOS image sensor, the reset operation in each row for starting the charge accumulation is performed ahead of the time required for the charge accumulation from the timing of the read operation of the signal level of the accumulated charge in each row. It will be. The speed of the reset operation can be made different from the scanning speed of the read operation of the stored charge signal level.

  Using this, Patent Document 2 discloses a configuration in which exposure control is performed by performing reset operation of the CMOS image sensor line by line at a speed along the travel of the mechanical shutter. In the configuration disclosed in Patent Document 2, reset operation is performed line by line at a speed along the travel of the mechanical shutter, charge accumulation is started, light is shielded by the mechanical shutter, and then the signal level of the accumulated charge is read. The operation is performed line by line. The exposure control of the image data can be performed by adjusting the reset operation and the interval between the mechanical shutters. In this method, the reset operation is performed line by line at a speed along the travel of the mechanical shutter. Therefore, the difference in accumulation time between the first line and the last line of the scanning line is provided with a light shielding blade as a front curtain (hereinafter referred to as a front blade) and a light shielding blade as a rear curtain (hereinafter referred to as a rear blade). This can be improved to the same extent as when a mechanical shutter is used. According to this configuration, since a CMOS image sensor is used, it is possible to suppress smear during moving image capturing and to enable a high-speed shutter and to capture a still image of a moving subject with the CMOS image sensor. It becomes.

  However, Patent Document 2 does not mention a method for obtaining an image having a wide dynamic range.

  Accordingly, the present invention has been made in view of the above-described problems, and an object of the present invention is to widen a dynamic range while maintaining good S / N in an imaging apparatus using a CMOS image sensor, and to make a moving object stationary. It is also necessary to ensure adaptability to image shooting.

In order to solve the above-described problems and achieve the object, an imaging apparatus according to the present invention includes an imaging element in which a large number of pixels are arranged in a matrix and a shutter curtain that travels so as to shield the imaging element. A mechanical shutter; reset scanning means for performing reset scanning for resetting charges accumulated in each pixel of the image sensor and starting exposure; and an exposure time of a second row than the first row of the image sensor. Using the reset scanning means and the control means for controlling the mechanical shutter, the first pixel data read from the first row, and the second pixel data read from the second row so as to be shortened a generating means for generating the image data, wherein the generating means, the data corresponding to said second row in the image data, read out from the second row The second pixel data if the value of the second pixel data is equal to or more than the first predetermined value based on the exposure time of the saturation level and the first and second row of the second pixel data and data based on, if the second value of the second pixel data read out from the row is less than the first predetermined value, characterized in that the data based on the first pixel data .

The image pickup apparatus control method according to the present invention includes an image pickup device in which a large number of pixels are arranged in a matrix, a mechanical shutter having a shutter curtain that travels so as to shield the image pickup device, and each of the image pickup devices. Reset scanning means for performing reset scanning for resetting charges accumulated in the pixels and starting exposure, and the reset scanning means so that the exposure time of the second row is shorter than that of the first row of the image sensor. And a control means for controlling the mechanical shutter, wherein the first pixel data read from the first row and the second pixel data read from the second row are obtained. has a generation step of generating image data by using the generating step, the data corresponding to the second row in the image data, the second The If the value of the second pixel data read out of the first predetermined value or more based on the exposure time of the saturation level and the first and second row of pixel data of the second row from the second pixel data, the a value of the second pixel data read out is less than the first predetermined value from the second row, characterized in that the data based on the first pixel data And

  A program according to the present invention causes a computer to execute the above control method.

  A storage medium according to the present invention stores the above program.

  According to the present invention, in an imaging apparatus using a CMOS image sensor, it is possible to widen the dynamic range and to ensure adaptability to still image shooting of a moving object while maintaining good S / N.

  Hereinafter, a preferred embodiment of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a diagram illustrating a configuration of an imaging apparatus according to an embodiment of the present invention.

  In FIG. 1, reference numeral 101 denotes an image sensor in which a large number of pixels are arranged in a matrix, and a CMOS image sensor is used. An analog front end (AFE) 102 has functions such as analog-digital conversion, OB clamp, and gain variable amplifier for the signal from the image sensor 101. The OB clamping process is a process of performing offset adjustment on the signal so that the output of the light shielding portion (optical black: OB) of the image sensor becomes a predetermined value. Reference numeral 103 denotes a DSP (Digital Signal Proseccer) that performs various correction processes and development processes on data from the AFE 102. Further, the DSP 103 performs control of various memories such as the ROM 106 and the RAM 107, and processing for writing image data to the recording medium 108.

  A drive signal generator 104 supplies a clock signal and a control signal to the image sensor 101, AFE 102, and DSP 103, and is controlled by the CPU 105.

  Reference numeral 105 denotes a CPU that controls the DSP 103 and the drive signal generator 104, and controls camera functions using various parts (not shown) such as photometry and distance measurement. The CPU 105 is connected with a power switch 109, a shutter switch (SW1) 110, a shutter switch (SW2) 111, and a mode dial 112, and executes processing corresponding to each state.

  A ROM 106 stores a camera control program and various correction data, and a RAM 107 temporarily stores image data and correction data processed by the DSP 103. The RAM 107 can be accessed at a higher speed than the ROM 106.

  Reference numeral 108 denotes a recording medium such as a compact flash (registered trademark) card for storing a photographed image, which is connected to a camera via a connector (not shown).

  Reference numeral 109 denotes a power switch for starting the camera, and reference numeral 110 denotes a shutter switch SW1 for instructing start of operations such as photometry processing and distance measurement processing. Reference numeral 111 denotes a shutter switch SW2 that drives the mirror member 113 and the mechanical shutter 115 and instructs the start of a series of imaging operations to write a signal read from the imaging element 101 to the recording medium 108 via the AFE 102 and the DSP 103. A mode dial 112 is a dial switch for setting various operation modes of the camera. The shutter switch (SW1) 110 is a switch that is turned on when a release button (not shown) is half-pressed, and the shutter switch (SW2) 111 is a switch that is turned on when the release button (not shown) is fully pressed.

  Reference numeral 113 denotes a mirror member located in the photographing optical path. When the imaging apparatus is in a non-photographing state (the state shown in FIG. 1), a part of the subject light flux that has passed through the photographing lens (not shown) is The light is reflected by the mirror member 113 and guided to the finder optical system 114. Thereby, the photographer can observe the subject image via the finder optical system 114.

  When the shutter switch SW2 is turned on and the imaging apparatus shifts from the non-photographing state to the photographing state, the mirror member 113 is retracted from the photographing optical path. As a result, the subject luminous flux from the photographing lens is directed toward the image sensor 101 that is a CMOS image sensor. Here, a focal plane shutter (hereinafter, mechanical shutter) 115 is arranged on the object side (lens side) with respect to the image sensor 101. The mechanical shutter 115 has a mechanical rear curtain (shutter curtain) composed of a plurality of light shielding blades.

  Reference numeral 116 denotes a shutter driving circuit, and the CPU 105 controls driving of the mechanical shutter 115 via the shutter driving circuit 116.

  The image pickup apparatus of the present embodiment employs two electronic shutters (electronic front curtain 1 and electronic front curtain 2) having different travel timings depending on the line of the image sensor 101 instead of a general mechanical front curtain. It is configured to shoot using a curtain and a mechanical rear curtain.

  FIG. 2A is a diagram illustrating a state in which the mechanical rear curtain of the image sensor 101 and the mechanical shutter 115 is observed from the lens side along the optical axis direction.

  201 indicates a pixel region of the image sensor 101, and 202 indicates a mechanical shutter 115 (mechanical rear curtain). At the time of shooting, charge accumulation start scanning is performed to clear the accumulated charge of the pixels line by line and start charge accumulation for shooting. Hereinafter, such a charge accumulation start scan is referred to as a reset scan, and a line that is actually being reset is referred to as a reset line. The reset scanning is performed twice during one imaging, and each of the odd and even rows is targeted. Here, the reset scan and reset line for the odd-numbered rows are referred to as reset scan 1 and reset line 1, respectively, and the reset scan and reset line for the even-numbered rows are referred to as reset scan 2 and reset line 2, respectively. In the figure, 203 represents the reset line 1 and 204 represents the reset line 2. In both cases, the reset scanning is performed in the direction of the arrow 205. Since the subject image formed on the image sensor 101 via the photographing lens is inverted up and down, the reset scanning is performed from the upper part to the lower part of the image.

After a predetermined time has elapsed from the start of reset scanning, the mechanical rear curtain starts to travel. Reference numeral 206 denotes a leading end of the mechanical rear curtain, and an area 207 formed by the reset line 1 (203) and the mechanical rear curtain leading end 206 is an area where charge accumulation of odd lines is performed by exposure (charge accumulation area 1). ). Similarly, a region 208 formed by the reset line 2 (204) and the mechanical rear curtain leading end 206 is a region (charge storage region 2) where even-numbered lines of charge are stored by exposure.

  FIG. 2B shows how the electronic front curtain and the mechanical rear curtain are driven with the vertical axis representing the vertical position of the image sensor and the horizontal axis representing time.

  Reference numerals 209 and 210 denote transitions of the reset line 1 (electronic front curtain 1) and the reset line 2 (electronic front curtain 2), respectively. 211 is a state of running of the mechanical rear curtain. The time (Δt12 + Δt23) from the start time t1 of the reset scanning 1 to the mechanical rear curtain travel start time t3 corresponds to the exposure time for odd lines. Similarly, the time Δt23 from the start time t2 of the reset scanning 2 to the mechanical rear curtain travel start time t3 corresponds to the exposure time of even lines. Since the electronic front curtain 1 and the electronic front curtain 2 are reset and scanned at a speed according to the running characteristics of the mechanical rear curtain, there is no difference in exposure time between the upper and lower images.

  As described above, by independently performing reset scanning in odd rows and even rows, it is possible to acquire image signals having different exposure times in one shooting operation.

  Next, the control flow of the camera will be described with reference to FIG.

  First, in step S301, it is determined whether or not the power switch 109 for starting the camera is turned on. If it is turned off, step S301 is repeated. If the power switch 109 is ON, it is determined in step S302 whether or not the mode dial 112 is set to the shooting mode. If the shooting mode is set, the process proceeds to step S304. If any other mode is set, the process corresponding to the mode selected in step S303 is performed, and the process returns to step S301.

  In step S304, it is determined whether the shutter switch (SW1) 110 is ON. If SW1 is OFF, the process of step S304 is repeated. If SW1 is ON, the process proceeds to step S305.

  In step S305, photometry processing for determining the aperture value and shutter speed and distance measurement processing for adjusting the focus of the taking lens to the subject are performed using a photometry control unit and a distance measurement control unit (not shown).

  When the photometry / ranging process is completed, the state of the shutter switch (SW2) 111 is determined in a subsequent step S306. If SW2 is OFF, step S306 is repeated. If SW2 is ON, the photographing process of step S307 is executed. Details of this photographing process will be described later.

  When the photographing process is completed, the process proceeds to step S308, where the DSP 103 performs a developing process on the photographed image data. In step S309, the image data that has undergone development processing is compressed and stored in an empty area of the RAM 107.

  In step S310, the image data stored in the RAM 107 is read and recording processing on the recording medium 108 is executed. After the recording process is completed, the process returns to step S301 to prepare for the next shooting.

  Next, details of the photographing process in step S307 will be described with reference to FIG.

  First, in step S401, the mirror member 113 is moved to the mirror up position. In step S402, the aperture is driven to a predetermined aperture value based on the photometric data obtained in the above-described photometric processing (step S305).

  In step S403, reset scanning 1 of the image sensor 101 is started via the drive signal generator 104, and the electronic front curtain 1 is caused to travel. As a result, in the pixels arranged on the odd lines, the charges accumulated so far are once cleared, and charge accumulation for actual photographing is started.

  In the next step S404, after waiting for the time Δt12 to elapse from the start of the electronic front curtain 1 travel, reset scanning 2 is started in step S405 and the electronic front curtain 2 travels. By this scanning, the charges accumulated so far in the pixels arranged in the even lines are once cleared, and charge accumulation for the actual photographing is started.

  After the electronic front curtain 2 travels, after waiting for a further time Δt23 to pass in step S406, the mechanical rear curtain travel starts in step S407 and the exposure ends. Through the operations so far, the charges corresponding to the exposure during (Δt12 + Δt23) are accumulated in the pixels arranged in the odd rows, and the charges corresponding to the exposure during Δt23 are accumulated in the pixels arranged in the even rows. It will be in the state.

  When the running of the mechanical rear curtain is completed, the aperture is driven to the open aperture value in step S408, and the mirror member 113 is driven to the mirror down position in step S409.

  In step S410, the process waits until the set charge accumulation time elapses, and ends the charge accumulation of the image sensor 101 (step S411). Finally, in step S412, the signal from the image sensor 101 is read out, and a series of processes is terminated and the process returns to the main process.

  The above is the flow of a series of shooting operations of the camera in this embodiment.

  Next, an example of a method for generating an image having a wide dynamic range from the image signal obtained by reading the imaging signal in step 412 will be described with reference to FIGS.

  Note that the process of synthesizing image signals to generate an image with a wide dynamic range may be performed by the DSP 103 inside the imaging apparatus or by an image processing apparatus outside the imaging apparatus.

FIG. 5 is a diagram showing the relationship between subject brightness and pixel data. Since the exposure times of the odd line (ODD) and the even line (EVEN) are (Δt12 + Δt23) and Δt23, respectively, the relationship between the pixel data in the region below Psat1 is
ODD = EVEN * (Δt12 + Δt23) / Δt23
Can be considered.

Sat1 is a saturation point of the pixel data, and is a value determined by a circuit configuration inside the image sensor 101, A / D conversion performed by the AFE 102 , and the like. Assuming that the subject luminance at which the pixel data of the odd-numbered line and the even-numbered line reach Sat1, the relation is also Psat2 = Psat1 * (Δt12 + Δt23) / Δt23
Thus, the even line has a dynamic range (Δt12 + Δt23) / Δt23 times that of the odd line.

  However, on the contrary, since the exposure time is short for the same luminance, the ratio of shot noise to the optical signal component is high in the even lines, and the S / N is low compared to the odd lines. For this reason, pixel data of odd lines with good S / N is used in an area where the subject brightness is less than Psat1, and pixel data is synthesized using data of even lines only in an area where the subject brightness is Psat1 or higher.

  FIG. 6A illustrates a partial pixel arrangement of the image sensor 101. Pixels shown in white are pixel groups arranged on odd lines, and pixels shown in hatching are pixel groups arranged on even lines. FIG. 6B shows pixel data corresponding to each line.

Now, focusing on the pixel data d53 and d54, the combined pixel data d53 ′ and d54 ′ are calculated by the following equations. If β = (Δt12 + Δt23) / Δt12,
d53 ′ = d53: d53 <Sat1
d53 ′ = β * (d52 + d54) / 2: d53 = Sat1
d54 ′ = (d53 + d55) / 2: d54 <Sat1 / β
d54 ′ = β * d54: d54 ≧ Sat1 / β
It is.

  As a result, it is possible to obtain an image with a wide dynamic range in which a signal is not destroyed even in a high luminance region while maintaining a good S / N from a low luminance region to a medium luminance region. The dynamic range is expanded by (Δt12 + Δt23) / Δt23 times compared to the case of only odd lines.

  The above is the description of the embodiment of the present invention. With this, it is possible to obtain an image with a wide dynamic range while maintaining good S / N. In addition, since pixel data having different exposure times can be simultaneously acquired by one shooting, an imaging device suitable for still image shooting of a moving subject can be provided.

  In this embodiment, the reset scanning target line of the electronic front curtain 1 and the reset scanning target line of the electronic front curtain 2 are odd lines and even lines, respectively, and are alternately arranged for each line. However, of course, the arrangement is not limited to this, and any arrangement may be used as long as it is arranged periodically. For example, as shown in FIG. 7, by arranging the lines alternately every two lines, R (red), G (green), and B (blue) of the Bayer array of color filters for each of two image signals having different exposure times. ) Signal component. By adopting this configuration, it becomes easy to combine pixel data having different exposure times.

  In the present embodiment, two types of regions having different exposure times are formed by the electronic front curtain 1 and the electronic front curtain 2, but it is needless to say that three types of regions may be formed. In that case, as shown in FIG. 8, reset scanning for the electronic front curtain needs to be performed once more (three times in total) during the photographing operation. At this time, as shown in FIG. 9, if the reset scanning target lines are arranged in a cycle of one line per three lines and different from each other, it is easy to perform the image signal composition processing after photographing.

  Further, the method for synthesizing pixel data having different exposure times is not limited to that described in the present embodiment, and may be based on another calculation formula.

  In addition, a driving mode for forming regions with different exposure times by a plurality of electronic front curtains as in this embodiment, and a driving mode for only one electronic front curtain for all rows are prepared separately. One of the drive modes may be selected according to the shooting mode. In the former mode, a wide dynamic range can be ensured. However, in some cases, the resolution of the image may be sacrificed in order to perform data calculation between pixels. In the latter mode, the dynamic range remains narrow but the resolution is the same as before. Therefore, any one of the drive modes may be selected according to the shooting mode of the camera.

(Other embodiments)
The object of each embodiment is also achieved by the following method. That is, a storage medium (or recording medium) in which a program code of software that realizes the functions of the above-described embodiments is recorded is supplied to the system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but the present invention includes the following cases. That is, based on the instruction of the program code, an operating system (OS) running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  Furthermore, the following cases are also included in the present invention. That is, the program code read from the storage medium is written in a memory provided in a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, based on the instruction of the program code, the CPU or the like provided in the function expansion card or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

  When the present invention is applied to the above storage medium, the storage medium stores program codes corresponding to the procedure described above.

1 is an overall configuration diagram of a digital camera according to an embodiment of the present invention. It is the figure which showed the relationship between the electronic front curtain and the mechanical rear curtain in one Embodiment of this invention. It is a flowchart which shows control of the camera in one Embodiment of this invention. It is a flowchart which shows the imaging | photography process operation | movement of the camera in one Embodiment of this invention. It is the figure which showed the relationship between to-be-photographed object brightness | luminance and pixel data in one Embodiment of this invention. It is the figure which showed pixel arrangement | positioning and pixel data in one Embodiment of this invention. It is the figure which showed pixel arrangement | positioning and pixel data in one Embodiment of this invention. It is the figure which showed the relationship between the electronic front curtain and the mechanical rear curtain in one Embodiment of this invention. It is the figure which showed pixel arrangement | positioning and pixel data in one Embodiment of this invention.

Explanation of symbols

101 Image sensor 102 AFE
103 DSP
104 Timing generation circuit 105 CPU
106 ROM
107 RAM
108 Recording medium 109 Power switch 110, 111 Shutter switch 112 Mode dial 113 Mirror member 114 Viewfinder optical system 115 Mechanical shutter 116 Shutter drive circuit

Claims (8)

An image sensor in which a large number of pixels are arranged in a matrix;
A mechanical shutter having a shutter curtain that travels to shield the image sensor;
Reset scanning means for performing reset scanning for resetting the charge accumulated in each pixel of the image sensor and starting exposure;
Control means for controlling the reset scanning means and the mechanical shutter so that the exposure time of the second row is shorter than the first row of the image sensor;
Generating means for generating image data using first pixel data read from the first row and second pixel data read from the second row;
The generating means is configured to obtain data corresponding to the second row in the image data, and the value of the second pixel data read from the second row is equal to a saturation level of the second pixel data and the second pixel data. When the first predetermined value or more based on the exposure times of the first and second rows is greater than or equal to the second pixel data, the second pixel data read from the second row imaging device if the value is less than said first predetermined value, characterized in that the data based on the first pixel data. The first row and the second row are arranged on the image sensor so as to be adjacent to each other at a constant period,
Said generating means, said data corresponding to the second row in the image data, the second case the value of the second pixel data read out from the row is less than the first predetermined value the The imaging apparatus according to claim 1, wherein the imaging device is data based on the first pixel data read from the first row adjacent to the second row. The generation unit generates data corresponding to the first row in the image data from a second value in which the value of the first pixel data read from the first row is greater than the first predetermined value. If the value is less than the predetermined value, the first pixel data is used. If the value of the first pixel data read from the first row is not less than the second predetermined value, the first row is used. The imaging apparatus according to claim 2, wherein the imaging device is data based on the second pixel data read from the second row adjacent to the second row. The first row and the second row are alternately arranged on the image sensor,
Said generating means, said data corresponding to the second row in the image data, the second case the value of the second pixel data read out from the row is less than the first predetermined value the The imaging apparatus according to claim 1, wherein the first pixel data read from the two first rows adjacent to the second row is averaged data. The generation unit generates data corresponding to the first row in the image data from a second value in which the value of the first pixel data read from the first row is greater than the first predetermined value. If the value is less than the predetermined value, the first pixel data is used. If the value of the first pixel data read from the first row is not less than the second predetermined value, the first row is used. The imaging apparatus according to claim 4, wherein the second pixel data read from the two second rows adjacent to each other is averaged data. An image sensor in which a large number of pixels are arranged in a matrix, a mechanical shutter having a shutter curtain that travels so as to shield the image sensor, and resetting the charge accumulated in each pixel of the image sensor to start exposure Reset scanning means for performing reset scanning, and control means for controlling the reset scanning means and the mechanical shutter so that the exposure time of the second row is shorter than the first row of the image sensor. A method for controlling an imaging apparatus,
A generation step of generating image data using first pixel data read from the first row and second pixel data read from the second row;
In the generating step, the data corresponding to the second row in the image data is set to a saturation level of the pixel data of the second row, the value of the second pixel data read from the second row. And the first pixel value based on the exposure time of the first and second rows, the second pixel data is read out from the second row when the value is equal to or greater than a first predetermined value. When the value is less than the first predetermined value, the data is based on the first pixel data.   A program for causing a computer to execute the control method according to claim 6.   A computer-readable storage medium storing the program according to claim 7.

Images