JPH05316413A - Image pickup device - Google Patents

Abstract

PURPOSE:To expand the dynamic range of the image pickup of an image by making an image memory minimum by picking up image by continuously dividing the dynamic range of the brightness value of an image to be determined in order of the increase of the brightness value. CONSTITUTION:A loopup table 3 inputs digital signal data to be outputted from an A/D converter 2 and outputs signal data after non-linearity is corrected. A write decision device 4 receives the signal data to be outputted by the lookup table 3 and decides whether the signal data of each picture element is written in an image memory 6 or not. When it decides to write, a write pulse is generated and the data is made to be written in the image memory 6 via a lookup table 5. The lookup table 5 outputs the result which the inputted signal data is multiplied by the magnification based on the exposure value at the time of the image pickup to be fixed by the signal from a control device 7.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup device such as a video camera or an electronic camera having an expanded dynamic range of an image pickup device.

[0002]

2. Description of the Related Art The dynamic range of an image pickup device is generally determined by the ratio between the noise level and the saturation level of an output signal.
When the dynamic range of the brightness of the imaging target is wider than the dynamic range of the output of the image sensor, it is necessary to adjust the exposure amount of the image sensor. In this case, for example, by providing a means for adjusting the exposure amount such as the illumination intensity, the diaphragm amount, the shutter speed, the transmission amount of the filter, and the like, the exposure amount is changed in multiple stages to capture an image. It is possible to obtain an image having a dynamic range wider than the range.

As a conventional technique for realizing this, for example,
There is a method of changing the exposure parameter in three steps to take an image. That is, imaging is performed with the first exposure parameter, the first image obtained by this imaging is stored in the first image memory, then imaging is performed with the second exposure parameter, and the second image obtained by this imaging is performed. Stored in the second image memory,
Next, imaging is performed with the third exposure parameter, and the third image and the first image memory obtained by this imaging, and the first image and the second image stored in the second image memory are combined to obtain a wide dynamic image. Generate a range image.

[0004]

However, in this conventional technique, it is necessary to store images picked up by different exposure parameters.
1) The number of image memories is required. Therefore, there is a problem that the device becomes large. Also, in the above method, 1
One each time the imaging of a set of exposure parameters is completed (in the above example, the imaging with three types of exposure parameters is completed)
Since it is a method that can obtain two wide dynamic range images, it is difficult to obtain a wide dynamic range image of a video rate necessary to reproduce smooth motion when applied to imaging a moving object. There was a point.

[0005]

The structure of the invention for solving the above-mentioned problems comprises an image pickup device, an A / D converter, and an exposure amount control means for controlling the exposure amount of the image pickup device in at least two stages. An image memory for storing the brightness value on the image plane corresponding to the pixel, an image pickup control unit for controlling the exposure amount control unit and the image pickup element to sequentially pick up the object with the exposure amount set stepwise, Each of the divided ranges obtained by continuously dividing the dynamic range of the lightness value in the increasing order of the lightness value is made to correspond to the decreasing order of each exposure amount, and the divided range or the image pickup device corresponding to the divided range is provided for each exposure amount. A range setting means for setting an output range, a conversion means for converting the output value of the A / D converter into a value within a divided range corresponding to the exposure amount at the time of imaging using the exposure amount at the time of imaging as a scale factor, and A / D For each pixel from the D converter The range determination means for determining whether or not the input output value or the conversion value output from the conversion means exists in the output range or the divided range corresponding to the exposure amount at the time of imaging, and the determination result of the range determination means is affirmative. Only in that case, the storage control means for storing the conversion value obtained by the conversion means in the image memory at the address corresponding to the output pixel is provided.

[0006]

As shown in FIG. 3, the dynamic range (extended dynamic range) of the brightness value of the image to be obtained is continuously divided in the increasing order of the brightness value to obtain each divided range.
Hereinafter, the first division range, the second division range, ... In this order. Each divided range corresponds to the increasing order of the brightness value and the decreasing order of the exposure amount. Hereinafter, the exposure amount will be referred to as a first exposure amount, a second exposure amount, ... In descending order. That is, the lightness values of the first divided range, the second divided range, ... Are respectively the first exposure amount,
The second exposure amount is obtained from the captured image.

On the other hand, as shown in FIG. 3, there are output ranges of the image pickup device corresponding to the first divided range, the second divided range, .... Hereinafter, in this order, the first output range, the second
The output range ... The correspondence relationship between the division range and the output range can be set to the following relationship, for example. The first output range is a saturation level (the upper limit of a linear range that can be distinguished from a completely saturated state) above the noise level of the image sensor (the lowest output signal level at which the required S / N is obtained). be able to. A value obtained by converting the value of the first output range from the first exposure amount to the reference exposure amount corresponds to the value of the first divided range. In this way
The first division range is determined. Also, the maximum value of the second output range (generally the saturation level) converted from the second exposure amount to the reference exposure amount corresponds to the maximum value of the second divided range. The second divided range is set so that the minimum value is equal to the maximum value of the first divided range.
The minimum value of the second output range is a value obtained by converting the minimum value of the second divided range from the reference exposure amount to the second exposure amount.

Based on the above relationship, each output range of the image sensor,
Correspondence between each division range and each exposure amount can be set. In particular, as shown in FIG. 8, in order to make each output range common to the entire range from the noise level to the saturation level,
It is also possible to set each exposure amount and set each divided range from the output range common to each exposure amount. That is,
The minimum value of the second divided range, the third divided range, ... Is converted from the reference exposure amount into the second exposure amount, the third exposure amount ,. Three
The exposure amount, ... Can be determined, and each division range can be determined.

By the image pickup control means, for example, the first exposure amount,
The second exposure amount, ... The nth exposure amount, the first exposure amount ,.
Each exposure amount is sequentially and periodically changed, and an image is captured corresponding to each exposure amount. Note that the imaging order need not be periodic.
The output value sequentially output from the A / D converter for each pixel is converted from the exposure amount during imaging to the reference exposure amount. This converted value is hereinafter referred to as a converted value.

When the image is captured with the first exposure amount, it is determined whether or not the output value or the converted value sequentially output from the A / D converter for each pixel exists in the first output range or the first divided range. It Similarly, when an image is taken with the second exposure amount, A /
It is determined whether the output value or the conversion value sequentially output from the D converter for each pixel exists in the second output range or the second divided range. The conversion value is stored in the image memory at the address corresponding to the output pixel only when the determination result is affirmative. That is, when the image is captured with the first exposure amount, only the brightness value of the pixel having the brightness value of the first divided range is rewritten in the image memory. Similarly, when the image is captured with the second exposure amount, only the brightness value of the pixel having the brightness value of the second divided range is rewritten in the image memory.

In this manner, each time the first exposure amount, the second exposure amount, ... The nth exposure amount is sequentially imaged, the first division range, the second division range ,. Only the lightness value of the pixel having the lightness value of is rewritten,
The above process is repeatedly executed. An image with an expanded dynamic range can be obtained by referring to the value of this image memory at an arbitrary time.

[0012]

As described above, according to the present invention, by providing one image memory for the image pickup screen, it is possible to obtain a grayscale image having an expanded dynamic range at an arbitrary time. Therefore, the device can be downsized. Further, since the value of the image memory is rewritten in time series for each divided range, even if the value of the image memory is referred to at an arbitrary time, it is possible to obtain the information of the moving image that changes smoothly.

[0013]

EXAMPLES The present invention will be described below based on specific examples. In FIG. 1, an image pickup device 1 is an image pickup device capable of controlling a charge accumulation time generated by a photoelectric conversion mechanism by an electric signal, and specifically, a CCD image pickup device with an electronic shutter function capable of obtaining a two-dimensional image. Is. Therefore, in this embodiment, the exposure parameter (exposure amount) is given by the accumulation time of the electronic shutter function. Therefore, in this embodiment, the exposure amount control means and the image pickup device are integrally incorporated as a CCD image pickup device. In the case of a camera, an image is formed on the light receiving surface of the image sensor by an optical system such as a lens (not shown).

The A / D converter 2 samples the signal level of the analog signal read from the image pickup device 1 in pixel units and converts it into digital signal data. The look-up table 3 stores in advance a correction coefficient for correcting the non-linearity of the image sensor 1 by the optical system,
The digital signal data output from the A / D converter 2 is input, and the digital signal data (digital value) whose nonlinearity is corrected is output.

The write determiner 4 which constitutes the range setting means, the range determining means, and the storage control means receives the digital signal data output from the look-up table 3, and stores the digital signal data of each pixel in the image memory. 6 is a device for determining whether or not to write data in No. 6 by a method described later. When the write determiner 4 determines that the data should be written, the data is stored in the lookup table 5.
It is a device that generates a write pulse to the image memory 6 at the time when it is converted by the above and is given to the data input terminal of the image memory 6 so that the data is written in the image memory 6.

The look-up table 5 constituting the converting means is configured to output a result obtained by multiplying the input digital signal data by a magnification based on the exposure amount at the time of image pickup which is determined by a signal from the control device 7. .. The image memory 6 is a bitmap format image memory that holds a wide dynamic range image generated by this apparatus. The control device 7 which constitutes the image pickup control means mainly controls each exposure time periodically, and includes the image pickup device 1,
Controls the write determiner 4, the lookup table 5, etc.,
This is a device for generating a wide dynamic range image in the image memory 6.

This device operates as follows under the control of the control device 7. FIG. 6 shows a timing chart when the number of image pickup pixels is 4 × 3 pixels. First, the image sensor 1
Performs imaging in the first accumulation time T1 that determines the first exposure amount according to a signal that controls the accumulation time length and the accumulation timing output from the control device 7. The analog image signal data obtained by this image pickup is sequentially read from the image pickup device 1 on a pixel-by-pixel basis in time series by a control signal from the control device 7, and the following processing is performed on the series of pixel data.

The analog image signal data (video signal) from the image pickup device 1 is sequentially transferred to the A / D converter 2, and in accordance with the sampling signal output from the control device 7, the A / D converter 2 outputs a digital signal pixel by pixel. Data D '(x, y)
Is converted to. Here, (x, y) indicates the coordinate position of this pixel on the image plane. The digital signal data D ′ (x, y) after A / D conversion output from the A / D converter 2 is
The data is transferred to the look-up table 3 and converted into data D (x, y) in which the non-linearity of the image sensor 1 is corrected. The non-linearity-corrected data D (x, y) is transferred from the look-up table 3 to the write determiner 4. Further, the write determination unit 4 is informed by the storage time length signal from the control device 7 that the data transferred to the write determination unit 4 is the data of the storage time T1.

The write determination unit 4 is a lookup table 3
When the data D (x, y) is received from the controller 7, R
It is determined whether or not the condition of min (T1) ≦ D (x, y) ≦ Rmax (T1) is satisfied. Here, as shown in FIG. 2, Rmin (T1) and Rmax (T1) are values determined corresponding to the captured image of the accumulation time T1. That is, it is the minimum value and the maximum value of the first output range of the image sensor 1 corresponding to the first exposure amount. When the write determiner 4 determines that the above condition is satisfied, as described later,
The data of D (x, y) is converted by the lookup table 5. Then, at the timing when the converted data (converted value) is given to the data input terminal of the image memory 6, the write determiner 4 issues a write pulse to the image memory 6.

The data D (x,
y) is transferred to the lookup table 5. The look-up table 5 is informed by the storage time length signal from the control device 7 that the data transferred to the look-up table 5 is the data of the storage time T1. When the data of D (x, y) is input, the lookup table 5 is A (T1) × D (x, y) multiplied by A (T1).
(Data converted into the first divided range converted into the reference exposure amount) is output to the data input terminal of the image memory 6. Here, A (T1) is a value determined corresponding to the accumulation time T1, and is a scale factor for converting the output value of the image sensor 1 into the value in the reference exposure state (the value of the first division range). Is. In addition, A (T
1) xD (x, y) data is applied to the data input terminal of the image memory 6 in synchronization with the memory address corresponding to the pixel (x, y) of the image memory 6 from the control device 7. 6 memory address terminals. A
At the time when the data of (T1) × D (x, y) is applied to the data input terminal of the image memory 6, if the write pulse is generated from the write determiner 4, this data is stored in the image memory 6. Is written to the address corresponding to the pixel position (x, y) of.

When the above-described processing is completed for the series of pixel data imaged at the accumulation time T1 (first exposure amount), the accumulation time length and accumulation timing of the next control cycle from the control device 7 are controlled next. In accordance with the signal, the image sensor 1 captures an image for the second storage time T2 (second exposure amount).
The image obtained by this image pickup is sequentially read from the image pickup element 1 for each pixel in time series similarly to the case of the image pickup by the first accumulation time, and the accumulation time T is calculated for these series of pixel data.
The same processing as in the case of 1 is performed. However, the accumulation time T
In the case of 2, the write determination unit 4 outputs Rmin by the accumulation time length signal output from the control unit 7 to the write determination unit 4.
(T1) and Rmax (T1) are Rmin (T2) and Rmax
(T2) and the accumulation time length signal output from the control device 7 to the lookup table 5,
The scale factor in the lookup table 5 is A (T
It is different from the case of the accumulation time T1 that 1) becomes A (T2). Note that Rmin (T2) and Rmax (T2) are the minimum value and the maximum value of the second output range. Further, A (T2) is a value determined corresponding to the accumulation time T2, and the image sensor 1
The output value of is the value in the standard exposure state (the value of the second divided range)
Is a scale factor for converting to.

By performing the above operation, a wide dynamic range image can be generated in the image memory 6. This principle will be described below together with an example of how to determine each constant described above. Now, for example, in the state after the nonlinearity is corrected by the look-up table 3 in the output signal of the image sensor, the zero level (the output in the state where the incident light intensity is zero when noise is ignored) is 0. , And the saturation level is Dmax. The linearity is maintained between 0 and Dmax. Further, it is assumed that the relationship between the accumulation times T1 and T2 is T1> T2. At this time, each constant

[0023]

## EQU1 ## Rmin (T1) = 0, Rmax (T1) = Dmax-1, Rmin (T2) = Dmax × T2 / T1, Rmax (T2) = Dmax, A (T1) = 1, A (T2) = T1 / T2

As shown in FIG. 2, for the pixel (dark portion) having the lightness value in the first divided range, the image picked up at the accumulation time T1 (first exposure amount) is used as shown in FIG.
An image captured during the accumulation time T2 (second exposure amount) is used for a pixel (bright portion) having a brightness value in the divided range, and as a result, a wide dynamic range image is obtained. That is, for each pixel (x, y), the accumulation time T
When the data D1 captured at 1 does not reach the saturation level (when present in the first output range), this D1 is used as the image data of the pixel (since the first exposure amount is the reference exposure amount, D1 is the value of the first division range), and when D1 is saturated, the data D2 when imaged at the accumulation time T2 is multiplied by T1 / T2 (the output value of the image sensor 1 is the reference exposure amount). The value of the second divided range is obtained by converting into the value of 1). Here, the reason for multiplying D2 by T1 / T2 is that the signal output of the image sensor when the amount of incident light is the same is proportional to the accumulation time in the non-saturated state, and therefore the data of D2 has a sensitivity T2 / T1 higher than that of D1. Because it is doubled.

In the above equation, the value Dmax -1 of Rmax (T1) means a value as large as possible within the range where the image pickup device has not reached the saturation level. Practically, there is a possibility that some pixels may not be filled in by either image of the accumulation time T1 or T2 due to noise of the image sensor or the movement of the object to be imaged, so Rmin (T2) is the value shown above. It is preferable to have a slightly overlapping area with a value slightly smaller than the above. The method of deciding the values of these constants shown here shows the fundamental idea in the present invention, and various modifications which can be easily considered can be added to this.

A concrete example of the numerical values of the respective constants is as follows: T1 =
When 1/60 seconds, T2 = 1/600 seconds, and Dmax = 255, as an example,

[0027]

## EQU00002 ## Rmin (T1) = 0, Rmax (T1) = 254, Rmin (T2) = 25, Rmax (T2) = 255, A (T1) = 1, A (T2) = 10

It is possible that At this time, the data D (x, y) captured at the accumulation time T1 (1/60 seconds)
Is 0 to 254, a write pulse is generated to write this data in the image memory 6, and the data D (x, y) is 2
In the case of 55, no write pulse is generated. When the data D (x, y) imaged at the accumulation time T2 (1/600 seconds) is 0 to 24, no write pulse is generated, and when the data D (x, y) is 25 to 255, no write pulse is generated. Lever data 1
A value multiplied by 0 will be written in the image memory. Therefore, in the image memory, image data having a lightness value in a wide dynamic range of 0 to 2550 can be obtained.

In the above description, an operation for obtaining a wide dynamic range image by performing a series of operations for the storage time T1 and then a series of operations for the storage time T2 has been described, but the present apparatus operates as follows. By doing so, it is possible to capture a moving object and obtain a moving image.

That is, as shown in FIG. 7, a series of operations for the storage time T1 is performed, then a series of operations for the storage time T2 is performed, then a series of operations for the storage time T1 is performed again, and then again. The operation is repeated such that a series of operations for the accumulation time T2 is performed. Then, when a moving object or the like is targeted for imaging, a wide dynamic range image corresponding to it is generated in the image memory 6 as a moving image that is updated at time intervals at which imaging is performed. Therefore, the image memory 6
A video signal having a wide dynamic range can be obtained by reading the contents of the above as a video signal at a video rate in parallel with the writing operation. Further, by stopping the repetitive operation at that time in synchronization with a certain trigger signal, the rewriting of the contents of the image memory 6 is stopped while the contents of the image memory 6 at that time are held. Therefore, it is possible to obtain the image at that moment as a still image.

In the above description, the case of T1> T2 has been described as an example, but in the case of T1 <T2,
Each of the above constants,

## EQU00003 ## Rmin (T1) = Dmax.times.T1 / T2, Rmax (T1) = Dmax, Rmin (T2) = 0, Rmax (T2) = Dmax-1, A (T1) = T2 / T1, A (T2 ) = 1

It is clear that the following is good. Further, in the above description, the accumulation time is selected to be T1 and T2, but by increasing the accumulation time to three or more,
Obviously, a wider dynamic range image can be obtained. For example, the accumulation times are different from each other by T
When three types of T1, T2, T3 (third exposure amount) are selected, for example, the relation of the accumulation times T1, T2, T3 is T1>
When T2> T3, as shown in FIG.

[0033]

## EQU00004 ## Rmin (T1) = 0, Rmax (T1) = Dmax-1, Rmin (T2) = Dmax * T2 / T1, Rmax (T2) = Dmax-1, Rmin (T3) = Dmax * T3 / T2 , Rmax (T3) = Dmax, A (T1) = 1, A (T2) = T1 / T2 A (T3) = T1 / T3. Similarly, it is easy to further increase the types of accumulation time and change the order. A series of accumulation times T1, T2, ..., Tn
Do not necessarily have to be different.

Further, as shown in FIG. 8, each output range can be shared between the noise level (Dmin) and the saturation level (Dmax). In this case, the accumulation time (T1
>T2> T3) is set as follows.

[0035]

## EQU00005 ## T2 = Dmin.times.T1 / Dmax T3 = Dmin.times.T2 / Dmax

As a modification of this embodiment, the accumulation time T
It is also possible to shorten the imaging interval by performing the imaging of the next accumulation time in parallel while processing the data of i.

Further, as another modification of the present embodiment, instead of controlling the storage time of the exposure parameter by the electronic shutter of the image pickup device, the light intensity is controlled by the amount of transmission of the diaphragm or the filter, and the image pickup device by other methods. It is also possible to use the sensitivity control of, the illumination light intensity control by strobe, and the like. For example, when using a filter whose light transmittance is electronically controllable in place of the electronic shutter of the above-mentioned embodiment,
Since the signal output of the image pickup device when the amount of incident light on the filter is the same is proportional to the transmittance of the filter, when the transmittance of the filter is changed to U1 and U2, and when U1> U2, the above constants are set to ,

[0038]

## EQU00006 ## Rmin (U1) = 0, Rmax (U1) = Dmax-1, Rmin (U2) = Dmax xU2 / U1, Rmax (U2) = Dmax, A (U1) = 1, A (U2) = It can be defined as U1 / U2.

As another modification of this embodiment, an image intensifier can be attached to the front part of the image pickup device. Mounting the image intensifier has the advantage of increasing the sensitivity of the imager. When an image intensifier is attached, the control of the accumulation time by the electronic shutter of the image sensor can be used as the control method of the exposure parameter, but in addition, the sensitivity control of the image intensifier is used as the control method of the exposure parameter. It is also possible.

Further, as another modification of the present embodiment, not only the multiplication of the constant A (Ti) in the lookup table 5 but also the multiplication of the constant A (Ti) and conversion of the value by some function It is also possible to relatively compress the bright portion so that the image memory 6 can be effectively used, or to make the contents of the image memory 6 into a preferable format for image processing. As a conventional technique for compressing the density level of image data, there is a method of logarithmically converting the density level.

However, the conventional method of logarithmically converting the density level and storing it in the image memory has the following problems. When the density level is logarithmically converted and stored in the memory, the value stored in the memory is quantized, and the density level corresponding thereto is also quantized. The graph of the logarithmic function log a is a
The closer to 0, the larger the gradient. Therefore, when the density level is logarithmically converted and stored in the memory, the density error becomes smaller as the density level becomes smaller. on the other hand,
It is considered that the density level itself contains a certain error due to causes such as a noise component of the image output signal of the image pickup device and a conversion error of the A / D converter.

Generally, in a region where the density level is high,
There is no problem because the density level quantization error is larger than the density level error itself, but in the low density level area, the density level quantization error is much smaller than the density level itself error. However, there arises a problem that the bits of the memory are not effectively used.

As a method for solving this problem, it is effective to use Log (Li (x, y) + Lo) as a function. Where Log is the natural logarithmic function, Li (x,
y) is the data D given to the lookup table 5
A value obtained by multiplying (x, y) by a constant A (Ti) is expressed as Li
(X, y) = A (Ti) × D (x, y). Also,
Lo is a positive constant, and for example, a series of accumulation times T1>
When T2 >> ... >> Tn, Lo = A (T1) * Dmax * T2
It can be / T1. The function Log (a + Lo) is
In the region where a is large, the graph shape is almost the same as Log a, but the slope of the graph becomes constant in the part where a is close to 0.
It does not become as large as the slope of the Log a graph. Therefore, it is possible to prevent the quantization error of the density level from becoming smaller than the error of the density level itself in the region where the density level is small. Therefore, by using this method, the above problems can be solved, and an image with a wide dynamic range can be stored in the image memory while saving bits in the image memory. If the common logarithmic function is Log ₁₀ , Log a =
Since Log ₁₀ a / Log ₁₀ e, Log (Li (x, y)
Log ₁₀ (Li (x, y) + Lo) / instead of + Lo)
Log ₁₀ e can be used. However, e is the base of natural logarithm.

As another modification of this embodiment, an A / D
It is conceivable that the converter 2 is configured to logarithmically perform A / D conversion. For example, logarithmic conversion or the above Log
When the image is stored in the image memory after the conversion of (Li (x, y) + Lo), the quantization error of the density level in the image memory becomes large in the area of high density level as described above. The conversion accuracy in the converter may be lower in the higher density level than in the lower density level. Therefore,
By performing logarithmic A / D conversion in the A / D converter, the number of bits of the A / D converter can be efficiently used, and by using the A / D converter of the same number of bits, There is an effect that the accuracy of the image obtained by the imaging device can be improved.

In this case, when the write decision unit 4 receives the data of Log (D (x, y)) from the lookup table 3, Log (Rmin (Ti)) ≤Log (D (x,
y)) ≦ Log (Rmax (Ti)) is determined. Also, look-up table 5
When the data of Log (D (x, y)) is input, is multiplied by A (Ti) and Log (A (Ti) × D (x,
y)) data, that is, Log ((D (x, y)) to Log
The data obtained by adding (A (Ti)) is output to the data input terminal of the image memory 6. In addition, the above Log (A (T
i) × D (x, y) + Lo) and store the image in the image memory, Log (A (Ti) × D (x, y)) to Log (A
Conversion to (Ti) × D (x, y) + Lo) is also performed at the same time. As can be easily understood from the above description, instead of performing the A / D conversion logarithmically, the A / D conversion accuracy of the lower density level becomes higher than that of the higher density level by using another function characteristic. It is also possible to perform / D conversion, and it is possible to easily consider the configurations of the write determination unit 4 and the look-up table 5 in accordance with the conversion.

As another modification of this embodiment, the look-up table 3, the write decision unit 4, and the look-up table 5 may be collectively replaced by one look-up table using a known technique. It is possible.

As another modification of the present embodiment, the contents of the image memory 6 can be fed back to the writing decision unit 4 as shown in FIG. As a result, when the write determination unit 4 determines that the data should be written instead of controlling the data writing to the image memory 6 depending on the presence or absence of the write pulse, the data is looked up in the look-up table 5.
It is also possible to output the data in the previous state of the image memory if it is not.

As another modification of this embodiment, as shown in FIG. 4, some bits of the image memory 6 are used to hold flags, and these bits are used by the write decision unit 4.
It is also possible to give feedback to. This allows
By controlling the flag, it is possible to prevent generation of pixels that are not filled with any of the above-mentioned images.

A case where an auto iris lens is further attached to the above-described embodiment will be described. Conventionally, the control of the auto iris lens has been performed by the output signal of the image sensor 1. That is, when the output signal of the image pickup device 1 is large, the intensity of the incident light is large, so that the diaphragm of the auto iris lens is narrowed down, and when the output signal of the image pickup device 1 is small, the intensity of the incident light is small, so the auto iris is small. The auto iris lens was feedback-controlled so as to open the lens diaphragm.

However, in this embodiment, since the sensitivity of the image pickup device 1 is sequentially changed, the output signal of the image pickup device 1 does not always correspond to the intensity of incident light in the same manner. Therefore,
In contrast to this embodiment, the conventional method has a problem that it is difficult to control the auto iris lens. On the other hand, in this embodiment, the video signal from the image memory 6 is a signal corresponding to the intensity of incident light. Therefore, by controlling the auto iris lens by the video signal from the image memory 6 instead of the output signal of the image sensor 1,
The above-mentioned problems can be solved. By attaching the auto iris lens to the present embodiment and performing the control as described above, it becomes possible to automatically adjust the light intensity range that can be imaged by the present imaging device, so that it is possible to image with a wider light intensity. effective.

As another modification of the present embodiment, as shown in FIG. 9, the relationship between each output range and the divided ranges is set in two ways so as to complement each other in the joints of the divided ranges. It is possible to use the divided ranges alternately. For example, in FIG. 9, the exposure times are T1A and T2.
A, T3A, T1B, T2B, T3B, T1A, T2
A, T3A, T1B, T2B, T3B, ... are set in this order, and imaging is performed sequentially. Correspondingly, the division ranges D1A, D2A, D3A, D1B, D2B, D3 are correspondingly respectively.
B, D1A, D2A, D3A, D1B, D2B, D3
The image data of B, ... Is rewritten. In this method, since the seams of the divided ranges are alternately changed, noise of the image pickup element or the like, movement of the object to be imaged, etc., even if the overlapping area of the divided ranges is reduced (or even if the overlapping area is not taken). Therefore, there is an effect that it is possible to prevent the problem that some pixels are not filled in any of the division ranges.

In the above embodiment, the case of the image signal of one channel (monochrome image signal) is shown. However, as another modification of this embodiment, an image pickup device capable of picking up a color image is used as an image pickup device, for example, as shown in FIG. As shown in FIG. 5, by applying the present invention to the image signals of the three primary colors from the color image pickup device 1RGB, it is possible to obtain a color wide dynamic range image.

In this embodiment, an example in which an analog signal is digitized by an A / D converter and processed by a digital signal is shown. However, as a modification of this embodiment, a known technique is used to perform one of these processes. It is also possible to adopt a configuration in which some or all of them are replaced with analog signal processing.

In the above embodiments, the case where a wide dynamic range image is obtained by using the image pickup device has been shown, but more generally, the sensitivity can be adjusted by the sensitivity parameter instead of the image pickup device. The sensor can be used to extend the dynamic range of the sensor. In this case, the storage time of the image sensor generally corresponds to the sensitivity of the sensor, and the image memory generally corresponds to the memory in which the sensing information is stored.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image pickup apparatus according to a specific first embodiment of the present invention.

FIG. 2 is an explanatory diagram showing a relationship between each output range and a division range.

FIG. 3 is an explanatory diagram showing a relationship between each output range and a division range.

FIG. 4 is a block diagram showing a configuration of an image pickup apparatus according to another embodiment of the present invention.

FIG. 5 is a block diagram showing a configuration of an image pickup apparatus according to another embodiment of the present invention.

FIG. 6 is a timing chart showing the operation timing of each unit in the first embodiment.

FIG. 7 is a timing chart showing the operation timing of each unit in the first embodiment.

FIG. 8 is an explanatory diagram showing a relationship between each output range and a division range in another embodiment.

FIG. 9 is an explanatory diagram showing a relationship between each output range and a division range in another embodiment.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Imaging element (imaging element, exposure amount control means) 2 ... A / D converter 4 ... Write determination device (range determination means, range setting means, storage control means) 5 ... Look-up table (conversion means) 6 ... Image memory 7 ... Control device (imaging control means)

Claims (1)

[Claims] 1. An image pickup device for picking up an image of an object, an A / D converter for sampling a video signal output from the image pickup device for each pixel and converting it into a digital value, and an exposure amount of the image pickup device in at least two stages. Exposure amount control means, an image memory for storing lightness values on the image plane corresponding to pixels, and the exposure amount control means and the image sensor to control the object in stages. The exposure control means for sequentially capturing images with different exposure amounts, and the respective exposure ranges by sequentially dividing the dynamic range of the lightness value in the increasing order of the lightness value and the divided ranges obtained by decreasing the exposure amount. Range setting means for setting the divided range or the output range of the image sensor corresponding to the divided range for each amount; and the output value of the A / D converter for the image exposure using the image exposure amount as a scale factor. A conversion unit for converting the value into a value within a divided range corresponding to the amount, and an output value output from the A / D converter for each pixel or a conversion value output from the conversion unit corresponds to the exposure amount during imaging. Range determination means for determining whether or not the output range or divided range exists, and only when the determination result of the range determination means is affirmative, the conversion value obtained by the conversion means is stored in the image memory. And an image pickup device having a storage control unit that stores the converted value at an address corresponding to the output pixel.