JPH05153473A - Processing method for picture - Google Patents

Abstract

PURPOSE:To expand a dynamic range effectively and to avoid deterioration in the picture quality. CONSTITUTION:Plural pictures with a different exposure condition are synthesized and in this synthesis, a primary processing in which a luminance of each picture element in each picture is processed to obtain a primary process luminance signal with a condition set to each picture of each exposure condition, a secondary processing in which primary processing luminance signals of picture elements of each picture corresponding to each light receiving cell are synthesized to obtain a synthesized luminance signal, and a ternary processing in which the synthesized luminance signal is processed to obtain an output luminance signal under the condition that the luminance corresponding to the maximum light quantity is set within the saturation luminance of a reproduction device are implemented. Furthermore, the processing condition preset for the ternary processing is converted to each picture element based on the relation with the synthesized luminance as to picture elements in the vicinity of a relevant picture element.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method.

[0002]

2. Description of the Related Art In image processing in which a solid-state image pickup device typified by a CCD device is used and the image is reproduced by a reproducing device, there is a problem that the dynamic range is narrower than that of a normal photographic system. That is, taking a CCD device as an example, the CCD device is composed of a large number of light receiving cells, and each light receiving cell provides one pixel, but each light receiving cell has a certain light receiving capacity. However, if the amount of light exceeds this, pixels cannot be provided, and that portion becomes halation. On the other hand, the reproduction device also has a limit on the reproducible brightness value, that is, a saturation brightness value. It also becomes halation.

Various techniques have already been proposed for improving such a narrow dynamic range.
As one of them, there is a technique of synthesizing a plurality of images having different exposure amounts obtained from the same subject (for example, Japanese Patent Laid-Open No. Sho 6-96).
3-306779). However, although the conventional image synthesizing technique can expand the dynamic range, it is inevitable that the image quality is deteriorated.

By the way, for example, Japanese Patent Application No. 2-224333.
Japanese Patent No. 3187242 discloses a video system type magnifying observation apparatus which is equipped with an image pickup tool having a built-in light source and is capable of obtaining an image of an observation object while illuminating with the built-in light source. When the observation target, that is, the imaging target is illuminated with a unidirectional light source from a short distance like this magnifying observation device, the problem of halation is particularly large.

However, in the case of such a video system type magnifying observation apparatus, since there is a high demand for high image quality, even if halation is prevented by expanding the dynamic range, it is possible to avoid deterioration of image quality as much as possible. It must be able to do.

[0006]

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image processing method capable of effectively expanding a dynamic range and avoiding deterioration of image quality.

[0007]

According to the image processing method of the present invention, different exposure conditions are used by using a solid-state image sensor in which a large number of light receiving cells for outputting pixel signals including a luminance signal corresponding to the amount of received light are arranged. The output image is formed by taking a plurality of images by synthesizing the plurality of images, and the dynamic range is substantially expanded by synthesizing the images having different exposure amounts.

In order to prevent the deterioration of image quality, processing is performed so as to reduce the disparity between the combined images. Specifically, the primary processing for processing the luminance value of each pixel in each image under the predetermined condition set for each image of each exposure condition to obtain the primary processed luminance signal, and each image corresponding to the same light receiving cell Secondary processing for synthesizing primary processing luminance signals of pixels of to obtain a synthesized luminance signal,
The output brightness signal obtained by performing the tertiary processing to process the combined brightness signal and obtain the output brightness signal under the conditions set so that the brightness value corresponding to the maximum light amount value is within the saturation brightness value of the playback device. Is used to form an output image.

Further, in order to preserve the contrast detail in the overall compressed image for expanding the dynamic range, the preset processing conditions for the tertiary processing are set for each pixel. ,
The relationship with the synthetic luminance value for pixels in the vicinity of the pixel,
For example, the conversion is applied based on a relationship such as a relationship with the average value or weighted average value of the combined luminance values of some neighboring pixels. As a result, the reproducibility of contrast details is improved, and the deterioration of image quality can be prevented more effectively.

In such a method, the primary processing conditions are set for each image under each exposure condition, and also set for each light amount value range obtained by appropriately dividing the received light amount value range. By doing so, the difference between the combined images can be made smaller, and higher image quality can be achieved.

[0012]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The embodiments of the present invention will be described below in connection with the image pickup apparatus. As shown in FIG. 1, the image pickup apparatus includes a solid-state image sensor such as a CCD element and a camera having a processing unit (not shown) for outputting a signal output from the solid-state image sensor as an image signal. The image processing apparatus is provided with a processing device for processing the image signal from the camera or for instructing the camera to set or switch the exposure condition at the time of image capturing, specifically, the shutter speed. The device mainly includes a luminance signal processing unit, a color signal processing unit, and a shutter speed control unit.

The brightness signal processing unit is for processing the brightness signal output from the camera for each pixel of each image (a plurality of images taken at different shutter speeds). For example, n seconds and n / 4 seconds. In this example, which is based on the assumption that images with two types of shutter speeds such as
As a frame-compatible memory, two kinds of memories, a normal shutter memory and a high-speed shutter memory, are provided, and a luminance signal for each pixel of an image at each shutter speed is stored in each memory. ..

The brightness signal for each pixel stored in each memory is combined in a brightness signal combining circuit. Specifically, as shown in FIG. 2, the luminance signal synthesizing circuit has a look-up table set for each image, and the look-up table is used to determine the luminance of each pixel for each image. First, the primary processing for individually processing the values is performed.

FIG. 3 and FIG. 4 show an example of a light quantity-luminance characteristic given with respect to three requirements of a light quantity value to be received, a brightness value obtained corresponding to this light quantity value, and an exposure condition. 3 shows the specific light amount peculiar to each of FIG. 3 and the luminance characteristics, and FIG.
FIG. 4 shows the luminance characteristic, and FIG. 4 shows the characteristic quantity-luminance characteristic in the case of the n / 4 second shutter. The% values shown on the vertical and horizontal axes in the characteristic diagram are 100% of the saturation value at the n-second shutter.
It shows relative relation as% and does not have a specific meaning.

The luminance signal output from the n-second shutter memory has a luminance value corresponding to the characteristic-luminance characteristic of FIG. 3, and the processing by the look-up table corresponding to this corresponds to the primary processing light amount as shown in FIG. -The luminance signal processed into the luminance value corresponding to the luminance characteristic, and the luminance signal output from the n / 4 second shutter memory has the luminance value corresponding to the specific light amount-luminance characteristic of FIG. 4, and the processing by the lookup table corresponding to this As a result, the luminance value corresponding to the primary processing light amount-luminance characteristic as shown in FIG. 6 is processed. That is, in this primary processing, the range of the received light amount value is appropriately set to 0 to 80%, 80 to 100% in this example,
And 100 to 400%, and each look-up table is created for each of the divided light amount value ranges and according to the conditions set for each image at each shutter speed, and both look-up tables are created. It is performed by using.

Then, the primary processing luminance signal obtained by such processing is added in an adder circuit to carry out secondary processing to obtain a combined luminance signal. Specifically, a new combined luminance signal is obtained by adding the primary processed luminance signals to the pixels of each image corresponding to the same light receiving cell. The synthesized luminance signal obtained by this has a luminance value corresponding to the synthesized light amount-luminance characteristic as shown in FIG.

The combined light quantity-luminance characteristic of FIG. 7 is smooth and continuous with only one inflection point at 100%, but this has different conditions for each divided light quantity value range as described above. This is brought about by the process of giving, and greatly contributes to maintaining the image quality.

The combined luminance signal output from the luminance signal combining circuit is further subjected to tertiary processing. This tertiary processing is performed by the brightness value (17) corresponding to the maximum light amount value (400% in this example).
5%) within the saturated luminance value of the reproducing device (100% on the vertical axis in FIG. 9 corresponds to this),
The compression processing is performed under the conditions described below. As shown in FIG. 8, the luminance signal amplitude compression circuit for the tertiary processing includes a neighborhood average calculation circuit and a look-up table array circuit.

The neighborhood average calculation circuit is for calculating the average value of the combined luminance values of the pixels in the vicinity of the pixel to be processed in the tertiary processing, and the look-up table selection signal is calculated based on the average value thus obtained. It is output to the look-up table array circuit. The look-up table array circuit has a plurality of different look-up tables. The look-up table selected by the look-up table selection signal is used to process the composite brightness signal into an output brightness signal.

Each look-up table of the look-up table array circuit is created for each of a plurality of characteristic curves as shown in FIG. A plurality of characteristic curves shown in FIG. 9 (in this example, they are actually straight lines with an inclination of 45 °) are converted under a certain condition based on one characteristic curve, for example, a characteristic curve passing through the origin. Is obtained by moving in parallel, the brightness value corresponding to the light amount decreases as the characteristic curve closer to the origin, and the brightness value corresponding to the light amount increases as the distance from the origin increases. The characteristics are provided so that the light quantity-luminance characteristics can be selected depending on whether it is used. For example, for a pixel having a light quantity of A, a brightness value of a ₁ is given by using a characteristic curve passing through the origin, and a brightness value of a ₂ smaller than a ₁ is given by using a certain characteristic curve on the right side. Therefore, the compression condition can be selected by selectively using the look-up table created for these characteristic curves.

The selection is made by the look-up table selection signal from the neighborhood averaging circuit as described above, and when the average value of the combined luminance values of the neighboring pixels is high,
That is, when the surroundings are bright, the compression is increased, and when the average value of the combined luminance values of neighboring pixels is low, that is, when the surroundings are dark, the compression is reduced. That is, when the periphery of the pixel is bright, the pixel is darkened by the compression of the large luminance value, and when the periphery of the pixel is dark, the pixel can be kept bright by the compression of the small luminance value. Therefore, the dynamic range is expanded by keeping the wide light amount range within the brightness value range below the saturation, and the contrast detail is saved, so that the dynamic range can be expanded while maintaining the image quality. ..

The output luminance signal thus obtained is output to a reproducing device (not shown) through the same processing path as the conventional one.

The chrominance signal processing section includes a normal shutter memory and a high speed shutter memory for separately holding pixels of respective images having different shutter speeds, as in the luminance signal processing section. The color signal for each pixel of the image according to the speed is stored.

As described above, the memory is provided for each image also for the color signal in order to improve the image quality by selectively using the pixels with high saturation from each image.
That is, the saturation is calculated by the saturation calculator for each pixel of each image, and the saturation is compared by the saturation comparator so that the pixel with a large saturation is selected and used by the color signal selector. The color signal of the selected pixel is further corrected by the color signal corrector on the low-luminance side and the high-luminance side in relation to the luminance signal so as to meet the image signal reference, and then output.

The shutter speed control unit controls the setting and switching of the shutter speed of the camera, and controls the storage of the luminance signal and the color signal for each image in the corresponding memory in response to this control. The shutter switching control is performed using the synchronization signal from the camera, and the storage control to the memory is performed via the memory switching device.

[0027]

As described above, according to the image processing method of the present invention, a plurality of images taken under different exposure conditions are combined to obtain an output image, thereby substantially expanding the dynamic range. Since the processing is performed so as to reduce the disparity between the combined images, the effective dynamic range can be expanded without deteriorating the image quality. Therefore, for example, it can greatly contribute to the improvement of the image quality in the magnifying observation apparatus of the video system type.

[Brief description of drawings]

FIG. 1 is a block diagram of an imaging device used in the method of the present invention.

FIG. 2 is a circuit diagram of a luminance signal combining circuit in FIG.

FIG. 3 is a characteristic diagram showing an intrinsic light amount-luminance characteristic in an n-second shutter.

FIG. 4 is a characteristic diagram showing an intrinsic light amount-luminance characteristic in an n / 4 second shutter.

FIG. 5: Primary processing light amount for image of n-second shutter-
The characteristic view which shows a brightness characteristic.

FIG. 6 is a characteristic diagram showing a primary processing light amount-luminance characteristic for an image of an n / 4 second shutter.

FIG. 7 is a characteristic diagram showing a combined light amount-luminance characteristic.

FIG. 8 is a circuit diagram showing a configuration of a luminance signal amplitude compression circuit.

FIG. 9 is a characteristic curve diagram that gives conditions for processing a combined luminance signal into an output luminance signal.

Claims (2)

[Claims] 1. A plurality of images are taken under different exposure conditions by using a solid-state image sensor in which a large number of light-receiving cells that output pixel signals including a luminance signal corresponding to the amount of received light are arranged, and the plurality of images are combined. When an output image is formed by performing the following, primary processing to obtain the primary processing brightness signal by processing the brightness value of each pixel in each image under the predetermined conditions set for each image under each exposure condition, corresponding to the same light receiving cell Secondary processing to combine the primary processing luminance signals of the pixels of each image to obtain a combined luminance signal, and under the condition that the luminance value corresponding to the maximum light amount value is within the saturation luminance value of the playback device. An image processing method in which a composite luminance signal is processed to obtain an output luminance signal, which is subjected to tertiary processing, and an output image is formed using the output luminance signal obtained by the processing. An image processing method, wherein the set processing conditions are converted and applied for each pixel based on a relationship with a combined luminance value of pixels in the vicinity of the pixel. 2. The primary processing condition is set for each image under each exposure condition, and is also set for each light amount value range obtained by appropriately dividing a range of received light amount values. The described image processing method.