JP3372209B2 - Imaging device - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an improvement in an image pickup apparatus, and more particularly, to electrically correct a flare ghost generated in an image pickup optical system.

[0002]

2. Description of the Related Art In an image pickup optical system, when there is a bright portion in a subject, the black level of the entire screen is uniformly pushed up, that is, a bright screen is displayed at the same time, and at the same time, the spread of light is seen around the bright portion. There is a phenomenon that comes, and I read it as flare ghost. According to Toshiro Kishikawa, "Introduction to Optics" published by Optronics, the phenomenon of forming images is called ghost, and the phenomenon of not forming images is called flare.

The ghost moves depending on the positional relationship between the optical lens and the subject, whereas the flare does not depend on the positional relationship. Here, we will deal with flare. Although flare is always caused by optical characteristics, its value is usually not so small as to be felt. However, it becomes noticeable especially when there is a bright part in the subject.
Since the correction becomes relatively easy with respect to the phenomenon of uniformly pushing up the black level, many countermeasures have been considered than in the past.

In video cameras, other than low-priced personal computer cameras, combined lenses are often used in the image pickup optical system in addition to the single lens. At this time, due to the influence of diffraction of the lenses themselves, the combination of lenses, the lens and the lens barrel, the lens and the image pickup element, the surface wiring of the image pickup element, or the like, incident light is repeatedly reflected, which causes flare on the screen. Due to the flare, light is fogged around the bright part, and it is difficult to see the dark part.

As disclosed in Japanese Patent Application Laid-Open No. 2-76475, "Flare Correction Device", a method has been proposed in which a filter having an inverse characteristic of an input optical system acts on photoelectric conversion output. In principle, this method can correct the flare generated in the optical system.

However, this processing requires a huge amount of calculation including convolution integration. Therefore, if processing is performed in real time, a large-capacity memory and an ultra-high-speed arithmetic unit are required, power consumption is large, large and expensive, and there is a problem that it cannot be easily applied to consumer equipment.

[0007]

As described above, if a filter having an inverse characteristic of the input optical system is made to act on the photoelectric conversion output, it becomes a huge and expensive filter, which is not practical. Therefore, the present invention electrically corrects flare that is optically generated by particularly strong light in a subject by a simple method.
An object of the present invention is to provide an imaging device that reduces the number of images and makes the screen easier to see.

[0008]

In order to achieve the above object, the present invention stores in advance the flare generated by a light source having a predetermined brightness according to at least a predetermined aperture value and a focal length parameter of an optical system. An image signal having a value larger than a predetermined value is detected, its brightness is estimated, and a flare generated when a light source having the estimated brightness is present is artificially generated to generate an image signal. Is subtracted from.

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT An embodiment of the present invention will be described below with reference to the drawings. In the present invention, first, an image pattern or an image pattern model of flare caused by particularly strong light in a subject is stored in advance under a predetermined optical condition. Next, the brightness and shape of the strong light portion in the subject is estimated from the video signal, and a predicted flare is artificially generated from the video pattern or video pattern model stored in advance, and the actual flare is generated. By subtracting the predicted flare generated in a pseudo manner from the video signal, the flare on the screen is corrected and reduced to such an extent that there is no practical problem.

An experiment was conducted and the following results were obtained. 1. From F16 to F1.2, there is almost no change in the flare shape due to changes in the aperture. 2. At F1.4, flare increases rapidly. This depends on the lens structure. 3. The flare shape of the flare spreads depending on the brightness of the subject and does not depend on the size of the subject. The result is schematically shown in FIG.

FIGS. 3 (A1) and 3 (A2) show comparisons of the shape and area of the light source when the brightness is changed, and FIGS. 3 (B1) and 3 (B2) are shown. , And shows the state when the brightness of the light source is constant and the area changes.

FIG. 1 shows an embodiment of the present invention.
Further, FIG. 2 shows the principle of flare correction in which pseudo flare is subtracted from a flared image to correct the flare from the viewpoint of the signal waveform.

In FIG. 1, the light from the subject 101 is
Image pickup element 104 through lens 102 and diaphragm device 103
An image is formed on the photoelectric conversion surface of. Output 105 of image sensor 104
Is input to the diaphragm drive circuit 108 via the amplifier 106, and is also input to the signal processing circuit 109. The diaphragm drive circuit 108 functions to control the diaphragm device 103 according to the output of the amplifier 106 and obtain an amplifier output of an appropriate level. That is, an appropriate exposure state is obtained.
The output video signal 110 of the signal processing circuit 109 is the subtractor 1
16, input to the state setting circuit 114.

In the flare data memory 111, reference flare generated by a predetermined light source is stored in advance in the form of data together with set values (aperture value and focal length) of the optical system. The state setting circuit 114 detects the shape of the strong light portion and the light intensity from the image portion in which flare exists by using the video signal. Then, based on the setting values of the optical system that are stored in advance and the shape and light intensity of the detected strong light portion, the size of the portion and the light source (center of flare generation) that are considered flare, etc. , And generate an estimated prediction flare 115. The predicted flare 115 is input to the subtractor 116 and is used to cancel the actual flare included in the video signal.

Reference numeral 107 denotes an image pickup element drive circuit, which generates timing pulses for driving the image pickup element 104, and outputs timing and drive pulses for the state setting circuit 114 and other circuits.

FIG. 2 shows the principle of flare correction from the viewpoint of the signal waveform. FIG. 2A shows a video signal including flare for one horizontal period. The brightness level becomes extremely high at the position corresponding to the light source, and there is a spread around it. FIG. 2B shows the predicted flare.
In the predicted flare, the state setting circuit 114 detects the shape of the strong light portion and a value corresponding to the light intensity from the video signal 110, and the detection result, the condition stored in advance, and the stored flare pattern are detected. It is created by reference. 2C shows the video signal of FIG.
It is a signal obtained as a result of the calculation processing with the predicted flare of (B), and is a video signal in which actual flare is reduced.

Next, the operation of fetching the storage flare and related data will be described. FIG. 4 shows an example of an apparatus for capturing and constructing flare data. Illuminator 41
The light from the light is diffused by the light diffusing plate 42, and the opening 4 is partially formed.
The light-shielding plate 44 having the number 3 is irradiated. The video camera 45 is arranged at a predetermined distance from the light blocking plate 44 side and images the light blocking plate 44 as a subject. At this time, set the optical condition of the imaging lens to a predetermined focal length and a predetermined aperture value,
Image the subject. The brightness of the illuminator 41 is appropriately adjusted so that flare is sufficiently generated. For example, the brightness is about 100 times the saturation level of the image sensor (CCD). Since the light source portion (the opening 43 portion) is sufficiently bright, in the image pickup element of the video camera 45, the portion corresponding to the light source portion is saturated. In the image captured by the monitor 46, a flare image 48 generated around the image 47 of the light source appears together with the image 47 of the light source. At this time, if the shape of the light source unit is not made small to some extent, the flare image 48 will protrude from the screen. Generates a flare image that fits within the screen. As shown in the waveform monitor 49, the signal corresponding to the light source unit 43 has reached the saturation level.

Next, a method of generating a reference storage or reference flare will be described. Here, a method of measuring and generating flare data will be described using the degree of saturation of the output signal of the image sensor as a scale.

FIG. 5 shows a signal waveform showing how flare data is generated. In the configuration of FIG. 4 described above, when the image is taken at a certain aperture value, the signal waveform diagram of FIG.
As shown in the monitor screen of (A2), the image sensor is in a saturated state. In order to find out how many times the saturation of the image sensor is saturated, the diaphragm device is narrowed down, and FIG.
The diaphragm is set at a point where the image sensor is not saturated, as shown in the signal waveform diagram of FIG. The extinction rate can be calculated from the signal level at this point and the value of the changed aperture. In this way, an image of the light source can be obtained with the flare being inconspicuous. From this image, the shape of the light source in the image signal can be accurately known.

Next, the acquisition of flare data will be described. When a video signal with flare as shown in the signal waveform diagram of FIG. 5 (A1) and the monitor screen of FIG. 5 (A2) is sliced at a predetermined value below the saturation level of the image sensor, FIG.
A waveform as shown in can be obtained. For the slice level, select a value that will not cut the flare peaks, but will preserve the flare shape and preserve it sufficiently. The value of the slice level that determines the flare portion with respect to the saturation signal output level of the image sensor is stored in the flare data memory.

The light source intensity can be calculated from the ratio of this value to the saturation signal and the extinction ratio. As the shape of the light source,
By slicing the signal of FIG. 5B1 at an appropriate level, the shape of a light source having a circular cross section with a constant cusp as shown in FIG. 5D can be obtained.

Next, as shown in FIG. 6, the flare 601 generated from the point light source can be obtained by making the diameter of the light source zero while maintaining the skirt shape of the flare 601. At the same time, the waveform is also obtained. Obtainable. The image from this light source is the PSF (POINT SPREAD) in the optical system.
FUNCTION), and flare caused by a light source having an arbitrary shape can be expressed by a linear combination of PSFs.

A flare 602 having a gradually decreasing spread from a point light source of height 1 in FIG. 6 defined by a slice level.
Is the reference flare. The reference flare created in a pseudo manner is stored in the flare data memory together with the light intensity value.

As a result, flare data and related data can be fetched. In the case of a general light source, the shape and intensity distribution are not simple, but similarly to the above, the shape and intensity of the light source can be obtained by narrowing down the mechanical aperture of the optical system. The intensity of the light source can be obtained from the ratio of the aperture value at the point where the image pickup element is no longer saturated due to the aperture stop and the aperture value at the time of normal image pickup.

Since the processing should be simple, it is sufficient to provide a model in which the intensity of the light source is constant. For example, the light source is represented by a model having a certain height, assuming that the intensity of 100 times or more of the saturation is constant intensity.

Regarding the height of the tube, that is, the intensity of light, the accuracy is high if a sufficiently large amount of light, for example, the average value of a portion of 100 times or more light is used, but a point may be used for simplicity. By filling the outside of the contour of the shape of the light source thus obtained with the PSF without any gap, it is possible to obtain the predicted flare by the light source of any shape.

FIG. 7 shows a specific example of the configuration of the state setting circuit 114 for generating the predicted flare based on the above principle. In FIG. 7, a video signal including a certain flare is input to the shape determination circuit 701. Here, the shape of an arbitrary light source is determined. A mechanical aperture of the optical system is narrowed down to obtain the shape and intensity of the light source, and an unsaturated signal 706 is obtained. The intensity of the light source can be obtained from the ratio of the aperture value at the point where the image pickup element is no longer saturated due to the aperture stop and the aperture value during normal imaging. This is obtained by the light source intensity determination circuit 703. Further, the shape of the light source can be obtained from the signal that is not saturated. Along with this shape, the reference flare obtained from the flare data memory 111 is used to perform PSF flare combining in the PSF flare combining circuit 702 to obtain a predicted flare.

FIG. 8 shows how predicted flare signals are generated by PSF synthesis. That is, the shape 801 (FIG. 8A) of the light source having an arbitrary shape is obtained. Now, it is assumed that this shape is the shape shown in the cross section 802 (FIG. 8B). Then, the reference flare 803 (FIG. 8C) is combined to obtain the predicted flare 804 (FIG. 8D). The cross-sectional waveform of this predicted flare has a shape as shown by a waveform 805 (FIG. 8 (E)). The inside of the cross section of the shape of the light source having an arbitrary shape has the normalized amplitude 1.

The pseudo flare signal thus obtained is adjusted to an appropriate level in the multiplier 705 and the subtracter 1
It is input to 16 and serves to cancel the actual flare contained in the video signal.

A coefficient is given to the multiplier 705 from the gain setting circuit 704. This gain setting circuit 704
Generates a coefficient using the light source intensity information from the light source intensity determination circuit 703 and the optical condition data (intensity information) from the flare data memory 111. The light source intensity determination circuit 703 is used when the aperture driving circuit 10 determines the light source shape.
8 is made to perform a narrowing operation. By this narrowing down, it is possible to determine the saturated state and the non-specific saturation state, obtain the light source intensity, and hold the information.

FIG. 9 shows still another embodiment of the present invention. The same parts as those in the previous embodiment are designated by the same reference numerals. There are various methods for dynamic expansion of a video camera, but there are roughly two types. One is a method of continuously logarithmically compressing and outputting a signal voltage by utilizing the non-linear characteristic of a transistor in a photoelectric conversion unit of an image sensor, and a circuit that captures signals under different sensitivity conditions with multiple exposure times. It is synthesized by processing and output.

Here, an example will be described in which signals picked up under different sensitivity conditions are combined by circuit processing and applied to a method of expanding a dynamic range. The configuration shown in FIG.
It is assumed that the camera is an SC system. A high sensitivity video signal 911 shot at a normal 1/60 second and a low sensitivity video signal 912 shot at an electronic shutter speed set according to the dynamic range expansion ratio are combined by a combining circuit 908 to capture a wide dynamic range. Is done. Low sensitivity is 1/6
When the exposure time is 000 seconds, the dynamic range expansion ratio is 100 times. The imaging element drive circuit 107 alternately outputs two imaging conditions of low sensitivity and high sensitivity for each field, and low sensitivity and high sensitivity imaging signals are alternately input to the signal processing circuit 109 from the imaging element.

The output of the signal processing circuit 109 is the switch 9
It is alternately sent to the one-field memories 904 and 905 via 02. The field memory 90
The outputs of 4, 905 are alternately and selectively output by the switch 903. In the illustrated example, the field memory 905
Indicates that writing is performed and the field memory 904 is in a reading state. Currently written
It is a highly sensitive signal. What is read from the field memory 904 is the low sensitivity signal one field before.
The high sensitivity signal is sent to the subtractor 116 via the switch 907. At the same time, the low-sensitivity and high-sensitivity signals are sent to the state setting circuit 114 and used for predictive flare generation. The image pickup device driving conditions are sent from the image pickup device driving circuit 107 to the state setting circuit 114, and the electronic shutter setting can be grasped.

If the state setting circuit 114 does not properly extract the light source due to improper sensitivity setting, a unique electronic shutter setting is given to the image pickup element drive circuit 107,
It also includes the function of correctly extracting the light source.

Also at this time, in the high-sensitivity image photographed at 1/60, depending on the image of the bright part in the subject or the light source,
That part is saturated on the image sensor, and flare occurs around a bright subject. For this reason, the contrast of the high-sensitivity side image taken at 1/60 is lowered, and the image is extremely hard to see.

Therefore, the bright part of the subject or the brightness of the light source and its shape are extracted from the image photographed by the electronic shutter for 1/6000 seconds, as described in the embodiment of FIG. Then, by using the stored flare stored from the estimated brightness and shape, a flare corresponding to the bright part of the subject actually photographed is artificially created and used as a predicted flare. The predicted flare is subtracted from the sensitive video signal. As a result, flare on the high sensitivity side is reduced to the extent that there is no practical problem. In this way, a wide dynamic range image with little flare can be obtained.

In FIG. 10, parts having the same functions as those of the previous embodiment are designated by the same reference numerals. It is predicted that a video portion having a value larger than a predetermined value is detected from the video signal, the brightness and shape of the video are estimated, and a light source having the estimated brightness is generated from the transfer function of the optical system and the image sensor system. The predicted flare is calculated by the transfer function calculation unit 921 of the image pickup system. The transfer function referred to here includes a lens, an aperture, a response characteristic due to an aperture of an image sensor, an optical LPF (low pass filter) characteristic used in the image sensor, and a flare characteristic which is a total action of these. . The other part is shown in FIG.
This is the same as the previous embodiment described in.

FIG. 11 shows still another embodiment of the present invention. FIG. 11 shows an embodiment in which the present invention is applied to a two-plate type wide dynamic range camera. The light from the subject that has passed through the lens 102 with the diaphragm device 103 enters the prism 930, and 50% of the light travels straight to form a subject image on the photoelectric conversion surface of the image sensor 104 and the reflection of the prism 930. The remaining 50% of light is reflected by the surface 931 and is dimmed through the ND (darkening) filter 932 to be separated into a subject image formed on the photoelectric conversion surface of the other image sensor 933. The subject image reflected by the prism 930 is dimmed by the dynamic expansion ratio when passing through the ND filter 932. Here, the transmittance is 2%. At this time, the dynamic range expansion ratio becomes 50 times. Image sensor 93
The output signal of No. 3 is amplified by the amplifier 934, is input to the signal processing circuit 935, becomes a video signal, the image is horizontally inverted by the horizontal inversion circuit 936, and is input to the synthesis circuit 937. A signal from which the flare has been canceled is supplied from the subtractor 116 to the synthesizing circuit 937. As a result, the flare-removed video signal is obtained from the combining circuit 937.

From the image pickup element drive circuit 107, low sensitivity,
Two high-sensitivity imaging conditions are output, a low-sensitivity signal is output from the image sensor 933, and a high-sensitivity image signal is output from the image sensor 104, and are input to the signal processing circuits 109 and 935. Here, the signal processing circuit 935 processes a high-sensitivity image pickup signal, and the signal processing circuit 109 processes a low-sensitivity image pickup signal.

Since the low-sensitivity image pickup signal is an image in which the right and left are inverted because it is reflected by the prism 930 once, it is processed by the left and right inversion circuit 936 to return it to the original. In the state setting circuit 114, the predicted flare is created and supplied to the subtractor 116 to cancel the flare in the actual video signal, as described in FIG. State setting circuit 114
The image pickup device drive conditions are sent from the image pickup device drive circuit 107, and the image pickup device drive conditions are known.
Further, the state setting circuit 114 is configured so that when the sensitivity setting is inappropriate and the image of the light source cannot be extracted successfully, the original electronic shutter setting is given to the image pickup element drive circuit 107 to correctly extract the light source. ing. By performing the flare correction, it is possible to obtain a wide dynamic range image in which the flare on the high-sensitivity side is reduced to the extent that there is no practical problem.

In the above description, the dynamic range expansion ratio is set by using the optical ND filter, but the sensitivity may be adjusted by using the electronic shutter as in the first and second embodiments. When optically performed, the transmittance may be changed by using an electrochromic filter or liquid crystal.

Although not shown in the figure, when applied to an electronic camera for still image shooting, the flare correction is performed by temporarily enlarging the aperture to capture flare and subtracting the flare from the captured video signal. It can be performed.

In the case of a moving image, an unnatural image may appear when capturing flare, but the unnaturalness is made inconspicuous by using the previous field or the previous frame again only when capturing flare. be able to. Further, when flare is taken in, the video signal level is narrowed down and the APL (average video signal level) is aligned with the surrounding fields or frames, so that the change can be made inconspicuous.

In the embodiment, an example of a monochrome camera is shown, but the present invention is not limited to this, and the principle of the present invention can be applied to a color camera, a single plate type or a three plate type. In the case of a color camera, flare cancellation processing may be applied to the multiplexed video signal or may be applied to each color signal. Further, cancellation processing may be performed for each color signal, and the estimated flare obtained by using the luminance signal may also be used in the color signal processing.

In the above description, the conditions of the diaphragm device are not included, but the diaphragm value of the diaphragm device may be reflected. For example, in the embodiment of FIG. 10, the aperture value is added to the transfer function of the video system.

As described above, according to the present invention, the flare generated by the predetermined light source is stored in advance together with the set value of the optical system, and the portion of strong light of the image is stored in advance based on the reference flare. The actual flare is canceled by artificially generating the predicted flare by using the set value of the system and subtracting the predicted flare from the video signal.

[0047]

As described above, according to the present invention,
Since flare caused by light scattering by the optical system is reduced in a bright subject portion, an image that is easy to see can be obtained.
Further, the flare reducing effect is great because the change in the flare amount is made to follow the flare that changes according to the parameters of the optical system. Further, since the flare is artificially generated by a simple calculation, the circuit scale can be small.

[Brief description of drawings]

FIG. 1 is a diagram showing an embodiment of the present invention.

FIG. 2 is an explanatory diagram showing the principle of flare correction.

FIG. 3 is an explanatory diagram showing the nature of flare.

FIG. 4 is a diagram showing an example of a flare data capturing device.

FIG. 5 is an explanatory diagram showing a method for generating flare data.

FIG. 6 is an explanatory diagram showing a method for generating a reference flare.

FIG. 7 is a diagram showing a specific configuration example of a state setting circuit.

FIG. 8 is an explanatory diagram of a method for generating a predicted flare by PSF synthesis.

FIG. 9 is a diagram showing an embodiment in which the present invention is applied to a wide dynamic range camera.

10 is a diagram showing a specific configuration example of the state setting circuit of FIG.

FIG. 11 is a diagram showing an embodiment in which the present invention is applied to a two-plate wide dynamic range camera.

[Explanation of symbols]

101 ... Subject, 102 ... Lens, 103 ... Aperture device,
104 ... Image sensor, 106 ... Amplifier, 107 ... Image sensor drive circuit, 108 ... Aperture drive circuit, 109 ... Signal processing circuit, 111 ... Flare data memory, 114 ... State setting circuit, 116 ... Subtractor.

─────────────────────────────────────────────────── ─── Continuation of front page (56) References JP-A-10-112799 (JP, A) JP-A-9-238357 (JP, A) JP-A-6-258258 (JP, A) (58) Field (Int.Cl. ⁷ , DB name) H04N 5/243

Claims (6)

(57) [Claims] 1. An image pickup camera, wherein flare generated by a light source whose photographing result has a predetermined brightness in accordance with at least a predetermined aperture value and focal length parameter of an optical system is stored in advance as a storage flare, and the predetermined flare is stored. A video portion having a value larger than the value is detected from the video signal, its brightness is estimated, and the flare that occurs when a light source having the estimated brightness is present is simulated as a predicted flare using the memory flare. And an image pickup device which subtracts from the video signal. 2. The predicted flare that is predicted is obtained by converting the stored flare stored under a predetermined optical condition by at least a parameter such as an aperture value and a focal length during shooting. The image pickup apparatus according to claim 1. 3. The function of performing sensitivity switching using an electronic shutter, the sensitivity is lowered to derive the brightness and shape of the light source, and the predicted flare predicted based on this is calculated. Imaging device. 4. The memory flare is represented by using a PSF of a cylindrical light source as a model.
The imaging device described. 5. The image pickup camera, which obtains one image by synthesizing images taken under at least high-sensitivity and low-sensitivity exposure conditions, wherein the image taken under low-sensitivity exposure conditions has a value larger than a predetermined value. The image pickup apparatus according to claim 1, wherein a video portion is detected from the video signal, and the predicted flare generated from the predicted value is subtracted from the video signal shot under a highly sensitive exposure condition. 6. A video portion having a value larger than a predetermined value is detected from a video signal, its brightness and the shape of an image forming the same are estimated, and a light source having the estimated brightness is an optical system and an image sensor system. An image pickup apparatus, wherein a predicted flare signal predicted to be generated from the transfer function is calculated and obtained and subtracted from the video signal.