JPWO2019058980A1 - Imaging device - Google Patents

Abstract

Provided is an image pickup device using a surface-illuminated CMOS image sensor having a pixel pitch of 5 wavelengths or less, which improves MTF at low frequencies and facilitates focusing even at the telephoto end. An image pickup device that uses a CMOS image sensor whose pixel pitch is 5 times or less the center wavelength of the image pickup light, from the color separation section 2 that separates the input light for each color and outputs a video signal, and the color separation section. For the video signal of, the difference between the video signal of the pixel to be corrected for contour correction and the video signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected is calculated, and the plurality of differences are calculated. It is provided with a contour correction unit 11 that generates a contour correction signal based on the added difference addition signal and adds the contour correction signal to the video signal of the pixel to be corrected to perform contour correction, and includes a high frequency component near fs / 2. It is an image sensor that generates no contour correction signal.

Description

The present invention relates to an imaging device, and more particularly to an imaging device capable of outputting a video signal having an improved degree of modulation at a low frequency by performing contour correction that does not include a high frequency component.

[Explanation of prior art]
CCD (Charge Coupled Device) CDS (Correlated Double Sampling) that removes noise from the signal output from the image pickup device, dark current correction, and variable gain control (AGC), and output to the display device. Therefore, AFE (Analog Front End) with a built-in ADC (Analog Digital Converter) that converts digital video signal Vi has become widespread, and the ADC gradation of AFE, which was 10 bits in the past, is generally 12 bits or 14 bits. It was.
In addition, CMOS (Compleme) that enables high-speed reading by integrating the drive circuit and reading circuit.
ntary Metal Oxide Semiconductor) Improvements in the image sensor have also progressed.

Furthermore, the integration of digital signal processing circuits has progressed, and it is easy to store output signals of multiple lines and perform arithmetic processing not only with a memory-integrated DSP dedicated to video but also with an inexpensive general-purpose FPGA (Field Programmable Gate Array). It has become possible.

As a result, HDTV (High Definition Te), a megapixel camera with more than one million pixels
leVision; High-definition TV) camera, high-definition HDTV camera, HDTV camera with recording unit, HDTV (1080 x 1920) camera with Internet Protocol (IP) transmission unit, higher-definition 4K (2160 x 3840) camera, 8K ( 4320 x 7680) Cameras and the like have been commercialized, and uncompressed recording devices using HDDs (Hard Disk Drives) have also been commercialized.

[Structure of image sensor: FIG. 6]
Here, the configuration of the image sensor will be described with reference to FIG. 6A and 6B are cross-sectional explanatory views showing the configuration of an image sensor, in which FIG. 6A is a CCD sensor (image sensor), FIG. 6B is a conventional CMOS sensor (surface illumination CMOS image sensor), and FIG. 6C is a BSI (c). A Back Side Illumination) type CMOS sensor (back side illumination type CMOS image sensor) is shown.
As shown in FIG. 6A, in the CCD sensor, the microlens 91, the color filter 92, and the photodiode 93 are laminated, and a metal wiring layer 94 is formed between the photodiode 93 and the color filter 92. ing.

The incident light collected by the microlens 91 passes through the color filter 92, is decomposed into R (red), G (green), and B (blue) components, and is incident on the photodiode 93 to generate an electric charge. Then, it is transferred in the vertical direction and the horizontal direction and converted into a voltage for each pixel.
Although the CCD sensor has high sensitivity, it consumes a large amount of power, and it is difficult to integrate peripheral circuits as compared with a CMOS image sensor.

The surface-illuminated CMOS image sensor shown in FIG. 6B has a configuration in which a transistor or the like is provided for each pixel, and although it has low power consumption and is easy to integrate, its sensitivity is low.
The back-illuminated CMOS image sensor shown in FIG. 6C has a metal wiring layer 93 provided on the back surface side of the photodiode 93, and has high sensitivity, low power consumption, and is easy to integrate.

In particular, a general low-priced surface-illuminated CMOS image sensor (4/3 type or less including 1.25 type at 8K or 2/3 type or less at 4K) has a photodiode 93 at a pixel interval of 5 wavelengths or less. Since the metal wiring layer 94 is laminated between the micro lens (on-chip lens) 91 and the micro lens (on-chip lens) 91, the degree of modulation in the low frequency range is low.
Further, if an optical low pass filter (O-LPF; Optical Low Pass Filter) is inserted in order to suppress the moire of the folded false signal of the high frequency component outside the counter frequency, the degree of modulation is further lowered.

Further, in the 2/3 type 4K and the 1.25 type 8K, the pixel pitch of the image sensor is about 2.5 μm, which is about 4.5 times the green wavelength of 0.55 μm, and the aberration of the high magnification zoom lens is optically reduced. Modulation Transfer Function that cannot be corrected and has a low frequency modulation (modulation transfer function characteristic: MTF)
) Decreases.

FIG. 7 is an explanatory diagram showing the relationship between the aperture value and the degree of modulation.
It is known that even with an ideal lens having no aberration, the degree of modulation decreases due to optical diffraction with a green center wavelength of 0.55 μm when the wavelength is narrowed down to the number of wavelengths of the pixel pitch or more.
For example, in a 1/3 type HDTV (1080 × 1920) camera, a pixel pitch is 4.9 wavelengths, and even with 400 TV lines at about F5, a blurred image with a modulation degree of 50% or less is obtained, and a 2/3 type 4K (2160 × 2160 ×). 3840) With a camera, for a pixel pitch of 4.5 wavelengths, an image with a modulation degree of 50% or less is obtained even with 800 TV lines at about F4.8.

In an actual lens, the degree of modulation is generally around F4 for an HDTV (1080 x 1920) camera, near F2.8 for a 4K (2160 x 3840) camera, and near F2 for an 8K (4320 x 7680) camera. It is the highest, and the degree of modulation decreases due to aberration near the maximum aperture, and the degree of modulation decreases due to optical diffraction near the maximum aperture.

On the other hand, the telephoto zoom lens has an aperture value close to 3.7 to 5.3 and 5.6 at the telephoto end, and when an auxiliary lens called an extender that doubles the focal length is inserted, the aperture value doubles and the telephoto lens is telephoto. The aperture value at the edge is 7.4 to 10.6, which is much larger than 5.6.
Further, at the telephoto end, the degree of modulation decreases due to aberration. Therefore, at the telephoto end, there is almost no signal component near fs / 2, and noise is the main component.

[Related technology]
As the prior art relating to the image pickup apparatus, Japanese Patent Application Laid-Open No. 2007-49216 "Contour enhancement processing apparatus and contour enhancement processing method" (Patent Document 1) and JP-A-7-264442 "Solid imaging apparatus" (Patent Document 2). ), Japanese Patent Application Laid-Open No. 2003-333369 "Image Processing Device and Method, Recording Medium, and Program" (Patent Document 3), Japanese Patent Application Laid-Open No. 2009-303206 "Individual Imaging Device and Monitoring System" (Patent Document 4). ..

Patent Document 1 describes a contour enhancement processing device that passes a contour correction signal through a low pass filter (LPF) in order to attenuate a signal component and a noise component in the vicinity of half of the sampling frequency fs. There is.
Patent Document 2 describes an imaging device that lowers the contour correction frequency when the degree of modulation is lowered at telephoto.

Further, Patent Document 3 describes that contour enhancement (contour correction of the difference between pixel intervals in units of 2 or 4) is performed from a delay corresponding to a power of 2 of the basic processing rate.
Patent Document 4 describes an imaging device that lowers the center frequency of contour correction when the aperture value is large and the degree of modulation is lowered by optical diffraction.

JP-A-2007-49216 Japanese Unexamined Patent Publication No. 7-264442 Japanese Unexamined Patent Publication No. 2003-333369 JP-A-2009-303206

As described above, in a conventional image pickup device, in a UHDTV (Ultra High Definition TeleVision) camera using a surface-illuminated CMOS image sensor, when the pixel pitch becomes 5 wavelengths or less of the center wavelength of the image pickup light. There is a problem that the MTF at a low frequency is lowered.
Further, at the telephoto end of the telephoto zoom lens, there is a problem that the MTF is lowered due to lens aberration and diffraction, and focusing is difficult.

The present invention has been made in view of the above circumstances, and in an image pickup device using a surface-illuminated CMOS image sensor having a pixel pitch of 5 wavelengths or less, the MTF at low frequencies is improved and focusing is easily performed even at the telephoto end. It is an object of the present invention to provide an image pickup device capable of performing.

The present invention for solving the problems of the above-mentioned conventional example is an image pickup apparatus using a CMOS image pickup element having a pixel pitch of 5 times or less the center wavelength of the image pickup light, comprising the image pickup element, and coloring the input light. A color separation unit that decomposes each time and outputs a video signal, a video signal of a pixel to be corrected for contour correction of the video signal from the color separation unit, and a pixel interval of a power of 2 from the pixel to be corrected. The difference from the video signal of the plurality of pixels is calculated, the contour correction signal is generated based on the difference addition signal obtained by adding the plurality of differences, and the contour correction signal is added to the video signal of the pixel to be corrected to correct the contour. It is characterized by having a contour correction unit for performing the above.

Further, in the above-mentioned imaging device, the contour correction unit calculates the difference between the video signal of the pixel to be corrected and the video signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected. , A difference addition unit that adds the plurality of differences to generate a difference addition signal, and adjusts the difference addition signal based on the video signal level of the pixel to be corrected to obtain a contour correction signal corresponding to the pixel to be corrected. It is characterized by including a contour correction signal generation unit for generating and a contour correction signal addition unit for adding a contour correction signal to the video signal of the pixel to be corrected.

Further, the present invention is characterized in that, in the above-mentioned imaging device, the pixel interval of the power of 2 is set from 2 pixel interval to 32 pixel interval.

Further, the present invention is characterized in that the image pickup device is a surface irradiation type CMOS image pickup device in the above image pickup device.

Further, according to the present invention, in the above-mentioned image pickup device, contour correction is performed on a video signal input from a lens in which the open aperture value at the telephoto end is larger than the pixel pitch / center wavelength of the image sensor or a lens having a large aberration. It is a feature.

According to the present invention, an image pickup device using a CMOS image sensor whose pixel pitch is 5 times or less the center wavelength of the image pickup light is provided, and the input light is decomposed for each color to output a video signal. With respect to the image signal from the color separation unit and the color separation unit, the image signal of the pixel to be corrected for contour correction and the image signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected. It is provided with a contour correction unit that calculates a difference, generates a contour correction signal based on a difference addition signal obtained by adding the plurality of differences, and adds a contour correction signal to the video signal of the pixel to be corrected to perform contour correction. Since it is an image sensor, it generates a contour correction signal that does not contain high-frequency components, emphasizes the contour correction signal in the low frequency range, improves the degree of modulation in the low frequency range, and focuses at the telephoto end with a shallow depth of view. There is an effect that can be facilitated.

Further, according to the present invention, the contour correction unit calculates the difference between the video signal of the pixel to be corrected and the video signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected, and the plurality of said. The difference addition unit that adds the differences of the above to generate the difference addition signal, and the contour that adjusts the difference addition signal based on the video signal level of the pixel to be corrected and generates the contour correction signal corresponding to the pixel to be corrected. Since the image pickup device is provided with a correction signal generation unit and a contour correction signal addition unit that adds a contour correction signal to the video signal of the pixel to be corrected, the difference addition signal is appropriately adjusted to generate the contour correction signal. This has the effect of improving the degree of modulation in the low frequency range.

Further, according to the present invention, since the image pickup device has the pixel interval of the power of 2 from the interval of 2 pixels to the interval of 32 pixels, the modulation degree of 1.05 MHz of the HDTV for the 2K jump operation is improved. The modulation degree of 27.5 MHz based on 1 MHz can be relatively lowered, and there is an effect that the modulation degree in the low frequency range can be sufficiently improved without increasing the circuit scale so much.

Further, according to the present invention, since the image pickup device is the above-mentioned image pickup device in which the image pickup device is a surface irradiation type CMOS image pickup device, there is an effect that an image pickup device having an improved low frequency modulation degree can be realized at low cost.

Further, according to the present invention, the image pickup device performs contour correction for a video signal input from a lens in which the open aperture value at the telephoto end is larger than the pixel pitch / center wavelength of the image sensor or a lens having a large aberration. Therefore, there is an effect that the degree of modulation in the low frequency range can be improved at low cost and focusing at the telephoto end can be facilitated without using an expensive and large UHDTV dedicated lens.

It is a block diagram which shows the schematic structure of this image pickup apparatus. It is a block diagram of the vertical contour correction part of this image pickup apparatus. It is a block diagram of the horizontal contour correction part of this image pickup apparatus. It is a frequency axis display of the difference addition signal of 2, 4, 8 and 16 pixel intervals. It is a schematic explanatory drawing which shows the contour correction signal of this image pickup apparatus and the signal example after contour correction. It is sectional drawing which shows the structure of an image sensor. It is explanatory drawing which shows the relationship between the aperture value and the degree of modulation.

Embodiments of the present invention will be described with reference to the drawings.
[Outline of Embodiment]
The image pickup device according to the embodiment of the present invention is an image pickup device using a CMOS image pickup device having a pixel pitch of 5 wavelengths or less of the center wavelength of the image pickup light, and the image signal of the pixel to be corrected and the correction thereof. The difference between the target pixel and the video signal of a plurality of pixels having a pixel interval of the power of 2 is calculated, and the contour correction signal is generated based on the difference addition signal obtained by adding the plurality of differences, and the video of the correction target pixel is generated. Since the contour correction signal is added to the signal, it can be a contour correction signal that does not include high frequency components near fs / 2, and the contour correction signal in the low frequency range is reinforced to improve the MTF in the low frequency range. However, focusing can be facilitated even at the telephoto end of the telephoto lens.

[Rough configuration of the imaging device according to the embodiment: FIG. 1]
The schematic configuration of the image pickup apparatus (the present imaging apparatus) according to the embodiment of the present invention will be described with reference to FIG. FIG. 1 is a block diagram showing a schematic configuration of the present imaging apparatus.
As shown in FIG. 1, the image sensor 1 includes a color separation optical system 2, an image sensor 3 (3R, 3G, 3B), a video signal processing unit 4, and a parallel / serial (P / S) conversion unit 5. And CPU (Cent
It includes a ral Processing Unit 6) and is connected to a lens 7 and a viewfinder (image display device) 8.

Each part of the image pickup apparatus 1 will be described.
The color separation optical system 2 color-separates the incident light incident from the lens 7 into red (R), green (G), and blue (B).
The image pickup element 3 is a CMOS image pickup element, the image pickup element 3R photoelectrically converts red light, the image pickup element 3G photoelectrically converts green light, and the image pickup element 3B photoelectrically converts blue light, and the intensity of light of each color is weak. Generates an electric charge according to.
Here, in the image pickup device 3, pixels are arranged at a pixel pitch of 5 wavelengths or less with reference to the wavelength of green light (0.55 μm).
The color separation optical system 2 and the image sensor 3 correspond to the color separation unit described in the claims.

The video signal processing unit 4 includes a contour correction unit 11, a gamma color correction unit 12, and a matrix unit 13, and is composed of an FPGA with a built-in large-capacity memory, an FPGA with an external DDR (Double-Data-Rate) memory, and the like. To.

The contour correction unit 11 is a characteristic portion of the image pickup apparatus, and performs contour correction processing for correcting the contour of an image for each of the red, green, and blue color components.
The contour correction unit 11 includes a straight contour correction unit that performs vertical contour correction processing and a horizontal contour correction processing unit that performs horizontal contour correction processing.

Then, as a feature of this image pickup apparatus, the contour correction unit 11 has an image signal of the pixel to be corrected and an image of pixels having a pixel interval of a power of 2 (for example, an interval of 2, 4, 8, 16, 32 pixels). A contour correction signal is generated based on the sum of the differences from the signal. The configuration and operation of the contour correction unit 11 will be described later.

The gamma color correction unit 12 performs a process of correcting the gradation of the image (gamma color correction).
The matrix unit 13 performs a process of adjusting the brightness and color of the image.
The parallel / serial conversion unit 5 converts the parallel data into serial data and outputs it to the outside.
The CPU 6 controls the entire image pickup apparatus, and in particular, controls the contour correction unit 11 to realize appropriate contour correction.
Further, the CPU 6 controls the level of the contour correction signal according to the lens aperture value.

The operation of this imaging device will be briefly described.
The light input from the lens 7 is separated into red, green, and blue components in the color separation optical system 2, and is photoelectrically converted by the image sensors 3R, 3G, and 3B, respectively, to obtain an electric signal.
Then, the signals of each color are subjected to vertical contour correction and horizontal contour correction processing in the contour correction unit 11, gamma color correction is performed in the gamma color correction unit 12, color adjustment is performed in the matrix unit 13, and parallel. / Serial conversion unit 5 converts the data into serial data and outputs it as a video signal to the outside.

[Structure of vertical contour correction unit: Fig. 2]
Next, the configuration of the vertical contour correction unit of the contour correction unit 11, which is a characteristic portion of the image pickup apparatus, will be described with reference to FIG. FIG. 2 is a configuration diagram of a vertical contour correction unit of this imaging device.
The vertical contour correction unit 11 compares the video signals between the pixels arranged in the vertical direction and corrects the contour in the vertical direction.
As shown in FIG. 2, the vertical contour correction unit includes a plurality of line memory units M20 to M29, a plurality of negative multipliers N20 to N24 and N26 to N30, a positive multiplier P25, and a plurality of adders 20. ~ 29, an image level determining device 28, multipliers 29 and 32, a small amplitude large amplitude compression limiter 31, and a correction signal adding unit 33.

Here, the line memory units M20 to M29, the negative multipliers N20 to N24 and N26 to N30, the positive multiplier P25, and the plurality of adders 20 to 29 correspond to the difference adder described in the claim. However, the image level determination device 28 and the multipliers 29 and 32 correspond to the contour correction signal generation unit.

Each part of the vertical contour correction part will be described.
The line memory unit is the image signal of the line to be corrected (assumed to be 32H) and the image of the line vertically separated by 2, 4, 8, 16, 32 pixels from the correction target line, which is a feature of this device. It holds and outputs the necessary video signal in order to obtain the difference from the signal.

Specifically, the line memory unit M20 stores 16 lines of video signals, the line memory unit M21 stores 8 lines of video signals, and the line memory unit M22 stores 4 lines of video signals. The line memory unit M23 accumulates video signals for two lines and outputs one line to each of the negative multipliers N21 to N24.

The line memory unit M24 holds the correction target line (32H) and the next line (31H), and transfers the correction target line to the positive multiplier P25, the image level determination device 28, and the contour correction signal addition unit 33. Output.

Further, the line memory units M25 and M26 each store video signals for two lines, the line memory unit M27 stores video signals for four lines, and the line memory unit M28 stores video signals for eight lines. The line memory unit M29 accumulates video signals for 16 lines and outputs one line to each of the negative multipliers N26 to N30.

The negative multipliers N21 to N24 multiply the video signals output from the line memory units M20 to M23 by a negative coefficient and output them to the adders 21 to 24.
The negative multipliers N26 to N30 multiply the video signals output from the line memory units M25 to M29 by a negative coefficient and output them to the adders 26 to 29.
The negative multiplier N20 multiplies the uncorrected signal by a negative coefficient and outputs it to the adder 20.
The positive multiplier P25 multiplies the video signal of the correction target line (32H) output from the line memory M24 by a positive coefficient.

The adders 20 to 29 add the outputs of the negative multipliers N20 to N24 and N26 to N30 and the positive multiplier P25, and output the difference addition signal from the adder 20. The adders 20 to 29 correspond to the difference addition unit in the claim.
Here, in the adder 20, the coefficient of the positive multiplier P25 and / and the negative multiplier so that the sum of the differences between the output from each negative multiplier and the output from the positive multiplier P25 is calculated. The coefficients of N20 to N24 and N26 to N30 are adjusted according to the instruction from the CPU 6.

That is, the difference addition signal output from the adder 20 is separated from the video signal of the correction target line (32H) by a power of 2 (here, 2, 4, 8, 16, 32) from the correction target line. It is the total difference from the video signal of the line.
By obtaining the difference signals at intervals of powers of 2 and adding them to generate the contour correction signal in this way, it is possible to generate the contour correction signal without high frequency components.

The image level determination device 28 determines the level of the image signal of the correction target line, and detects the dark portion by the threshold value determination.
The multiplier 29 variably adjusts the positive / negative and the amplification degree in order to attenuate the contour correction signal in the dark portion.
In the region where the image level is low, the image level determination device 28 and the multiplier 29 control to reduce the contour correction signal in order to reduce the noise due to the addition of the contour correction signal, and when the image level is higher than an arbitrary image level, the contour correction signal is reduced. Control is performed to keep the level of the contour correction signal constant.

The CPU 6 controls the coefficients of the negative multipliers N20 to N24, N26 to N30 and the positive multiplier P25, and also controls the positive / negative and the degree of amplification of the amplification value multiplied by the multiplier 29.
Specifically, the CPU 6 outputs only the video signal of the correction target line (32H) and the power of 2 from the correction target line (here, 2, 4, 8, 16, 32) as the output from the adder 20. The coefficient is controlled so as to be the sum of the differences from the video signals of the distant lines.
Further, the CPU 6 of the present imaging apparatus controls the level of the contour correction signal according to the lens aperture value in order to improve the deterioration of the video signal modulation degree when the lens aperture is stopped down. Specifically, when the lens iris is on the aperture side, the coefficient of the multiplier 29 is adjusted to control the contour correction signal to be amplified.

The small-amplitude large-amplitude compression limiter (hereinafter referred to as a limiter) 31 compresses when the difference addition signal has a large amplitude, and limits the compression when the difference addition signal has a small amplitude.
The multiplier 32 multiplies the output from the limiter 31 by a coefficient from the multiplier 29 to generate a vertical contour correction signal of appropriate polarity and level.
The correction signal addition unit 33 adds a vertical contour correction signal to the correction target line (32H) output from the line memory unit M24 to perform vertical contour correction.

As described above, since the peripheral pixel difference processing at one pixel interval is not performed in this imaging device, the degree of modulation in the ultra-low region can be increased and the degree of modulation in the high region is relatively lowered.
For example, NTSC (National Television System Committee) 0.5MHz is defined as 100% 5MHz modulation degree, and 2K jump scanning HDTV 1MHz is defined as 100% 27.5MHz. The high-frequency modulation degree such as the modulation degree is relatively lowered.

[Operation of vertical contour correction unit: Fig. 2]
The operation of the vertical contour correction unit of this image pickup apparatus will be described with reference to FIG.
The pre-correction signal is input to the negative multiplier N20 and the line memory unit M20, and is accumulated in the line memory units M20 to M29 at the clock timing, and the negative multipliers N21 to N30 and the positive multiplication are performed line by line. It is output to the device P25.

Specifically, assuming that the correction target line output from the line memory unit M24 is 32H, 0H, 16H, 24H, and 30H are input to the negative multipliers N20 to N24, respectively, and the positive multiplier P25 is input. H32 is input, and 34H, 36H, 40H, 48H, and 64H are input to the negative multipliers N26 to N30, respectively.
That is, the video signal of the line separated by the power of 2 from the correction target line is input to the negative multipliers N21 to N24 and N26 to N30.

Then, the negative or positive coefficients are multiplied by the negative multipliers N20 to N24, N26 to N30, and the positive multiplier P25, and added by the adders 20 to 29, so that the video signal of the correction target line and other lines are added. The total difference from the video signal is calculated and output from the adder 20 as a difference addition signal.

The amplitude of the difference addition signal is adjusted by the limiter 31, and further adjusted according to the image level by the multiplier 32 to generate a vertical contour correction signal. In the correction signal addition unit 33, the correction target line The video signal and the vertical contour correction signal are added to perform vertical contour correction.
In this way, the operation of the vertical contour correction unit is performed.

In the present embodiment, the vertical contour correction signal is generated based on the total difference up to 32 pixel intervals in the vertical direction, but the total difference between 64 pixel intervals and 128 pixel intervals is obtained. A vertical contour correction signal may be generated.

[Structure of horizontal contour correction unit: Fig. 3]
Next, the configuration of the horizontal contour correction unit, which is a characteristic portion of the image pickup apparatus, will be described with reference to FIG. FIG. 3 is a configuration diagram of a horizontal contour correction unit of this imaging device.
The horizontal contour correction unit compares video signals between pixels arranged in the horizontal direction and corrects the contour in the horizontal direction.
As shown in FIG. 3, the horizontal contour correction unit includes a plurality of pixel delay units D40 to D49, a plurality of negative multipliers N40 to N44 and N46 to N50, a positive multiplier P45, and a plurality of adders 40. ~ 49, an image level determining device 51, multipliers 52 and 54, a small amplitude large amplitude compression limiter 53, and a correction signal adding unit 55.

That is, the horizontal contour correction unit is provided with pixel delay units D40 to D49 in place of the line memory units M20 to M29 of the vertical contour correction unit shown in FIG. 2, and processes the pixels arranged in the horizontal direction. However, since the other basic configurations and operations are the same as those of the vertical contour correction unit, detailed description thereof will be omitted.

Then, in the horizontal contour correction unit, the sum of the differences between the video signal of the correction target pixel and the video signal of the pixels horizontally separated from the correction target pixel by 2, 4, 8, 16, 32 pixels is the sum of the adder 40. Is output as an addition difference signal from, and the amplitude correction of the addition difference signal is performed in the small amplitude large amplitude compression limiter 53, and is adjusted by the multiplier 54 based on the determination result in the image level determination device 51 to correct the horizontal contour. A signal is generated, and in the correction signal addition unit 55, the video signal of the correction target pixel (32d) and the horizontal contour correction signal are added, and the contour correction in the horizontal direction is performed.

Even in the contour correction in the horizontal direction, since the peripheral pixel difference processing at one pixel interval is not performed, the degree of modulation in the ultra-low region can be increased and the degree of modulation in the high region is relatively lowered.

[Example of difference addition signal in this imaging device: FIG. 4]
Here, an example of the difference addition signal in the present imaging apparatus will be described with reference to FIG. FIG. 4 is a frequency axis display of the difference addition signal at 2, 4, 8, and 16 pixel intervals.
Taking FIG. 2 as an example, in the difference addition signal with a two-pixel interval shown in FIG. 4A, a coefficient other than 0 (zero) is set in the positive multiplier P25 and the negative multipliers N24 and N26. The other negative multiplier is a difference addition signal obtained from the adder 40 with a coefficient of 0 set.

Similarly, the 4-pixel interval difference addition signal shown in FIG. 4B is a difference addition signal when the negative multipliers N23 and N27 have a coefficient other than 0 in addition to the case of 2-pixel interval. , The difference addition signal with an interval of 8 pixels is further set with a coefficient other than 0 in the negative multipliers N22 and N28, and the difference addition signal with an interval of 16 pixels is further set with a coefficient other than 0 in the negative multipliers N21 and N29. Is the difference addition signal when is set.

As shown in FIG. 4, since the difference addition signal of this imaging device does not have a high frequency component near fs / 2 including noise, it outputs a video signal with an improved degree of modulation in the low frequency (low frequency range). Is something that can be done.
Therefore, even if the input signal has a low degree of modulation, it is possible to easily focus at the telephoto end having a shallow depth of field.

[Contour correction in this imaging device: FIG. 5]
An example of the contour correction signal and the signal after contour correction in this imaging device will be described with reference to FIG. FIG. 5 is a schematic explanatory view showing a contour correction signal of the present imaging apparatus and a signal example after contour correction.
5A shows a pre-correction signal, FIG. 5B shows a contour correction signal based on a 17H component / 17 pixel component (8 pixel interval) difference addition signal, and FIG. 5C shows a 9H component / 9 pixel component (4). Pixel spacing) contour correction signal based on difference addition signal, (d) is contour correction signal based on 7H component / 7 pixel component (3 pixel spacing) difference addition signal, (e) is 5H component / 5 pixel component The contour correction signal based on the difference addition signal of (2 pixel interval), (f) is the contour correction signal based on the difference addition signal of 3H component / 3 pixel component (1 pixel interval), and (g) is 17H9H5H component / 17 The signal after contour correction by the contour correction signal of 9 pixels 5 pixels component (8 pixels 4 pixels 2 pixel intervals), (h) is the contour correction signal of 9H5H component / 9 pixels 5 pixel component (4 pixels 2 pixel intervals). The signal after contour correction by is shown.
Here, (d) and (f) are signals that are not generated by the present imaging apparatus, but are shown for reference.

The signal (g) is based on the sum of the difference addition signal of (b) 8-pixel interval, the difference addition signal of (c) 4-pixel interval, and the difference addition signal of (e) 2-pixel interval. This is a signal obtained by adding the generated contour correction signal and the pre-correction signal (a).
The contour-corrected signal shown in (g) is, for example, a 17H15H13H11H9H7H5H3H contour correction signal (contour correction signal based on a difference addition signal of 17H15H13H11H9H7H5H3H component / 17 pixel 15 pixel 13 pixel 11 pixel 9 pixel 7 pixel 5 pixel 3 pixel component). Compared with the case of correction with, the fine contour of the high frequency component cannot be reproduced, but the high frequency noise is small, so the contour correction signal can be increased, the contour can be strongly corrected, and the contour can be made sufficiently clear. It is a thing.

Further, the signal of (h) is a contour correction signal generated based on the sum of the difference addition signal of (c) 4-pixel interval and the difference addition signal of (e) 2-pixel interval, and (a). It is a signal obtained by adding the signal before correction of.
The signal after contour correction in (h) has a high frequency component as compared with the case of correction with, for example, a 9H7H5H3H contour correction signal (a contour correction signal based on a difference addition signal of 9H7H5H3H component / 9 pixel 7 pixel 5 pixel 3 pixel component). Although it is not possible to reproduce the fine contours of the above, since there is little high-frequency noise, the contour correction signal can be increased and the contours can be made clear.

Therefore, even when a lens having a low degree of modulation at a low frequency such as a lens in which the open aperture value at the telephoto end is larger than the value of the pixel pitch / center wavelength of the imaging light or a lens having a large aberration is used, the lens is covered. This makes it easy to focus at the telephoto end, which has a shallow depth of field.

Therefore, even if the degree of modulation of the super-telephoto zoom lens with the aperture ratio F decreases, as in the change in the degree of modulation due to the size of the image pickup element with an aspect ratio of 16: 9 and the aperture ratio of the lens shown in FIG. 7, the contour Correction becomes easy.

[Effect of Embodiment]
According to the image pickup device according to the embodiment of the present invention, in the image pickup device using the CMOS image pickup element having a pixel interval of 5 wavelengths or less, the contour correction unit 11 determines the video signal of the pixel to be corrected and the image signal of the pixel to be corrected. The difference between the pixel to be corrected and the video signal of a plurality of pixels having a pixel interval of a power of 2 (2, 4, 8, 18, 32, etc.) is calculated, and the difference is added based on the difference addition signal obtained by adding the plurality of differences. Since the contour correction signal is generated and the correction signal addition unit adds the contour correction signal to the video signal of the pixel to be corrected, it is possible to generate a contour correction signal that does not include a high frequency component near fs / 2. It has the effect of reinforcing the contour correction signal in the low frequency range, improving the MTF in the low frequency range, and facilitating focusing even at the telephoto end of a telephoto lens having a shallow field of view.

Further, in this imaging device, the total difference between the compensation target pixel and the pixel having the interval of 2, 4, 8, 16, 32 pixels is calculated, but more line memory such as 64 pixels and 128 pixels is calculated. A configuration may include a unit or a pixel delay unit, a multiplier, and an adder, and the more the number is, the greater the effect of contour correction. Therefore, if the circuit scale is acceptable, it is desirable to increase the number.
Further, since the image level judge output signal controls the contour correction signal level output from the small signal large amplitude compression limiter, the contour correction signal is controlled by controlling the image level judge output signal level according to the lens aperture value. It also makes it easier to control the level.

If the contour correction signal is generated by adding the difference of 64 pixel intervals, for example, the 0.5 MHz modulation degree of the HDTV of the 2K jump operation can be improved, and if the difference of 128 pixel intervals is further added, the 2K jump can be improved. There is an effect that the modulation degree of 0.25 MHz of the operation HDTV can be improved.

In addition, the 1 / 3-inch HD2K image sensor for monitoring, the 2 / 3-inch 4K image sensor for news coverage, and the 4 / 3-inch 8K image sensor for relay have the same pixel spacing of 2.5 μm, and are super-telephoto. The aberration is large, the aperture is dark, and the MTF is low. However, according to this device, it is possible to improve the MTF at low frequencies by allowing these characteristics, and to reduce the price of the image sensor. There is an effect that can be realized.
That is, this device has an effect of enabling contour enhancement at low cost by electronically correcting without using an expensive and large UHDTV lens having a high low-frequency MTF.

Furthermore, according to this device, not only sales expansion of 1 / 3-inch HD2K cameras, 4K cameras and 8K cameras for surveillance, but also image quality improvement by upgrading the firmware of 2K cameras owned by many broadcasting stations and relay companies. There is an effect that can be utilized.

The present invention is suitable for an imaging device capable of outputting a video signal with an improved degree of modulation at low frequencies by performing contour correction without high frequency components. This application claims the benefit of priority on the basis of Japanese application Japanese Patent Application No. 2017-183338 filed on September 25, 2017, the entire disclosure of which is incorporated herein by reference.

1 ... Imaging device, 2 ... Color separation optical system, 3,3R, 3G, 3B ... Image sensor, 4 ... Video signal processing unit, 5 ... Parallel / serial conversion unit, 6 ... CPU, 7 ... Lens, 8 ... Viewfinder , 11 ... Contour correction unit, 12 ... Gamma color correction unit, 13 ... MATRIX unit, M20 to M29 ... Line memory unit, N20 to N24, N26 to N30, N40 to N44, N46 to N50 ... Negative multiplier, P25, P45 ... Positive multiplier, 20-29, 40-49 ... Adder, 28, 51 ... Video level judge, 29, 30, 52, 54 ... Multiplier, 31, 53 ... Small amplitude Large amplitude compression limiter, 33, 55 ... Correction signal adder, D40 to D49 ... Pixel delay, 91 ... Microlens, 92 ... Color filter, 93 ... Photodiond, 94 ... Metal wiring layer

Claims (5)

An image pickup device using a CMOS image sensor whose pixel pitch is 5 times or less the center wavelength of the image pickup light.
A color separation unit provided with the image sensor, which separates input light into colors and outputs a video signal
With respect to the video signal from the color separation unit, the difference between the video signal of the pixel to be corrected for contour correction and the video signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected is calculated. It is characterized by having a contour correction unit that generates a contour correction signal based on the difference addition signal obtained by adding the plurality of differences and adds the contour correction signal to the video signal of the correction target pixel to perform contour correction. Imaging device. The contour correction unit calculates the difference between the video signal of the pixel to be corrected and the video signal of a plurality of pixels having a pixel interval of a power of 2 from the pixel to be corrected, and adds the plurality of differences to make a difference. The difference addition part that generates the addition signal and
A contour correction signal generation unit that adjusts the difference addition signal based on the video signal level of the pixel to be corrected and generates a contour correction signal corresponding to the pixel to be corrected.
The imaging device according to claim 1, further comprising a contour correction signal addition unit that adds the contour correction signal to the video signal of the pixel to be corrected. The imaging device according to claim 1 or 2, wherein the pixel interval of a power of 2 is set from a pixel interval of 2 pixels to an interval of 32 pixels. The image pickup apparatus according to any one of claims 1 to 3, wherein the image pickup device is a surface-illuminated CMOS image pickup device. Any of claims 1 to 4, wherein the contour correction is performed on a video signal input from a lens whose open aperture value at the telephoto end is larger than the pixel pitch / center wavelength of the image sensor, or a lens having a large aberration. The imaging device described.

Images