JP2016072649A - Imaging apparatus and imaging method thereof - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize a camera system with less moire and high resolution and modulation degree with a simple constitution in combination with a zoom lens for three.SOLUTION: An imaging apparatus is a four-plate type imaging apparatus which comprises a screen position control part and an interpolation processing part. The interpolation processing part generates an asymmetrical interpolation signal of an imaging pixel signal for interpolating the comatic aberration in which image formation expands asymmetrically in the radiation direction in the coma (comet) shape in a direction where image formation converges in point symmetry with the direction where the image formation expands from the imaging pixel signal output from two green image pick-up devices, and adds the asymmetrical interpolation signal of the imaging pixel signal to red, green and blue video signals. The screen position control part controls the interpolation processing part in accordance with the screen position and the asymmetrical comatic aberration of the screen position, and corrects the asymmetrical comatic aberration for each screen position.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an improvement of an imaging apparatus using a solid-state imaging device and an imaging (video signal interpolation) method.

A CDS (Correlated Double Sampling) that removes noise from a signal output from a CCD (Charge Coupled Device) image sensor, a dark current correction and variable gain amplifier (AGC), and an ADC (ADC) that converts the digital video signal Vi AFE (Analog Front End) with built-in Analog Digital Converter) has become widespread, and ADC gradation of AFE has been 10 bits in the past, but 12 bits and 14 bits have become common. Furthermore, improvements have been made to CMOS (Complementary Metal Oxicide Semiconductor) image sensors that integrate high-speed readout by integrating drive circuits and readout circuits.
As digital signal processing circuits are further integrated, it is easy to store and process output signals of multiple lines not only with video-only memory integrated DSPs but also with inexpensive general-purpose FPGAs (Field Programmable Gate Arrays). I can do it now. Megapixel cameras with more than 1 million pixels, HDTV (High Definition TeleVision) cameras, high-speed imaging HDTV cameras, HDTV cameras with recording units, HDTV cameras with Internet Protocol (IP) transmission units, and higher-definition 2K x 4K cameras In addition, uncompressed recording devices using UHDTV (Ultra High Definition TeleVision) and HDD (Hard Disk Drive) with 4K × 8K cameras have been commercialized. Flat-screen image display devices have also been advanced in higher definition 2K × 4K and 4K × 8K UHDTV display, high-speed display, and ultra-thinning.

  Since the refractive index of the lens varies depending on the wavelength of light, the focal length also varies depending on the wavelength of light, and the focal length of the lens varies depending on the wavelength. A chromatic aberration of magnification occurs with different magnifications and different image sizes.

  Further, the degree of modulation of the entire screen is reduced by spherical aberration in which the position of the condensing point in the optical axis direction changes depending on the distance from the optical axis of the incident point. The light emitted from one point off the optical axis does not converge to one point on the image plane, and image formation spreads on one side in the radiation direction like a coma (comet), and the other side is relatively imaged. Therefore, the outline collapses differently between the outer side and the inner side in the radiation direction around the screen. Furthermore, due to the astigmatism that the image point in the concentric direction and the image point in the radial direction are shifted due to the light beam coming from one point off the optical axis, the circumferential outline and the radial outline are broken around the screen. Is different.

  Spherical aberration is proportional to the third power of NA and is the only aberration that appears at the center of the screen, regardless of the width of the field of view. If the refractive index of the concave lens is higher than that of the convex lens, a double lens doublet is better than a single lens. Spherical aberration is reduced by one digit or more. Further, coma is proportional to the square of the aperture ratio NA that is the reciprocal of the aperture ratio F and the first power of the field of view, and image formation spreads on one side in the radiation direction around the screen, and image formation is relatively on the opposite side. Because of convergence, the outline collapses differently between the outside and inside in the radiation direction. Astigmatism is proportional to the first power of NA and the second power of the field of view.

The phenomenon in which the light collected by the lens does not converge at one point is aberration, and the spherical aberration and coma aberration are corrected in the aplanat. Further, the focal position shift due to the difference in the wavelength of the light is changed to the red C line ( 656.3nm) and the blue F line (486.1nm) are corrected at two points called the achromatic lens achromat. In addition, apochromat and Abbe have named those that satisfy the conditions such as the addition of purple g-line (435.8nm), correction of chromatic aberration at three wavelengths, and correction of spherical aberration and coma at two wavelengths.
High magnification zoom lenses often used for relay (such as 88 times the box shape and 42 times the cylindrical shape) can easily correct spherical aberration and coma with two wavelengths at intermediate focal lengths. Even at the telephoto end, it is difficult to correct spherical aberration and coma with two wavelengths. A lens in which spherical aberration and coma aberration are corrected at three wavelengths is large and expensive even for a single focus lens or a low magnification zoom lens, like a super 35 mm movie lens. A high-power zoom lens in which spherical aberration and coma aberration are corrected with three wavelengths is very large and very expensive, so it has not been commercialized.
A lens whose spherical aberration is insufficiently corrected due to insufficient correction and whose modulation degree is lowered even at the center of the screen is insufficient for UHDTV.
Incidentally, the residual aberration varies depending on the aberration correction method.
In addition, the reflection optical system does not use light refraction by glass like a lens, but uses light reflection by a mirror surface. As a result, it is easier to increase the aperture and resolution of the reflecting optical system than the optical system having only a lens. Therefore, the reflection optical system is not only a large-diameter reflective telescope with a diameter of about 0.2 m to 10 m and a super telephoto reflective telephoto lens with a focal length of about 500 mm to 2000 mm for a single lens reflex camera, but also an ArF laser with a wavelength of 193 nm. It has been widely used in semiconductor pattern printing optical systems using extreme ultraviolet light having a wavelength of 10 nm or less. However, the reflecting optical system such as a reflecting telescope is neither a Newton type nor a Cassegrain type, and has no chromatic aberration but has coma aberration. Therefore, adding a lens for correcting coma aberration is large and expensive.

By the way, in an image pickup apparatus having a lens, an image pickup device, and a video signal processing circuit including a contour correction function, a vertical contour correction signal is obtained from a plurality of video signals each having eight or more line memories and delayed by an integer horizontal period. A horizontal contour correction signal is generated from each of the plurality of video signals delayed by an integer pixel, and at the time of confirmation, the vertical contour correction signal and the horizontal contour correction signal are included in the video signal. (See Patent Document 1).
In addition, there is also an imaging device that performs image sharpening processing of aperture correction processing or edge enhancement processing only in the concentric direction, and does not perform image sharpening processing in the radiation direction for an image subjected to distortion correction by image processing (Patent Document). 2).

Japanese Patent Application 2013-008260 (Patent application before the applicant's publication) JP2014-53700

An object of the present invention is to realize an imaging apparatus that interpolates pixels corresponding to coma aberration of an optical system such as a reflective optical system and a lens that spreads on one side in the radiation direction like a coma (comet).
For example, particularly in a high-power zoom lens or a general-purpose zoom lens, chromatic aberration is corrected at three wavelengths, and at an intermediate focal length, light of two wavelengths such as red and green has a spherical aberration and a field that are independent of the field of view. Even with a lens that has been sufficiently corrected for coma and chromatic aberration proportional to the first power of the area, the coma that is proportional to the first power of the field of view is insufficiently corrected at the wide-angle end and telephoto end, especially blue The coma aberration is insufficiently corrected with light of one wavelength, and the image formation spreads on one side in the radiation direction and the image formation is relatively converged on the opposite side. Therefore, the outline collapses differently on the outside and inside of the radiation direction. In addition, a reflection optical system that can easily achieve a large aperture and high resolution has no chromatic aberration but coma aberration.
Therefore, the coma aberration at the wide-angle end and the telephoto end of a high-power zoom lens or a general-purpose zoom lens also corresponds to the coma aberration of the reflection optical system, and the higher resolution 4K or 8K of the 4-plate type of the color separation optical system and RGGB. In such a camera, image formation spreads on one side in the radiation direction and image formation relatively converges on the opposite side. Therefore, pixel interpolation is performed on the outside and inside of the radiation direction in accordance with the collapse of individual contours, and a high magnification zoom lens Alternatively, an imaging apparatus that achieves downsizing and cost reduction of the entire camera including a general-purpose zoom lens or a reflective optical system.

  The image pickup apparatus of the present invention is a four-plate type image pickup apparatus, and includes a screen position control unit and an interpolation processing unit, and the interpolation processing unit has a frame (comet) shape from image pickup pixel signals output from two green image pickup devices. In the radiation direction, image formation spreads on one side of the radiation direction, and on the other side the image formation is relatively convergent, so that the image formation is asymmetric. An asymmetric interpolation signal that interpolates in the direction of convergence is generated, and the asymmetric interpolation signal is at least one of a red video signal, a blue video signal, and a green video signal (spherical aberration and coma at two wavelengths if the zoom lens has an intermediate focal length) Therefore, only the blue video signal is added to the wide-angle end, the telephoto end of the high-power zoom lens, and the red / blue / green image signal for the reflective optical system). Interpolation processing unit according to the aberration And your, and correcting the production of asymmetric interpolation signal for each screen position of the image pickup pixel signal.

  The imaging apparatus of the present invention is the above-described imaging apparatus, further including a storage unit, and the storage unit stores a control value for generating the asymmetric interpolation signal of the imaging pixel signal according to a lens to be attached. It is characterized by doing.

The imaging device of the present invention is
The imaging device described above is an imaging device having a wide aspect ratio of 16: 9 or the like, and the screen position control unit corresponds to the horizontal position (near the center or near the left and right edges) of the pixel, and an asymmetric interpolation signal of the imaging pixel signal. Controlling the generation of
In the above-described imaging device, the screen position control unit determines the screen position as the screen position at the center of the screen, the center of the screen, the radial direction at the left and right ends of the screen, the circumferential direction at the left and right ends of the screen, and the diagonal corners of the screen Controlling the generation of the asymmetric interpolation signal of the imaging pixel signal by substantially dividing the upper left diagonal direction and the upper right diagonal direction at the four diagonal corners of the screen,
The generation of an asymmetric interpolation signal of at least one of the imaging pixel signals is controlled.

  The imaging apparatus of the present invention is the above-described imaging apparatus, wherein the screen position control unit controls generation of an asymmetric interpolation signal of the imaging pixel signal in accordance with a video format input to the interpolation processing unit. And

  Further, the modulation degree correction method of the imaging apparatus according to the present invention acquires R, G1, G2, and B pixel signals output from the R, G1, G2, and B imaging elements, and uses the G1 pixel signal and the G2 pixel signal. An asymmetric interpolation signal is generated, and the generated asymmetric interpolation signal of the image pickup pixel signal is converted into at least one of a red video signal, a blue video signal, and a green video signal (two wavelengths for an intermediate focal length of the zoom lens). Since the spherical aberration and coma aberration are corrected by the above, only the blue image signal is added to the wide-angle end and the telephoto end of the high-magnification zoom lens and the red-blue-green image signal in the case of a reflective optical system).

  Also, the modulation degree correction method of the imaging apparatus of the present invention is a method for controlling the generation of the asymmetric interpolation signal of the imaging apparatus described above, and controls the generation of the asymmetric interpolation signal of the imaging pixel signal according to the lens to be mounted. It is characterized by doing.

Further, a method for controlling the generation of the asymmetric interpolation signal of the imaging apparatus of the present invention is as follows:
The above-described imaging device is a method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of an imaging device having a wide aspect ratio of 16: 9 or the like. Control according to horizontal screen horizontal position,
A method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of the imaging device described above, wherein the generation of the asymmetric interpolation signal of the imaging pixel signal is performed by a radial direction at a center of the screen and a left and right end of the screen and a circumference at the left and right ends of the screen. Control according to the screen position of the upper left diagonal direction at the corner and the diagonal corner of the screen and the upper right diagonal direction at the diagonal corner of the screen,
It is characterized by controlling at least one of the following.

  Furthermore, the method for controlling the generation of the asymmetric interpolation signal of the imaging pixel signal of the imaging device of the present invention is a method for controlling the generation of the asymmetric interpolation signal of the imaging pixel signal of the imaging device described above, and the imaging pixel signal The generation of the asymmetric interpolation signal is controlled according to the video format.

According to the present invention, at least one of a red video signal, a blue video signal, and a green video signal (if the intermediate focal length of the zoom lens corrects spherical aberration and coma aberration with two wavelengths, The wide-angle end and the telephoto end of the magnification zoom lens and red, blue, and green image signals in the case of a reflective optical system) The image of the coma of the lens spreads to one side in the radiation direction like a coma (comet), and the other side is relatively imaged Can be interpolated in consideration of convergence, and moire can be reduced and the modulation degree can be increased.
In addition, the coma aberration at the wide-angle end and the telephoto end of a high-magnification zoom lens or a general-purpose zoom lens (compact compared to a large zoom lens for movies with a super 35 mm size) corresponds to the coma aberration of the reflective optical system, In combination with a high-magnification zoom lens, a general-purpose zoom lens, or a reflective telephoto lens, a camera system with little moiré and high resolution and modulation can be realized with a small and simple configuration.

1 is a block diagram showing a configuration example 1 of an imaging apparatus according to an embodiment of the present invention. 1 is a block diagram showing a configuration example 2 of an imaging apparatus according to an embodiment of the present invention. FIG. 3 is a block diagram illustrating a configuration example 3 of an imaging apparatus according to an embodiment of the present invention. The block diagram for demonstrating operation | movement of the interpolation process part which interpolates the pixel signal (RG1G2B) of the imaging device which concerns on one Example of this invention with the pixel signal of G1G2. FIG. 4 is a schematic diagram showing a bonding position of an image pickup device of an image pickup apparatus according to an embodiment of the present invention (the spatial position of each pixel of R and B is the same position as G1 (on the extension line of the image pickup device of G1 and the image pickup device of G2)) For RB thinning transmission) Schematic diagram showing the bonding position of the image sensor of the image pickup apparatus according to one embodiment of the present invention (R image sensor and B image sensor are arranged at the approximate center of the G1 image sensor and G2 image sensor for RB thinning transmission) ) FIG. 4 is a schematic diagram illustrating a bonding position of an image pickup device of an image pickup apparatus according to an embodiment of the present invention (the image pickup device of R and the image pickup device of B are a gap between the image pickup device of G1 and the image pickup device of G2 (the image pickup device of G1 and G2); Placed on both sides of the center of the image sensor (Bayer array) for all pixels) Schematic diagram showing the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and symmetric interpolation of the RG1G2B pixel near the center of the screen ((a) frame) (Comet) No aberration tail and RB position are the same as G1, (b) No coma tail and RB position is the center of G1G2) The spatial position of the center of each pixel of the four-plate image sensor bonded to the image sensor of the image pickup apparatus according to the embodiment of the present invention and the asymmetrical emission direction of the RG1G2B pixel whose tail of the coma aberration at the left and right ends of the screen is horizontal Schematic diagram showing the phase position of the interpolated video signal ((b) coma tail is right and RB is at the same position as G1, (c) coma tail is left and RB is at the center of G1G2) Asymmetric Radiation Direction Interpolation of RG1G2B Pixels with Vertical Positions of Centers of Each Pixel and Coma Aberration at the Left and Right Edges of the Screen of the Four-Piece Image Sensors Bonded to the Image Sensors of the Imaging Device According to an Embodiment of the Present Invention (E) The coma aberration tail is upward and the RB position is the same as G1. (E) The coma (comet) aberration tail is downward and the RB position is the center of G1G2. ) The spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to the embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are RG1G2B in the lower right direction and the upper left direction. Schematic diagram showing the phase position of the video signal for asymmetrical radial interpolation of the pixel ((g) Coma (comet) aberration tail upwards left, RB at the same position as G1, (h) Coma (comet) aberration tail Is facing down to the right and the RB position is at the center of G1G2) RG1G2B pixels in which the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are the lower left direction and the upper right direction Schematic diagram showing the phase position of the video signal of asymmetric radial direction interpolation ((i) Coma (comet) aberration tail is in the upper right direction, RB is at the same position as G1, (j) Coma (comet) aberration tail is (Left-down and RB position is center of G1G2) Schematic diagram showing the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and symmetric interpolation of the RG1G2B pixel near the center of the screen ((a) frame) (Comet) No aberration tail and RB position are the same as G1, (b) No coma tail and RB position is the center of G1G2) The spatial position of the center of each pixel of the four-plate image sensor bonded to the image sensor of the image pickup apparatus according to the embodiment of the present invention and the asymmetrical emission direction of the RG1G2B pixel whose tail of the coma aberration at the left and right ends of the screen is horizontal Schematic diagram showing phase position of interpolated video signal ((b) Coma aberration tail pointing to the right and asymmetrical radiation direction interpolation (RB position is the same as G1), (c) Coma aberration tail pointing to the left and asymmetrical radiation direction Interpolation (RB position is the center of G1G2)) Asymmetric Radiation Direction Interpolation of RG1G2B Pixels with Vertical Positions of Centers of Each Pixel and Coma Aberration at the Left and Right Edges of the Screen of the Four-Piece Image Sensors Bonded to the Image Sensors of the Imaging Device According to an Embodiment of the Present Invention (E) The coma aberration tail is upward and the RB position is the same as G1. (E) The coma (comet) aberration tail is downward and the RB position is the center of G1G2. ) The spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to the embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are RG1G2B in the lower right direction and the upper left direction. Schematic diagram showing the phase position of the video signal for asymmetrical radial interpolation of the pixel ((g) Coma (comet) aberration tail upwards left, RB at the same position as G1, (h) Coma (comet) aberration tail Is facing down to the right and the RB position is at the center of G1G2) RG1G2B pixels in which the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are the lower left direction and the upper right direction Schematic diagram showing the phase position of the video signal of asymmetric radial direction interpolation ((i) Coma (comet) aberration tail is in the upper right direction, RB is at the same position as G1, (j) Coma (comet) aberration tail is (Left-down and RB position is center of G1G2) FIG. 5 is a schematic diagram illustrating a symmetrical interpolation of a spatial position of each pixel center of a four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and a RB pixel thinned out near the center of the screen. is there. ((A) No coma (comet) aberration tail and RB position is the same as G1, (b) No coma tail and RB position is the same as G1) An RB pixel in which the spatial position of each pixel center of the four-plate image pickup device and the tail of the coma aberration at the left and right ends of the screen of the image pickup apparatus according to an embodiment of the present invention are thinned and transmitted in the horizontal direction. Schematic diagram showing the phase position of the video signal of asymmetric radial direction interpolation ((b) coma tail is rightward and RB position is the same as G1), (c) coma tail is leftward and RB position is Same position as G1)) Asymmetric Radiation Direction Interpolation of RG1G2B Pixels with Vertical Positions of Centers of Each Pixel and Coma Aberration at the Left and Right Edges of the Screen of the Four-Piece Image Sensors Bonded to the Image Sensors of the Imaging Device According to an Embodiment of the Present Invention (E) The coma aberration tail is upward and the RB position is the same as G1, (e) The coma (comet) aberration tail is downward and the RB position is the same as G1 position) The spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to the embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are RG1G2B in the lower right direction and the upper left direction. Schematic diagram showing the phase position of the video signal for asymmetrical radial interpolation of the pixel ((g) Coma (comet) aberration tail upwards left, RB at the same position as G1, (h) Coma (comet) aberration tail Is facing down to the right and the position of RB is the same as G1) RG1G2B pixels in which the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor of the image pickup apparatus according to an embodiment of the present invention and the tail of the coma aberration at the diagonal corners of the screen are the lower left direction and the upper right direction Schematic diagram showing the phase position of the video signal of asymmetric radial direction interpolation ((i) Coma (comet) aberration tail is in the upper right direction, RB is at the same position as G1, (j) Coma (comet) aberration tail is (Left-down and RB position is the same as G1) Flowchart for explaining the operation of FIGS. 4A to 4E (interpolation of all RB pixels by G1G2) Flowchart for explaining the operation of FIGS. 4A to 4E (interpolated RB pixels which are alternately thinned and interpolated with G1G2) Flowchart for explaining the operation of FIGS. 5A to 5E (interpolation of all RB pixels by G1G2) Flowchart for explaining the operation of FIG. 5A to FIG. 5E (interpolation of RB pixels which are alternately thinned and transmitted by G1G2) 6A to 6E are flowcharts for explaining the operation (interpolation of RB pixels which are alternately thinned and transmitted by G1G2).

Hereinafter, an imaging apparatus according to an embodiment of the present invention will be described with reference to the drawings.
FIG. 1A is a block diagram illustrating a configuration example 1 of an imaging apparatus according to an embodiment of the present invention.
In FIG. 1A, an image pickup apparatus 102 includes a color separation optical system 105, a G1 (Green1, first green) image pickup device 103G1, G2 (Green2, second green) image pickup device 103G2, an R (Red, red) image pickup device 103R, B. (Blue) The image pickup device 103B, the video signal processing unit 104, the image pickup device driving unit 190, the screen position control unit 105, and a CPU (Central Processing Unit) unit 106 are configured.

In the imaging device 102, the incident light 100 is imaged by the lens 100, and color-separated into four plates by the color separation optical system 105, and the G1 (first green) imaging element 103G1 and the G2 (second green) imaging element 103G2. It is photoelectrically converted by the R (red) image sensor 103R and the B (blue) image sensor 103B to form four sets of 1080 / 60p image signals, and is configured by an FPGA (Field Programmable Gate Array) or the like controlled by the CPU unit 106. The video signal processing unit 104 including the interpolation processing circuit converts it to 2160 / 60p (sequential scan output) or 2160 / 60i (interlaced scan output).
The video output in FIG. 1A is, for example, two 3G HD-SDI (High Definition Serial Digital Interface) squares or bi-line 4K 4: 4: 4.

FIG. 1B is a block diagram illustrating a configuration example 2 of the imaging apparatus according to the embodiment of the present invention.
In FIG. 1B, the imaging apparatus 102 includes an imaging unit (camera head) 102A and a control unit (CCU, Camera Control Unit) 102B. For example, a 14-bit G1G2 and a 12-bit RB are provided between the imaging unit 102A and the control unit 102B. 60p 4: 2: 2 is converted into two 6G SDIs and transmitted.

In the imaging unit 102A, the incident light 100 is imaged by the lens 100, color-separated into four plates by the color separation optical system 105, and the G1 (first green) imaging element 103G1 and G2 (second green) imaging element 103G2. Photoelectric conversion is performed by the R (red) image sensor 103R and the B (blue) image sensor 103B, various signal processing is performed by the signal processing unit 108, and an HD-SDI video signal is output to the control unit 102B.
The imaging unit 102A is controlled by the CPU unit 107.

The control unit 102B performs various signal processing on the HD-SDI video signal output from the imaging unit 102A in the video signal processing unit 104, and outputs a video signal such as HD-SDI.
The video signal output from the video signal processing unit 104 of the control unit 102B is, for example, two 6G HD-SDI squares or bi-line 4K 4: 4: 4.

FIG. 1C is a block diagram illustrating a configuration example 3 of the imaging apparatus according to the embodiment of the present invention.
In FIG. 1C, the imaging apparatus 102 includes an imaging unit (camera head) 102A and a control unit (CCU, Camera Control Unit) 102B. For example, a 10-bit G1G2 is defined as Y1Y2 between the imaging unit 102A and the control unit 102B. RB is PbPr, 10 bits of 60p 4: 1: 1 are converted into two 3G SDIs and transmitted.
The video signal output from the video signal processing unit 104 of the control unit 102B is, for example, two 3G HD-SDI squares or bi-line 4K 4: 1: 1.

(Image sensor bonding position)
Next, the bonding position of the image pickup element of the image pickup apparatus according to an embodiment of the present invention will be described with reference to FIGS. 3A, 3B, and 3C.
FIG. 3A is a schematic diagram showing the bonding position of the image pickup device of the image pickup apparatus according to the embodiment of the present invention. The spatial position of each pixel of R and B is the same position as G1 (the image pickup device of G1 and the image pickup device of G2). For RB decimation transmission.
FIG. 3B is a schematic diagram illustrating a bonding position of the image sensor of the imaging apparatus according to the embodiment of the present invention.
The R image sensor and the B image sensor are arranged at the approximate center of the G1 image sensor and the G2 image sensor for RB decimation transmission.
FIG. 3C is a schematic diagram illustrating the bonding position of the image pickup device of the image pickup apparatus according to the embodiment of the present invention. The image pickup device of R and the image pickup device of B are separated by a gap between the image pickup device of G1 and the image pickup device of G2. This is for use of all pixels arranged (Bayer array) on both sides of the image sensor and the G2 image sensor.

(Explanation of operation of interpolation processing unit)
The image pickup apparatus of the present invention is a four-plate type image pickup apparatus, and includes a screen position control unit and an interpolation processing unit, and the interpolation processing unit (from a pixel signal output from two green image pickup devices (in a frame (comet) shape). Generates an asymmetric interpolation signal that interpolates (coma) aberration in which the image is spread asymmetrically (in the radiation direction) in the direction in which the image is converged (point-symmetric) opposite to the direction in which the image is spread, and the asymmetric interpolation signal is red At least one of the video signal, the blue video signal, and the green video signal (if the intermediate focal length of the zoom lens corrects spherical aberration and coma with two wavelengths, only the blue video signal is The screen position control unit controls the interpolation processing unit according to the screen position and coma aberration of the screen position, and the asymmetric interpolation for each screen position. Specially compensates for signal generation To.
Next, interpolation processing according to an embodiment of the present invention will be described with reference to FIG.
FIG. 2 is a block diagram for explaining the operation of the interpolation processing unit for interpolating the (RG1G2B) pixel signal with the G1G2 pixel signal of the imaging apparatus according to the embodiment of the present invention.
FIG. 2 is a block diagram showing a detailed configuration of the video signal processing unit 104.
In FIG. 2, the video signal processing unit 104 includes a lateral chromatic aberration / bonding error correction unit 10 and an interpolation processing unit 11.
The chromatic aberration of magnification and bonding error correction unit 10 corrects the chromatic aberration of magnification generated in the lens for the input G1, G2, R, and B signals, and combines the image sensor and the color separation optical system 105. The error is corrected and output to the interpolation processing unit 11.

The interpolation processing unit 11 adds the input G1 signal and G2 signal by the selection unit 41 to generate a G1 + G2 signal, and further outputs it as a pixel high-frequency signal G video signal between G1 and G2.
An LPF (Low Pass Filter) unit 21 attenuates a frequency higher than a specific frequency of the R signal and matches the timing of the R video signal with the G video signal.
The LPF unit 21 delays the R signal by the pixel delay unit 31 for a predetermined time and outputs the delayed signal to the addition unit 61 and the selection unit 42.
The adder 61 adds the R signal and the R signal delayed for a predetermined time, attenuates a frequency higher than the specific frequency of the R signal, and outputs the attenuated signal to the bit shift unit 23.
The bit shift unit 23 performs a predetermined bit shift to match the level of the R signal attenuated in the high frequency with the G video signal, and outputs the result to the adder 63.
The adding unit 63 adds the R signal output from the LPF unit 21 and the interpolation signal output from the adding unit 65 and outputs the result to the selection unit 42.
The selection unit 42 switches between the R signal output from the addition unit 63 and the R signal output from the pixel delay unit 31 according to the image sensor pixel clock signal, and outputs the R signal as an R video signal.

Similar to the LPF unit 21, the LPF unit 22 attenuates a frequency higher than a specific frequency of the B signal and matches the timing of the B video signal with the G video signal.
The LPF unit 22 delays the B signal by the pixel delay unit 32 for a predetermined time and outputs it to the addition unit 62 and the selection unit 43.
The adder 62 adds the B signal and the B signal delayed for a predetermined time, attenuates a frequency higher than a specific frequency of the B signal, and outputs the attenuated frequency to the bit shift unit 24.
The bit shift unit 24 performs a predetermined bit shift to match the level of the B signal attenuated by the high frequency with the G video signal, and outputs the result to the adder 64.
The adder 64 adds the B signal output from the LPF unit 22 and the interpolation signal output from the adder 65 and outputs the result to the selector 43.
The selection unit 43 switches between the B signal output from the addition unit 64 and the B signal output from the pixel delay unit 32 according to the image sensor pixel clock signal, and outputs the B signal as a B video signal.
When the FPGA is installed in the LPFs 21, 22, and 23, low-speed parallel processing that reduces power consumption or time-division processing that reduces the number of gates may be used.

Next, an interpolation signal generation block will be described.
G1n, m + 1 is input to the multiplication unit 44, the pixel delay unit 33, and the line memory unit 25.
The multiplication unit 44 increases or decreases the input G1n, m + 1 level according to the interpolation direction control signal from the screen position control unit 105 and outputs the result to the addition unit 65. Note that n and m are integers.
Adder 65 adds the output signal of multiplier 44 and the output signal of adder 66 and outputs the result to adder 63 and adder 64.

The pixel delay unit 33 delays the input G1n, m + 1 to G1n, m and outputs the result to the multiplication unit 45 and the pixel delay unit 34.
The multiplier 45 increases or decreases the input G1n, m level according to the interpolation direction control signal from the screen position controller 105 and outputs the result to the adder 66.
Adder 66 adds the output signal of multiplier 45 and the output signal of adder 67 and outputs the result to adder 65.

The pixel delay unit 34 delays the input G1n, m to G1n, m−1 and outputs it to the multiplication unit 46.
The multiplication unit 46 increases or decreases the input level of G1n, m−1 according to the interpolation direction control signal of the screen position control unit 105 and outputs the result to the addition unit 67.
Adder 67 adds the output signal of multiplier 46 and the output signal of adder 68 and outputs the result to adder 66.

The line memory unit 25 delays the inputted G1n, m + 1 by one scanning line (1H) to give G1n-1, m + 1, and outputs it to the multiplication unit 47 and the pixel delay unit 35.
The multiplying unit 47 increases or decreases the input G1n−1, m + 1 level according to the interpolation direction control signal from the screen position control unit 105, and outputs it to the adding unit 68.
Adder 68 adds the output signal of multiplier 47 and the output signal of adder 69 and outputs the result to adder 67.

The pixel delay unit 35 delays the inputted G1n−1, m + 1 to G1n−1m, and outputs it to the multiplication unit 48 and the pixel delay unit 36.
The multiplication unit 48 increases or decreases the input G1n−1, m level according to the interpolation direction control signal from the screen position control unit 105, and outputs the result to the addition unit 69.
Adder 69 adds the output signal of multiplier 48 and the output signal of adder 70 and outputs the result to adder 68.

The pixel delay unit 36 delays the input G1n−1, m to G1n−1, m−1 and outputs it to the multiplication unit 49.
The multiplication unit 49 increases or decreases the level of the input G1n−1, m−1 according to the interpolation direction control signal of the screen position control unit 105 and outputs it to the addition unit 70.
Adder 70 adds the output signal of multiplier 49 and the output signal of adder 71 and outputs the result to adder 69.

G2n, m + 1 is input to the multiplication unit 50, the pixel delay unit 37, and the line memory unit 26.
The multiplier 50 increases or decreases the input G2n, m + 1 level according to the interpolation direction control signal from the screen position controller 105 and outputs the result to the adder 71.
Adder 71 adds the output signal of multiplier 50 and the output signal of adder 72 and outputs the result to adder 70.

The pixel delay unit 37 delays the input G2n, m + 1 to G2n, m and outputs it to the multiplication unit 51 and the pixel delay unit 38.
The multiplying unit 51 increases or decreases the input level of G2n, m according to the interpolation direction control signal of the screen position control unit 105 and outputs it to the adding unit 72.
The adder 72 adds the output signal of the multiplier 51 and the output signal of the adder 73 and outputs the result to the adder 71.

The pixel delay unit 38 delays the input G2n, m to G2n, m−1 and outputs it to the multiplication unit 52.
The multiplier 52 increases or decreases the input G2n, m-1 level according to the interpolation direction control signal from the screen position controller 105 and outputs the result to the adder 73.
Adder 73 adds the output signal of multiplier 52 and the output signal of adder 74 and outputs the result to adder 72.

The line memory unit 26 delays the inputted G2n, m + 1 by one scanning line (1H) to be G2n-1, m + 1, and outputs it to the multiplying unit 53 and the pixel delay unit 39.
The multiplier 53 increases or decreases the input G2n−1, m + 1 level according to the interpolation direction control signal from the screen position controller 105, and outputs it to the adder 74.
The adder 74 adds the output signal of the multiplier 53 and the output signal of the adder 75 and outputs the result to the adder 73.

The pixel delay unit 39 delays the input G2n-1, m + 1 to G2n-1, m and outputs the result to the multiplication unit 54 and the pixel delay unit 40.
The multiplication unit 54 increases or decreases the input level of G2n−1, m according to the interpolation direction control signal from the screen position control unit 105, and outputs it to the addition unit 75.
Adder 75 adds the output signal of multiplier 54 and the output signal of multiplier 55 and outputs the result to adder 74.

The pixel delay unit 40 delays the input G2n−1, m to G2n−1, m−1 and outputs it to the multiplication unit 55.
The multiplication unit 55 increases or decreases the input level of G2n−1, m−1 according to the interpolation direction control signal of the screen position control unit 105, and outputs it to the addition unit 75.

Next, the operation of the screen position control unit 105 will be described.
The screen position control unit 105 inputs the horizontal synchronization signal, the pixel clock, and the vertical synchronization signal output from the image sensor driving unit 190.
In addition, the screen position control unit 105 acquires lens type information, focal length information, and aperture ratio information from the CPU unit 106.
Further, the screen position control unit 105 determines the distance (v−V / 2) (h−H / 2) from the screen center based on the scanning line number V, the scanning line number v, the horizontal pixel number H, and the horizontal pixel number h. Interpolation is controlled by calculating the interpolation method from the relationship between the scanning line number, the horizontal pixel number, and the interpolating direction, and the horizontal pixel number from the horizontal pixel counter.

The image pickup apparatus of the present invention is a four-plate type image pickup apparatus, and includes a screen position control unit and an interpolation processing unit. The interpolation processing unit emits radiation in the form of a frame (comet) from pixel signals output from two green image pickup devices. Since image formation spreads on one side of the direction and the image formation converges relatively on the opposite side, coma aberration, which differs in how the outline collapses on the outside and inside of the radiation direction, is opposite to the direction in which the image is spread (point-symmetric) An asymmetric interpolation signal that interpolates in the direction in which the image converges is generated, and the asymmetric interpolation signal is at least one of a red video signal, a blue video signal, and a green video signal (spherical aberration at two wavelengths if the zoom lens has an intermediate focal length) -Since coma is corrected, only the blue image signal is added to the wide-angle end, telephoto end of the high-power zoom lens, and the red-blue-green image signal for the reflective optical system). Interpolation according to position coma And it controls the management unit, and correcting the production of asymmetric interpolation signal for each screen position.
Specifically, the difference between the adjacent interpolation is taken at the center of the screen, and the coma is asymmetrical in the horizontal direction by taking the difference in the direction in which the image is spread (point-symmetric) opposite to the spreading direction at the left and right edges. When the coma aberration is up and down and asymmetrical, the top and bottom are asymmetrically interpolated by taking the difference in the direction of convergence of the image (point symmetry) opposite to the direction in which the image is spread vertically. On the diagonal of the screen, the difference is made in the direction in which the image is converged diagonally opposite to the direction in which the image is spread (point-symmetric), and the difference is asymmetrically interpolated, and the image is spread in proportion to the distance from the center. Differences are taken in the direction in which the opposite (point-symmetric) imaging converges, and asymmetrical interpolation is performed.
In wide aspect ratios such as 16: 9, the difference between the adjacent interpolation is easily obtained at the center of the screen, and the opposite (point-symmetric) image is converged at the left and right ends where the coma aberration is asymmetric at the left and right. The difference in direction is taken and the horizontal is asymmetrically interpolated, and the amount of horizontal asymmetric interpolation is proportional to (h−H / 2).

Example 1
Next, pixel interpolation using the RB all pixels in the vicinity of the center of the screen and the spatial position of each pixel center of the four-plate image pickup device of the image pickup device bonded to the image pickup device according to the embodiment of the present invention. 1A, FIG. 2, FIG. 4A to FIG. 4E, and FIG. A method of transmitting RBs by alternately decimating and interpolating with G1G2 will be described later with reference to FIGS. 1A, 2, 4A to 4E, and 7B.
FIG. 4A is a schematic diagram showing the spatial position of the center of each pixel of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to an embodiment of the present invention and B pixel interpolation near the center of the screen. It is the same position as the G1 image sensor.
FIG. 4A is a schematic diagram illustrating the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image pickup apparatus according to the embodiment of the present invention and symmetric interpolation of the RG1G2B pixel near the center of the screen. . (A) has no coma (comet) aberration tail and the RB position is the same as G1, and (b) has no coma tail and RB is the center of G1G2.
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 4A is omitted.

  The pixel interpolation of one embodiment of the present invention takes a difference from the adjacent interpolation at the center of the screen, takes asymmetrical difference in the horizontal at the left and right asymmetrical coma, and moves up and down in the asymmetrical top and bottom. The difference is asymmetric, and the coma aberration is diagonal and the asymmetrical screen diagonal is diagonal and the difference is asymmetric, and the difference is asymmetric in proportion to the distance from the center.

In FIG. 1A, when the zoom is at an intermediate focal length, the lens 101 has chromatic aberration corrected at three wavelengths and spherical aberration and coma corrected at two wavelengths (apochromatic named by Abbe and having large astigmatism). A high-power zoom in which coma is not corrected near the wide-angle end or near the telephoto end, and a reflective telephoto lens in which coma is not corrected are used.
The imaging device 102 includes a color separation optical system 105, an R (red) imaging element 103R, a G1 (first green) imaging element 103G1, a G2 (second green) imaging element 103G2, and a B (blue) imaging element 103B. The G2 image sensor 103G2 is arranged by shifting the G1 image sensor 103G1 by 1/2 pixel pitch in the vertical direction and 1/2 pixel pitch in the horizontal direction.

  In FIG. 2, the G output video signal is switched between the G1 signal of the G1 image sensor 103G1 and the G2 signal of the G2 image sensor 103G2 by the selection unit 41, and G1 and G2 that do not overlap diagonally with G1 are horizontal. Is generated by interpolating from the surrounding G1 and G2 the horizontal central position of G2 that does not overlap diagonally with G1 and G2.

Next, a method for generating an interpolation signal based on the screen position will be described.
The interpolation signal at the center of the screen will be described with reference to FIGS. 4A and 2.
The interpolation signal is generated by controlling the multiplication units 44 to 57 in FIG. 2 with the interpolation direction control signal output from the screen position control unit.

  In FIG. 4A, the interpolation signal takes a difference from the adjacent interpolation at the center of the screen, the coma aberration is asymmetric at the left and right ends, and the horizontal difference is asymmetrical, and the coma aberration is asymmetrical at the top and bottom. When the diagonal of the coma is diagonal and asymmetrical, the diagonal is asymmetrical and the difference is asymmetrically in proportion to the distance from the center.

The B pixel signal can be interpolated by (Equation 1). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 1)
The R pixel signal can be interpolated by (Equation 2). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 2)

The G pixel signal can be interpolated by (Equation 3). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Equation 3)

Next, an asymmetric interpolation signal in the radial (horizontal) direction at the left and right ends of the screen will be described with reference to FIGS. 4B and 2.
FIG. 4B shows the spatial position of each pixel center of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to the embodiment of the present invention and the pixel of the RG1G2B pixel where the tail of the coma aberration at the left and right ends of the screen is horizontal. It is a schematic diagram which shows the phase position of the video signal of asymmetrical radiation direction interpolation, (b) is the coma aberration tail rightward and the RB position is the same position as G1, (c) is the coma aberration tail leftward and RB The position of is the center of G1G2.
In addition, since the position of R is the same position as B, illustration of an R pixel or a B pixel is omitted.

First, asymmetric interpolation when there are many rightward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 4). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 4)
The R pixel signal can be interpolated by (Equation 5). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Bn, m + 1) / 2 + k (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 5)
The G pixel signal can be interpolated by (Equation 6). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 6)

Then, asymmetric interpolation when there is much leftward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 7). The B pixel signal after interpolation is Bo.
Bo = (Bn.m + Bn.m + 1) / 2 + k (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m (Equation 7)
The R pixel signal can be interpolated by (Equation 8). Let R be the interpolated R pixel signal.
Ro = (Rn.m + Rn.m + 1) / 2 + k (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Equation 8)
The G pixel signal can be interpolated by (Equation 9). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Equation 9)

Next, an asymmetric interpolation signal in the vertical direction at the upper and lower ends of the screen will be described with reference to FIGS. 4C and 2.
FIG. 4C shows the asymmetry of the pixel of RG1G2B in which the spatial position of each pixel center of the four-plate image sensor of the image sensor bonded to the image sensor according to one embodiment of the present invention and the tail of the coma aberration at the left and right edges of the screen are vertical. It is a schematic diagram showing the radial direction interpolation and the phase position of the video signal, (d) coma aberration tail upward, RB position is the same as G1, (e) coma (comet) aberration tail downward And the position of RB is the center of G1G2.
In addition, since the position of R is the same position as B, illustration of R pixel or B pixel is abbreviate | omitted.

First, asymmetric interpolation in the case where there are many upward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 10). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Equation 10)
The R pixel signal can be interpolated by (Equation 11). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 11)
The G pixel signal can be interpolated by (Equation 12). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 12)

Then, asymmetric interpolation in the case where there are many downward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 13). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 13)
The R pixel signal can be interpolated by (Equation 14). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n-1, m−G1n, m−G1n, m + 1−G2n, m) (Formula 14)

The G pixel signal can be interpolated by (Equation 15). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 15)

Next, an interpolated signal for downward coma and an interpolated signal for upward coma at the left corner of the screen will be described with reference to FIGS. 4D and 2.
FIG. 4D shows the spatial position of each pixel center of the four-plate image pickup device of the image pickup device bonded to the image pickup device according to the embodiment of the present invention, and the tails of coma aberration at the four diagonal corners in the lower right direction and the upper left direction. It is a schematic diagram which shows the phase position of the video signal of the asymmetrical radiation direction interpolation of the pixel of RG1G2B of (RG), (g) is the tail of coma (comet) aberration upward and the position of RB is the same position as G1, (h) The tail of coma (comet) aberration is pointing to the lower right, and the position of RB is the center of G1G2.
In addition, since the position of R is the same position as B, illustration of an R pixel or a B pixel is omitted.

First, asymmetric interpolation is shown when there are many upper left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 16). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G1n, m + 1 + G2n, m−G1n, m + 1−G2n−1, m) (Formula 16)
The R pixel signal can be interpolated by (Equation 17). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G1n, m + 1 + G2n, m−G1n, m + 1−G2n−1, m) (Equation 17)
The G pixel signal can be interpolated by (Equation 18). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Equation 18)

Then, asymmetric interpolation is shown in the case where there is much interpolation of coma aberration in the lower right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 19). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Equation 19)
The R pixel signal can be interpolated by (Equation 20). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Equation 20)
The G pixel signal can be interpolated by (Equation 21). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 21)

Next, an interpolation signal when there are many interpolations of upper-right coma or lower-left coma at the four corners of the screen will be described with reference to FIGS. 4E and 2.
FIG. 4E shows the spatial position of the center of each pixel of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to the embodiment of the present invention and the upper right coma aberration or the lower left coma at the diagonal corners of the screen. It is a schematic diagram which shows the phase position of the image signal of B pixel interpolation with much aberration interpolation.
FIG. 4E shows the spatial position of each pixel center of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to one embodiment of the present invention, and the coma aberration tails at the four corners of the screen in the lower left direction and the upper right direction. It is a schematic diagram which shows the phase position of the video signal of the asymmetrical radiation direction interpolation of the pixel of RG1G2B of (1), (i) is a coma (comet) aberration tail is directed to the lower left, and the position of RB is the same position as G1, (j) The coma (comet) aberration tail is in the upper right, and the position of RB is the center of G1G2.
In addition, since the position of R is the same position as B, illustration of an R pixel or a B pixel is omitted.

First, asymmetric interpolation is shown when there are many coma aberrations in the upper right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 22). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G1n, m + G2n, m−G1n, m + 1−G2n−1, m) (Formula 22)
The R pixel signal can be interpolated by (Equation 23). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G1n, m + G2n, m−G1n, m + 1−G2n−1, m) (Equation 23)
The G pixel signal can be interpolated by (Equation 24). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 24)

Then, asymmetric interpolation is shown in the case where there is much interpolation of the lower left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 25). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Equation 25)
The R pixel signal can be interpolated by (Equation 26). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Equation 26)
The G pixel signal can be interpolated by (Equation 27). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Equation 27)

Next, the detailed operation of the first embodiment will be described with reference to FIG. 7A.
FIG. 7 is a flowchart for explaining the operation of FIGS. 4A to 4E.
The screen position control unit 105 detects the image position and the coma aberration information (S701), and proceeds to the process of S702.
In the process of S702, it is determined whether or not the image signal is in the center of the screen. If it is in the center of the screen (YES), the process proceeds to S703, and if it is not in the center of the screen (NO), the process proceeds to S704.
In the processing of S703, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 1), (Equation 2), and (Equation 3), and the process returns to S701.

In the processing of S704, it is determined whether the image signal is rightward at the left and right edges of the image and whether there is a lot of coma. If the image signal is rightward at the left and right edges of the screen and there is a lot of coma (YES), the processing of S705 is performed. If the image signal is not directed to the right at the left and right edges of the screen (NO), the process proceeds to S706.
In the processing of S705, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 4), (Equation 5), and (Equation 6), and the processing returns to S701.
In the process of S706, it is determined whether or not the image signal is leftward at the left and right edges of the screen and there is a large amount of coma. If the image signal is leftward and the coma is large at the left and right edges of the screen (YES), the process of S707 is performed. If the image signal is facing left at the left and right edges of the screen and the coma is not large (NO), the process proceeds to S708.
In the processing of S707, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 7), (Equation 8), and (Equation 9), and the processing returns to S701.

In the process of S708, it is determined whether or not the image signal is upward at the upper and lower ends of the image and there is a large amount of coma. If the image signal is upward and the coma is large at the upper and lower ends of the screen (YES), the process of S709 is performed. If the image signal is not upward at the left and right edges of the screen (NO), the process proceeds to S706.
In the processing of S709, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 10), (Equation 11), and (Equation 12), and the processing returns to S701.
In the processing of S710, it is determined whether or not the image signal is downward at the upper and lower ends of the screen and there is a large amount of coma. If the image signal is downward and the coma is large at the left and right ends of the screen (YES), the processing of S711 is performed. If the image signal is downward at the left and right edges of the screen and the coma is not large (NO), the process proceeds to S712.
In the processing of S711, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 13), (Equation 14), and (Equation 15), and the processing returns to S701.

In the processing of S712, it is determined whether or not the image signal has a large amount of coma aberration at the upper left corner of the screen diagonal corner. If the upper coma aberration is large in the upper left corner of the screen diagonal corner (YES), the processing of S713 is performed. If the coma aberration is not large in the upper left corner of the screen diagonal four corners (NO), the process proceeds to S714.
In the processing of S713, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 16), (Equation 17), and (Equation 18), and the processing returns to S701.
In the processing of S714, it is determined whether or not the image signal has a large amount of coma aberration in the lower right corner of the screen diagonal corner, and if the coma aberration is large in the lower right corner of the screen diagonal corner (YES), the processing of S715 is performed. The process proceeds to S716 when the coma aberration is not large in the lower right direction at the four corners of the screen (NO).
In the processing of S715, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 19), (Equation 20), and (Equation 21), and the processing returns to S701.

In the process of S716, it is determined whether or not the image signal has a large amount of coma in the upper left corner of the screen diagonal corner, and if the coma aberration is large in the upper left corner of the screen diagonal corner (YES), the process of S717 is performed. If the coma aberration is not large in the upper left corner of the screen diagonal corner (NO), the process proceeds to S718.
In the processing of S717, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 22), (Equation 23), and (Equation 24), and the processing returns to S701.
In the process of S718, it is determined whether or not the image signal has a large amount of coma aberration in the lower right corner of the screen diagonal corner. If the coma aberration is large in the lower right direction of the screen diagonal corner, the process of S719 is performed. The process proceeds to S716 when the coma aberration is not large in the lower right direction at the four corners of the screen (NO).
In the process of S719, the interpolation processing unit 11 is controlled to cause the interpolation processing unit 11 to execute the signal processing of (Equation 25), (Equation 26), and (Equation 27), and the process returns to S701.

  Next, a method for transmitting and interpolating the above-described RB by alternately thinning out the spatial position and RB of the center of each pixel of the four-plate image pickup device of the image pickup device bonded to the image pickup device according to the embodiment of the present invention. Only the difference from the pixel interpolation method using all the pixels is described with reference to FIG. 7B of the flowchart for explaining the operation of FIGS. 4A to 4E for interpolating RB pixels that are alternately thinned and transmitted by G1G2. It will be described later.

The interpolation signal at the center of the screen will be described.
The B pixel signal can be interpolated by (Equation 31). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Equation 31)
The R pixel signal can be interpolated by (Equation 32). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 32)
The G pixel signal can be interpolated by (Equation 33). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 33)

First, asymmetric interpolation when there are many rightward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 34). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 34)
The R pixel signal can be interpolated by (Equation 35). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 35)
The G pixel signal can be interpolated by (Equation 36). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 36)

Then, asymmetric interpolation when there is much leftward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 37). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m (Equation 37)
The R pixel signal can be interpolated by (Equation 38). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 38)
The G pixel signal can be interpolated by (Equation 39). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 39)

First, asymmetric interpolation in the case where there are many upward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 40). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 40)
The R pixel signal can be interpolated by (Equation 41). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 41)
The G pixel signal can be interpolated by (Equation 42). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 42)

Then, asymmetric interpolation in the case where there are many downward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 43). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 43)
The R pixel signal can be interpolated by (Equation 44). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 44)
The G pixel signal can be interpolated by (Equation 45). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 45)

First, asymmetric interpolation is shown when there are many upper left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 46). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 46)
The R pixel signal can be interpolated by (Equation 47). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 47)
The G pixel signal can be interpolated by (Equation 48). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 48)

Then, asymmetric interpolation is shown in the case where there is much interpolation of coma aberration in the lower right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 49). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 49)
The R pixel signal can be interpolated by (Equation 50). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 50)
The G pixel signal can be interpolated by (Equation 51). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 51)

First, asymmetric interpolation is shown when there are many coma aberrations in the upper right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 52). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 52)
The R pixel signal can be interpolated by (Equation 53). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 53)
The G pixel signal can be interpolated by (Equation 54). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 54)

Then, asymmetric interpolation is shown in the case where there is much interpolation of the lower left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 55). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 55)
The R pixel signal can be interpolated by (Equation 566). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 56)
The G pixel signal can be interpolated by (Equation 57). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 57)

(Example 2)
Next, pixel interpolation using the RB all pixels in the vicinity of the center of the screen and the spatial position of each pixel center of the four-plate image pickup device of the image pickup device bonded to the image pickup device according to the embodiment of the present invention. Only differences from the first embodiment will be described with reference to 1A, FIG. 2, FIG. 5A to FIG. 5E, and FIG. Only a difference from the first embodiment will be described later with reference to FIGS. 1A, 2, 5A to 5E, and FIG. 8B.
FIG. 5A is a schematic diagram illustrating the spatial position of the center of each pixel of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to the embodiment of the present invention and symmetric interpolation of the RG1G2B pixel near the center of the screen. Yes, (a) has no coma (comet) aberration tail and the position of RB is the same as G1, and (b) has no coma tail and RB is at the center of G1G2.
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 5A is omitted.

FIG. 5B shows an RG1G2B pixel in which the spatial position of the center of each pixel of the four-plate image pickup device and the tail of the coma aberration at the left and right ends of the screen are horizontal in the image pickup apparatus according to an embodiment of the present invention. It is a schematic diagram which shows the phase position of the video signal of asymmetrical radiation direction interpolation of (a), (b) is asymmetrical radiation direction interpolation (the position of RB is the same as G1) with the tail of the coma aberration pointing to the left, and (c) is the frame position. Aberration tail is rightward and asymmetric radial interpolation (RB position is center of G1G2).
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 5A is omitted.

FIG. 5C is a diagram of the RG1G2B pixel in which the spatial position at the center of each pixel of the four-plate image pickup device and the tail of the coma aberration at the left and right ends of the screen are vertical in the image pickup device of the image pickup apparatus according to an embodiment of the present invention. It is a schematic diagram showing the phase position of the asymmetrical radiation direction interpolation and video signal, (d) is the coma aberration tail downward, RB position is the same position as G1, (e) is the coma (comet) aberration tail Upward and the position of RB is the center of G1G2.
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 5A is omitted.

FIG. 5D shows the spatial position of the center of each pixel of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to the embodiment of the present invention and the coma aberration tails at the four corners of the screen in the lower right direction and the upper left portion. FIG. 6B is a schematic diagram showing the phase position of the asymmetrical radial direction video signal of the RG1G2B pixel in the direction, and (g) shows the coma (comet) aberration tail to the lower right and the RB position is the same position as G1, (h ) Is the top of coma (comet) aberration, and the RB position is the center of G1G2.
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 5A is omitted.

FIG. 5E shows the spatial position of the center of each pixel of the four-plate image pickup device of the image pickup device of the image pickup apparatus according to the embodiment of the present invention, and the coma aberration tails at the four corners of the screen in the lower left direction and the upper right direction. It is a schematic diagram which shows the phase position of the video signal of the asymmetrical radiation direction interpolation of the pixel of RG1G2B of (1), (i) is a coma (comet) aberration tail is directed to the lower left, and the position of RB is the same position as G1, (j) The coma (comet) aberration tail is in the upper right, and the position of RB is the center of G1G2.
Note that the bonding position and pixel interpolation of the R image sensor 103R are the same pixel interpolation at the same position as the B image sensor 103B, and therefore the illustration of the R pixel or B pixel in FIG. 5A is omitted.

  In the image pickup apparatus according to an embodiment of the present invention, G1G2 is set to Y1Y2, RB is set to PbPr, and 60p 4: 1: 1 is converted into two 3G SDIs and transmitted. Is interpolated between G1 at the same position as the average of adjacent RBs, adjacent G2 and adjacent G1.

In FIG. 1C, when the zoom is at an intermediate focal length, the lens 101 has chromatic aberration corrected at three wavelengths and spherical aberration and coma aberration corrected at two wavelengths (apochromatic named Abbe has a large astigmatism). A high-power zoom in which coma is not corrected near the wide-angle end or near the telephoto end, and a reflective telephoto lens in which coma is not corrected are used.
The imaging unit 102A includes a color separation optical system 105, an R (red) imaging element 103R, a G1 (first green) imaging element 103G1, a G2 (second green) imaging element 103G2, and a B (blue) imaging element 103B. The image sensor of G2 is arranged with a shift of 1/2 pixel pitch in the horizontal direction and 1/2 pixel pitch in the horizontal direction with respect to the G1 image sensor.
Either the R image sensor and the B image sensor are arranged at substantially the same position as the G1 image sensor, or the R image sensor and the B image sensor are arranged at the approximate center of the G1 image sensor and the G2 image sensor. Either one.

   In FIG. 2, the G video signal is generated by switching the G1 signal of the G1 image sensor 103G and the G2 signal of the G2 image sensor 103G by the selection unit 41.

Next, a method for generating an interpolation signal based on the screen position will be described.
The interpolation signal at the center of the screen will be described with reference to FIGS. 5A and 2.
The interpolation signal is generated by controlling the multiplication units 44 to 57 in FIG. 2 with the interpolation direction control signal output from the screen position control unit.

  In FIG. 5A, the interpolation signal takes a difference from adjacent interpolation at the center of the screen, the coma aberration is asymmetrical at the left and right edges, and the horizontal difference is asymmetrical, and the coma aberration is asymmetrical at the top and bottom. When the diagonal of the coma is diagonal and asymmetrical, the diagonal is asymmetrical and the difference is asymmetrically in proportion to the distance from the center.

The interpolation signal at the center of the screen will be described.
The B pixel signal can be interpolated by (Equation 61). The B pixel signal after interpolation is Bo.
Bo ＝ Bn, m
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 61)
The R pixel signal can be interpolated by (Equation 62). Let R be the interpolated R pixel signal.
Ro = Rn, m, m + 1) / 2
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 62)
The G pixel signal can be interpolated by (Equation 63). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
+ G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Equation 63)

First, asymmetric interpolation when there are many rightward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 64). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 64)
The R pixel signal can be interpolated by (Equation 65). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Bn, m + 1) / 2 + k (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 65)
The G pixel signal can be interpolated by (Equation 66). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 66)

Then, asymmetric interpolation when there is much leftward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 67). The B pixel signal after interpolation is Bo.
Bo = (Bn.m + Bn.m + 1) / 2 + k (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m (Equation 67)
The R pixel signal can be interpolated by (Equation 68). Let R be the interpolated R pixel signal.
Ro = (Rn.m + Rn.m + 1) / 2 + k (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 68)
The G pixel signal can be interpolated by (Equation 69). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 69)

First, asymmetric interpolation in the case where there are many upward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 70). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 70)
The R pixel signal can be interpolated by (Equation 71). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 71)
The G pixel signal can be interpolated by (Equation 72). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 72)

Then, asymmetric interpolation in the case where there are many downward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 73). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 73)
The R pixel signal can be interpolated by (Equation 74). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n-1, m−G1n, m−G1n, m + 1−G2n, m) (Formula 74)
The G pixel signal can be interpolated by (Equation 75). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Equation 75)

First, asymmetric interpolation is shown when there are many upper left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 76). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G1n, m + 1 + G2n, m−G1n, m + 1−G2n−1, m) (Formula 76)
The R pixel signal can be interpolated by (Equation 77). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G1n, m + 1 + G2n, m−G1n, m + 1−G2n−1, m) (Formula 77)
The G pixel signal can be interpolated by (Expression 78). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 78)

Then, asymmetric interpolation is shown in the case where there is much interpolation of coma aberration in the lower right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 79). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 79)
The R pixel signal can be interpolated by (Equation 70). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (3G2n-1, m−G1n, m−G1n, m + 1−G2n, m) (Equation 80)
The G pixel signal can be interpolated by (Expression 81). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 81)

First, asymmetric interpolation is shown when there are many coma aberrations in the upper right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 82). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 82)
The R pixel signal can be interpolated by (Equation 83). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G1n, m + G2n, m−G1n, m + 1−G2n−1, m) (Formula 83)
The G pixel signal can be interpolated by (Equation 84). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 84)

Then, asymmetric interpolation is shown in the case where there is much interpolation of the lower left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 85). The B pixel signal after interpolation is Bo.
Bo = (Bn, m + Bn, m + 1) / 2 + k (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 85)
The R pixel signal can be interpolated by (Equation 86). Let R be the interpolated R pixel signal.
Ro = (Rn, m + Rn, m + 1) / 2 + k (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 86)
The G pixel signal can be interpolated by (Equation 87). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 88)

Only a difference from the first embodiment will be described later with reference to FIGS. 1A, 2, 5A to 5E, and FIG. 8B.

The interpolation signal at the center of the screen will be described.
The B pixel signal can be interpolated by (Equation 91). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Equation 91)
The R pixel signal can be interpolated by (Equation 92). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Formula 92)
The G pixel signal can be interpolated by (Equation 93). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (2G1n, m +, 2G1n, m + 1-G1n-1, m-G1n-1, m + 1-G1n + 1, m-G1n + 1, m + 1
G2n-1, m-1 + G2n, m-G2n-1, m-1-G2n-1, m + 1-G2n, m-1-G2n, m + 1) (Equation 93)

First, asymmetric interpolation when there are many rightward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 94). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 94)
The R pixel signal can be interpolated by (Equation 95). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 95)
The G pixel signal can be interpolated by (Equation 96). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m-G1n, m + 1-G2n-1, m-G2n1, m) (Formula 96)

Then, asymmetric interpolation when there is much leftward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 97). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m (Equation 97)
The R pixel signal can be interpolated by (Equation 98). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 98)
The G pixel signal can be interpolated by (Equation 99). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G1n, m + 1-G1n1, m-G2n-1, m-G2n, m) (Formula 99)

First, asymmetric interpolation in the case where there are many upward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 100). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Equation 100)
The R pixel signal can be interpolated by (Equation 101). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 101)
The G pixel signal can be interpolated by (Equation 102). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n, m-G1n, m-G1n, m + 1-G2n-1, m) (Formula 102)

Then, asymmetric interpolation in the case where there are many downward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 103). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 103)
The R pixel signal can be interpolated by (Equation 104). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 104)
The G pixel signal can be interpolated by (Equation 105). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 105)

First, asymmetric interpolation is shown when there are many upper left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 106). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 42
The R pixel signal can be interpolated by (Equation 107). Let R be the interpolated R pixel signal.
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 106)
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 107)
The G pixel signal can be interpolated by (Equation 108). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + 1 + G2n, m-G1n, m + 1-G2n-1, m) (Formula 108)

Then, asymmetric interpolation is shown in the case where there is much interpolation of coma aberration in the lower right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 109). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 109)
The R pixel signal can be interpolated by (Equation 110). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 110)
The G pixel signal can be interpolated by (Equation 111). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (3G2n-1, m-G1n, m-G1n, m + 1-G2n, m) (Formula 111)

First, asymmetric interpolation is shown when there are many coma aberrations in the upper right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 112). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 112)
The R pixel signal can be interpolated by (Equation 113). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 113)
The G pixel signal can be interpolated by (Equation 114). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G1n, m + G2n, m-G1n, m + 1-G2n-1, m) (Formula 114)

Then, asymmetric interpolation is shown in the case where there is much interpolation of the lower left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 115). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 115)
The R pixel signal can be interpolated by (Equation 116). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) /
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 116)
The G pixel signal can be interpolated by (Equation 117). Let the G pixel signal after interpolation be Go.
Go = (G1n, m + G1n, m + 1 + G2n-1, m + G1n + 1, m) / 4
+ K (G2n-1, m + G1n, m + 1-G1n, m-G2n-1, m) (Formula 117)

(Example 3)
In the third embodiment, the spatial position of each pixel center and the RB near the center of the screen of the four-plate image pickup device bonded to the image pickup device of the image pickup apparatus according to one embodiment of the present invention are alternately thinned and transmitted to transmit G1G2. Only the differences from the first embodiment will be described with reference to FIG. 1A, FIG. 2, FIG. 5A to FIG. 5E, and FIG.

The interpolation signal at the center of the screen will be described.
The B pixel signal can be interpolated by (Equation 121). The B pixel signal after interpolation is Bo. Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (4G1n, m-G2n-1, m-1 -G2n-1, m-G2n, m-1 -G2n, m) (Formula 121)
The R pixel signal can be interpolated by (Equation 122). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (4G1n, m-G2n-1, m-1 -G2n-1, m-G2n, m-1 -G2n, m) (Formula 122)

First, asymmetric interpolation when there are many rightward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 123). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K ((2G1n, m-G2n-1, m-1-G2n, m-1) (Formula 123)
The R pixel signal can be interpolated by (Equation 124). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (2G1n, m-G2n-1, m-1-G2n, m-1) (Formula 124)

Then, asymmetric interpolation when there is much leftward coma at the left and right edges of the screen is shown.
The B pixel signal can be interpolated by (Equation 125). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (2G1n, m-G2n-1, m-G2n, m) (Formula 125)
The R pixel signal can be interpolated by (Equation 126). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (2G1n, m-G2n-1, m-G2n, m) (Formula 126)

First, asymmetric interpolation in the case where there are many upward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 127). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K ((2G1n, m-G2n, m-1-G2n, m-1) (Formula 127)
The R pixel signal can be interpolated by (Equation 128). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
k (2G1n, m−G2n, m−1−G2n, m−1) (Formula 128)

Then, asymmetric interpolation in the case where there are many downward coma at the upper and lower ends of the screen is shown.
The B pixel signal can be interpolated by (Equation 129). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (2G1n, m-G2n-1, m-1-G2n-1, m) (Formula 129)
The R pixel signal can be interpolated by (Equation 130). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (2G1n, m-G2n-1, m-1-G2n-1, m) (Formula 130)

First, asymmetric interpolation is shown when there are many upper left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 131). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K ((G1n, m−G2n, m) (Equation 131)
The R pixel signal can be interpolated by (Equation 132). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m−G2n, m) (Formula 132)

Then, asymmetric interpolation is shown in the case where there is much interpolation of coma aberration in the lower right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 133). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m-G2n-1, m-1) (Formula 133)
The R pixel signal can be interpolated by (Equation 134). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m-G2n-1, m-1) (Formula 134)

First, asymmetric interpolation is shown when there are many coma aberrations in the upper right direction at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 135). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m-G2n, m-1) (Formula 135)
The R pixel signal can be interpolated by (Equation 136). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m-G2n, m-1) (Formula 136)

Then, asymmetric interpolation is shown in the case where there is much interpolation of the lower left coma at the four corners of the screen.
The B pixel signal can be interpolated by (Equation 137). The B pixel signal after interpolation is Bo.
Bo = (Bn-1, m + Bn, m-1 + Bn, m + 1 + Bn + 1, m) / 4
+ K (G1n, m-G2n-1, m) (Formula 137)
The R pixel signal can be interpolated by (Equation 138). Let R be the interpolated R pixel signal.
Ro = (Rn-1, m + Rn, m-1 + Rn, m + 1 + Rn + 1, m) / 4
+ K (G1n, m-G2n-1, m) (Formula 138)

The screen position control unit 105 according to the present embodiment generates an interpolation signal when the lens 101 is mounted, and stores the modulation degree correction control value of the generated interpolation processing unit 11 in a storage unit (not shown).
The storage unit may store a plurality of modulation degree correction control values for each lens.
Note that the screen position control unit 105 may control the interpolation processing unit 11 in real time.

In the present invention, the chromatic aberration is corrected at three wavelengths and the spherical aberration and coma aberration are corrected at two wavelengths (as shown in FIG. 4A, FIG. 5A and FIG. 6A). A lens having a large point aberration), and a four-plate configuration using a color separation optical system and R (red), G (green) 1, G2, and B (blue) image sensors. In an imaging apparatus in which imaging elements are arranged with a 1/2 pixel pitch shifted in the horizontal direction by a 1/2 pixel pitch,
The R image sensor and the B image sensor are arranged at the approximate center of the G1 image sensor and the G2 image sensor, or the R image sensor and the B image sensor are arranged between the G1 image sensor and the G2 image sensor (G1). And the image sensor of G2 and the image sensor of G2 are arranged on either side)
(The pixel output is interpolated corresponding to the collapse of the individual contours on the outside and inside in the radiation direction.) The G output video signal is switched between the G1 image sensor signal and the G2 image sensor signal.
Interpolating R and B pixels (half the number of pixels of G or 1/4 of the number of pixels) from the surrounding G1 image sensor and G2 image sensor pixels in the direction opposite to the collapse of the outline,
The G output pixel orthogonal to the diagonal pixel shift between G1 and G2 to be interpolated at the center of the screen (the pixel corresponding to the R pixel and the B pixel in the Bayer array and below is abbreviated as G1G2 gap pixel) is the signal of the adjacent G1 pixel. The average of the G2 pixel signal and the average of the G2 pixel signal are subtracted from the average of the G2 pixel signal, and the R output pixel to be interpolated at the center of the screen is the average of the adjacent R pixel signals. The pixel of B1 to be interpolated in the center of the screen is obtained by adding the average of the signal of the G1 pixel and the signal of the G2 pixel adjacent to the image and subtracting the signal of the G1 pixel and the average of the G2 pixel of the second peripheral pixel. Adds the average of the adjacent G1 pixel signal and the average of the G2 pixel signal to the average of the adjacent B pixel, and subtracts the average of the second G1 pixel signal and the average of the G2 pixel signal. This is an imaging method characterized by the above.

In addition, the present invention uses a lens in which chromatic aberration is corrected at three wavelengths and spherical aberration and coma aberration are corrected at two wavelengths (as the method of the embodiment of FIG. 6A). ), G (green) 1, G2, and B (blue) image pickup elements, and the G2 image pickup element is ½ pixel pitch in the vertical direction and ½ pixel pitch in the horizontal direction with respect to the G1 image pickup element. In an image pickup apparatus arranged with a pixel pitch shifted, an R image pickup element and a B image pickup element are arranged at substantially the same position as the G1 image pickup element, and is configured by an image pickup unit (camera head) and a control unit (CCU). Y1Y2 and RB as PbPr (60p) 4: 1: 1 (converted to 2 3G SDI) and transmitted
It is an imaging method characterized by performing at least one of the following.

In addition, the present invention uses a lens in which chromatic aberration is corrected at three wavelengths and spherical aberration and coma aberration are corrected at two wavelengths (as a method of the embodiment of FIG. 4A), and a color separation optical system and R (red) ), G (green) 1, G2, and B (blue) image pickup elements, and the G2 image pickup element is ½ pixel pitch in the vertical direction and ½ pixel pitch in the horizontal direction with respect to the G1 image pickup element. In the image pickup apparatus arranged with the pixel pitch shifted, the R image sensor and the B image sensor are arranged at approximately the same position as the G1 image sensor,
It is an imaging method characterized by performing at least one of the following.

  Although the present invention has been described in detail above, it is needless to say that the present invention is not limited to the imaging apparatus described here, and can be widely applied to imaging apparatuses other than those described above.

According to the present invention, a zoom lens for 3 lenses (compact with a large zoom lens for a movie with a super 35 mm size) (for various 2K HDTVs such as a high-magnification super telephoto purchased by a broadcasting station) In combination with the lens, the coma of the lens spreads to one side of the radiation direction like a coma (comet), and the image on the opposite side converges relatively even at the wide-angle end and telephoto end where there is much coma. Interpolate in consideration, and realize a relatively small camera system (with 4K television) that has low moire and high resolution and modulation even at the wide-angle end and telephoto end where there are many coma aberrations. Can do. In particular, axial chromatic aberration is reduced by axial offset bonding of the image sensor, axial chromatic aberration is reduced by correcting chromatic aberration of magnification and bonding error, and it becomes easy to increase resolution and modulation with less moire. .
In particular, R and B that are lost during color difference thinning transmission of YPbPr are asymmetrically interpolated by interpolating the coma aberration of the lens in the direction in which the image is spread like a coma (comet) and the direction in which the point-symmetric image converges, and moire. And the degree of modulation can be increased.
On the other hand, a single-plate color camera using a super 35 mm size 4K resolution on-chip color filter-equipped imaging device cannot correct axial chromatic aberration and lateral chromatic aberration unlike the present invention. A large and expensive dedicated lens that optically corrects axial chromatic aberration and lateral chromatic aberration is required.
Therefore, according to the present invention, using a B4 mount lens widely used for HDTV cameras using three 1920 × 1080 pixel (2K) 2/3 type image sensors and a color separation optical system, 1920 × 1080 pixels are used. Using four (2K) 2/3 type image sensors and a color separation optical system leads to higher resolution and higher modulation degree of a 4K camera such as 3840 × 2160 pixels. In addition, using four 4K 2/3 type image pickup devices such as 3840 × 2160 pixels and a color separation optical system leads to higher resolution and higher modulation degree of an 8K camera such as 7680 × 4320 pixels.
Furthermore, the former type 1 mount lens for three HDTV cameras can be applied to the former 1 + 1/4 type mount lens for three tube HDTV cameras.

101: Lens, 102: Imaging device, 102A: Imaging unit, 102B: Control unit, 103G1: G1 (first green) imaging device, 103G2: G2 (second green) imaging device, 103R: R (red) imaging device, 103B: B (blue) image sensor, 104: Video signal processing unit, 105: Color separation optical system, 106, 107: CPU unit, 108: Interpolation processing unit, 10: Correction unit for chromatic aberration of magnification and bonding error, 11 : Interpolation processing unit, 21, 22: LPF unit, 23, 24, 39: bit shift unit, 31-40: pixel delay unit, 41: selector unit, 44-57: multiplier unit (positive / negative multiplier), 58-75: Adder.

Claims (8)

  This is a four-plate type imaging device having a screen position control unit and an interpolation processing unit. The interpolation processing unit generates coma aberration from the imaging pixel signals output from the two green imaging elements in a direction opposite to the direction in which image formation spreads. Generating an asymmetric interpolation signal of the imaging pixel signal to be interpolated in a direction in which the image converges, and adding the asymmetric interpolation signal of the imaging pixel signal to at least one of a red video signal, a blue video signal, and a green video signal; The control unit controls the interpolation processing unit according to the screen position and coma aberration of the screen position, and corrects the generation of the asymmetric interpolation signal of the imaging pixel signal for each screen position. The imaging device according to claim 1,
Furthermore, it has a storage part,
The storage unit stores a control value for generating the asymmetric interpolation signal of the imaging pixel signal in accordance with a lens to be attached. In the imaging device according to claim 1 or 2,
The above imaging device is a wide aspect imaging device such as 16: 9,
The screen position control unit controls the generation of the asymmetric interpolation signal of the imaging pixel signal corresponding to the horizontal position of the pixel;
In the above-described imaging device, the screen position control unit determines the screen position as the screen position at the center of the screen, the center of the screen, the radial direction at the left and right ends of the screen, the circumferential direction at the left and right ends of the screen, and the diagonal corners of the screen Controlling the generation of the asymmetric interpolation signal of the imaging pixel signal by substantially dividing the upper left diagonal direction and the upper right diagonal direction at the four diagonal corners of the screen,
An image pickup apparatus that controls generation of an asymmetric interpolation signal of at least one of the image pickup pixel signals. The imaging apparatus according to claim 1, wherein
The image position control unit controls generation of an asymmetric interpolation signal of the image pickup pixel signal in accordance with a video format input to the interpolation processing unit.   An imaging method of a four-plate imaging apparatus, wherein R, G1, G2, and B pixel signals output from R, G1, G2, and B imaging elements are acquired, and the imaging pixels are obtained from G1 pixel signals and G2 pixel signals. The imaging pixel of the imaging device, wherein an asymmetric interpolation signal of the signal is generated, and the generated asymmetric interpolation signal of the imaging pixel signal is added to at least one of a red video signal, a blue video signal, and a green video signal A method for controlling the generation of an asymmetric interpolated signal.   A method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of the imaging device according to claim 5, wherein the generation of the asymmetric interpolation signal of the imaging pixel signal is controlled according to a lens to be mounted. Method for correcting modulation degree of imaging apparatus. In the method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of the imaging device according to claim 5 or 6,
The above imaging device is a method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of a wide aspect imaging device such as 16: 9,
Controlling the generation of the asymmetric interpolation signal of the imaging pixel signal according to the screen horizontal position in the horizontal direction of the screen center and the left and right edges of the screen;
A method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of the imaging device described above, wherein the generation of the asymmetric interpolation signal of the imaging pixel signal is performed by a radial direction at a center of the screen and a left and right end of the screen and a circumference at the left and right ends of the screen. Control according to the screen position of the upper left diagonal direction at the corner and the diagonal corner of the screen and the upper right diagonal direction at the diagonal corner of the screen,
A modulation degree correction method for an imaging apparatus, characterized by controlling at least one of the following. In the method for controlling generation of an asymmetric interpolation signal of the imaging pixel signal of the imaging device according to claim 5,
A modulation degree correction method for an imaging apparatus, wherein the generation of the asymmetric interpolation signal of the imaging pixel signal is controlled according to a video format.

Images