JPH10248069A - Image pickup device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To avoid the generation of false signal at an edge part and to provide an image having no mosaic-shaped distortion by processing the pixel interpolation of image information only with the pixels of that image information. SOLUTION: While using a spatial pixel shifting method, an image pickup device picks up the image having the edge part, for example. On the other hand, an image pickup means 1 generates image information by performing the photoelectric conversion of edge image, and another image pickup means 1 generates image information B by performing the photoelectric transduction of edge image. Next, an interpolating means 2 prepares image information C by performing interpolating processing to the information A. Besides, image information D is prepared by performing interpolating processing to the image information B. Then, an image synthesizing means 3 prepares a synthetic image E by adding correspondent pixel output concerning the image information C and D. Thus, the interpolating means 2 prepared the image information C only from the pixel output of image information A and prepared the image information D only from the pixel output of image information B. Thus, interpolating processing can be performed without mutually utilizing the pixel output of the other image pickup means 1.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image pickup apparatus for picking up images using a spatial pixel shifting method, and more particularly to an image pickup apparatus for performing an interpolation process before performing image synthesis.

[0002]

2. Description of the Related Art Generally, a video camera or an electronic camera is equipped with a multi-plate type image pickup apparatus using a color separation prism. In this multi-panel imaging apparatus, an imaging method called a spatial pixel shift method is employed to realize high resolution. A known imaging apparatus using this spatial pixel shifting method (Technical Report of the Institute of Television Engineers of Japan, 17 [51] (199)
3) p.1-6) will be described with reference to FIGS.

In FIG. 12, a dichroic prism 52 is arranged on the optical axis of a photographing lens 51, and the incident light from the photographing lens 51 is mixed light of an R component and a B component (magenta component) and two G light components. Separates into component light. Note that the photolysis of the G component is decomposition by a half mirror, and is not a wavelength-dependent division.

In the traveling direction of the mixed light, the RB image sensor 5
A color filter array 52a in which red and blue color filters are arranged in a striped or checkered pattern is attached to the imaging surface of the color filter array 3a. RB image sensor 53
In a, the R component and the B component of the optical image are photoelectrically converted. On the other hand, in the traveling directions of the two G component lights, the G image pickup devices 53b and 53c are individually arranged, and the G component of the light image is photoelectrically converted in each device. The pixel output of the G image sensor 53b is indicated by a circle in FIG. 13A, and the pixel output of the G image sensor 53c is indicated by a circle in FIG. 13B.

The G image pickup devices 53b and 53c are arranged with a shift of 1/2 pixel in the vertical and horizontal directions. The signal processing circuit 54 alternately rearranges the pixel outputs of the G imaging element 53b (indicated by a circle in the figure) and the pixel outputs of the G imaging element 53c (indicated by a circle in the figure) on a composite grid point. FIG.
A composite image as shown in D is created. Note that a cross in the figure indicates that there is no corresponding pixel.

Further, the signal processing circuit 54 performs an interpolation process on a vacant point having no corresponding pixel (marked by x in the figure). As the interpolation method, upper and lower average interpolation (FIG. 13E) for calculating and interpolating the average value of the upper and lower pixels of the adjacent pixels, previous value interpolation (FIG. 13F) for interpolating with the immediately preceding (left adjacent) pixel, and left and right Left-right average interpolation for calculating and interpolating the average value of pixels (Fig. 1
3G).

By performing such image processing, an image pickup apparatus using the spatial pixel shifting method can obtain a resolution approximately twice that of an image obtained from a single-chip image pickup element.

[0008]

FIG. 14 and FIG. 15 are diagrams for explaining the problem by using specific numerical values for the above image processing.

As shown in FIG. 14A, for example, if a vertical stripe image is taken, a vertical stripe image is formed on the imaging surface of each of the G imaging elements 53b and 53c. Since the light receiving cells constituting the G image pickup device 53b correspond to either the bright part or the dark part of the stripe, the light receiving cells have a light receiving amount of “0” or “4”. Further, the light receiving cells constituting the G image pickup device 53c correspond to the boundary between the bright and dark portions of the stripe, so that all the light receiving cells have a light receiving amount of "2".

The signal processing circuit 54 includes a G image pickup device 53b.
The image output shown in FIG. 14B is created by replacing the pixel output photoelectrically converted by (1) with the position on the composite grid point. Also,
Similarly, the pixel output of the image sensor 53c is replaced with a position on the composite grid point to create an image shown in FIG. 14C. Further, the signal processing circuit 54 combines the two images,
A composite image shown in FIG. 14D is created. This is shown in FIG.
14B (for example, the pixel a is relocated to the position of the lattice point a).

Next, the signal processing circuit 54 performs an interpolation process on the vacancy points of the composite image. Here, proximity uniform interpolation may be performed in addition to the interpolation method described above. The proximity uniform interpolation is a process of calculating a weighted sum of a matrix for each pixel adjacent to a vacancy to be interpolated and using the calculated sum as an interpolation value. For example, in the case of using the matrix of FIG. 15E, each pixel positioned vertically and horizontally on a vacancy is multiplied by ４, and the sum is used as an interpolation value.

The results of the interpolation are shown in FIGS. FIG. 15A shows vertical interpolation, FIG. 15B shows previous value interpolation, and FIG.
FIG. 15D is a diagram showing a case where left-right average interpolation is performed, and FIG. By the way, paying attention to the edge portions in FIGS. 15A to 15D, in all the interpolation methods except the vertical average interpolation method in FIG. 15A, a false signal is generated in the edge portion, and a mosaic image distortion occurs. This is the G image sensor 53
This is because not only the pixel output of b but also the pixel output of the G image pickup device 53c, which is another image information, contributes to the interpolation and causes generation of a false signal.

When a horizontal stripe image is taken, a false signal is not generated for left-right average interpolation because a vertical stripe image is simply rotated by 90 degrees, but a false signal is generated for vertical average interpolation. As a result, the distortion of the mosaic image occurs at the edge portion. That is, when an interpolation process is performed on an image including a vertical stripe grid image and a horizontal stripe grid image (image having an edge portion), a false signal always occurs at the edge portion.

In the conventional imaging apparatus, the false signal is removed by performing an image smoothing process using a linear filter. However, the false signal cannot be completely removed. In the imaging device of the above, generation of a false signal at an edge portion caused deterioration of image quality. Therefore, an object of the present invention is to provide an imaging apparatus using a spatial pixel shifting method capable of avoiding generation of a false signal at an edge portion in order to solve the above-described problem. .

[0015]

FIG. 1 is a block diagram showing the principle of the first aspect of the present invention. According to the first aspect of the present invention, image information is generated by individually photoelectrically converting light images divided and formed on respective light receiving surfaces via an imaging optical system and a light splitting unit, and image information is generated along the light receiving surfaces. In an imaging apparatus that has a plurality of imaging units 1 that are arranged to be shifted from each other in the direction and performs imaging by the spatial pixel shifting method, pixel interpolation is performed on image information generated by each of the imaging units 1, and the space between the image information is It comprises an interpolating means 2 for adjusting the phase and an image synthesizing means 3 for adding a pixel output corresponding to each piece of image information interpolated by the interpolating means 2 to create a synthesized image.

(Operation) FIG. 2 is a diagram for explaining the operation of the first aspect of the present invention. It is assumed that the image pickup apparatus according to claim 1 has picked up, for example, an image having an edge portion (hereinafter, referred to as an edge image) by using the spatial pixel shifting method. One imaging unit 1 generates image information A by photoelectrically converting the edge image. The other imaging means 1 generates image information B by photoelectrically converting the edge image. It is assumed that one imaging unit 1 has a light reception amount of “0” or “4” for each light receiving cell, and the other imaging unit 1 has a light reception amount of “2”.

Next, the interpolation means 2 performs an interpolation process on the image information A to create image information C. Further, an interpolation process is performed on the image information B to create the image information D.
Note that the image information C and D obtained as a result of the interpolation processing are examples. The image combining means 3 adds the corresponding pixel outputs of the image information C and D to create a combined image E.

As described above, the interpolation means 2 outputs the image information A
The image information C is created only by the pixel output of the image information B, and the image information D is created only by the pixel output of the image information B. Therefore, since the interpolation processing can be performed without using the pixel outputs of the other imaging units 1, no false signal is generated at the edge portion of the composite image E.

[0019]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) FIG. 3 is a diagram showing a configuration of the first embodiment. In FIG. 3, the imaging device 21
The photographing lens 22 is disposed inside the camera.
A prism 23 is arranged on the optical axis of the optical disk. The taking lens 22 corresponds to the image pickup optical system according to the first aspect, and the prism 23 corresponds to the split optical unit.

A half mirror is formed on the bonding surface inside the prism 23, and the light image is divided into two directions, a reflection direction and a transmission direction.
The image sensors 24a and 24b are arranged at positions where the image is divided and the image is received. This image sensor 24
“a” and “24b” are shifted from each other by １ ／ pixel in the vertical and horizontal directions. Further, a green color filter is attached to the imaging surfaces of the imaging devices 24a and 24b, and an image of only the G component is extracted.

The output of the image sensor 24a is supplied to the A / D converter 2
The data is input to the frame memory 26a via the terminal 5a. Further, the output of the frame memory 26a is input to the frame memory 28a via the interpolation calculator 27a. On the other hand, the output of the image sensor 24b is input to the frame memory 26b via the A / D converter 25b. Further, the output of the frame memory 26b is input to the frame memory 28b via the interpolation calculator 27b.

The outputs of the frame memory 28a and the frame memory 28b are input to the combining operation unit 29, and the output is input to the frame memory 30. The output of the frame memory 30 is input to an external device such as a monitor. The functions of the interpolation calculation unit 27a, the interpolation calculation unit 27b, and the synthesis calculation unit 29 are realized by a single microprocessor, DSP, or hardware having equivalent functions.

Regarding the correspondence between the first embodiment and the first embodiment, the image pickup means 1 corresponds to the image pickup devices 24a and 24b and the A / D converters 25a and 25b, The means 2 corresponds to the interpolation calculation units 27a and 27b, and the image synthesis means 3 corresponds to the synthesis calculation unit 29.
FIG. 4 is a flowchart for explaining the operation of the first embodiment. FIG. 5 is a diagram (1) for explaining the image processing of the first embodiment. FIG. 6 is a flowchart of the first embodiment. FIG. 4 is a diagram (2) illustrating image processing.

The operation of the first embodiment will now be described with reference to FIGS.
This will be described with reference to FIG. The light image transmitted through the taking lens 22 is
The light enters the prism 23 and is divided into two directions of reflected light and transmitted light by the reflection / transmission layer inside the prism 23. The imaging element 24a receives the reflected light, but performs imaging through a color filter, and thus photoelectrically converts a light image of only the G component. The pixel output at this time is indicated by a circle in FIG.

The A / D converter 25a performs A / D conversion of the photoelectric signal to generate image information (step S1). The image information is stored and accumulated one by one in the frame memory 26a, and is read out to the interpolation calculation unit 27a in parallel with the writing operation (step S2). The interpolation calculation unit 27a interpolates a pixel (square mark in FIG. 5C) at a vacant grid point using each pixel output (circle mark in FIG. 5C) (step S3). Details of this pixel interpolation will be described later.

The interpolated image information is stored in the frame memory 2
8a. On the other hand, the image sensor 24
Similarly, the A / D of the pixel output b (marked by ● in FIG. 5)
A / D conversion is performed in the conversion unit 25b to generate image information (step S1). The image information is stored and accumulated one by one in the frame memory 26b (step S2), and each pixel output (indicated by ● in FIG. 5D) is output by the interpolation calculation unit 27b.
Is used to interpolate the pixel (the mark x in FIG. 5D) into the vacancy (step S3). The interpolated image information is stored and stored one by one in the frame memory 28b.

In the combining operation unit 29, the image pickup devices 24a,
4b corresponding to the image information of FIG.
The pixel A of C and the pixel B of FIG. 5D are sequentially added to create a composite image (FIG. 5E) (step S4). The composite image is stored and accumulated in the frame memory 30. Next, in describing the effects, the above processing will be described using specific numerical values.

As shown in FIG. 6A, for example, when a vertical stripe image is taken, a vertical stripe image is formed on the imaging surfaces of the image pickup devices 24a and 24b. Since the light receiving cells constituting the image sensor 24a correspond to either the bright portion or the dark portion of the stripe, the light receiving cells have a light receiving amount of "0" or "4". Further, the light receiving cells constituting the image sensor 24b correspond to the boundary between the bright and dark portions of the stripe, so that all the light receiving cells have a light receiving amount of "2".

The interpolation calculator 27a performs pixel interpolation on the vacant grid points of the image information generated by the image sensor 24a and the A / D converter 25a using the interpolation matrix shown in FIG. 6B. This interpolation processing will be described with reference to FIG. The pixel interpolation uses two pixels “0” or “4” positioned above and below the vacancy, or four pixels “0” and “4” positioned diagonally, and uses these pixels as an interpolation matrix. Are interpolated by performing a local product-sum operation for calculating the sum. The result of this interpolation is shown in FIG. 6C.

On the other hand, the same pixel interpolation processing is performed on the pixel output of the interpolation calculation unit 27b to create image information shown in FIG. 6D. Pixel-interpolated image information (FIGS. 6C and 6D)
Since the spatial phases are the same, the corresponding pixels are sequentially added (for example, the pixel A and the pixel B are added to generate the pixel C) to create a composite image (FIG. 6E).

As shown in FIG. 6E, no false signal is generated at the edge portion of the composite image, and no distortion occurs in the mosaic image. This is because there is no need to perform interpolation using pixels of another image sensor at the time of pixel interpolation. With this, in the image pickup apparatus of the present invention, it is possible to avoid generation of a false signal at an edge portion. Further, in the imaging device according to the present embodiment, all the pixels forming the composite image are generated as a result of the addition processing. Therefore, at the time of pixel addition, random noise such as thermal noise can be more reliably reduced, and S / N can be improved.

(Second Embodiment) In the first embodiment, G
Although the imaging apparatus captures only the component image, the second embodiment is an imaging apparatus that captures a color image. FIG. 8 is a diagram illustrating the configuration of the second embodiment. In FIG. 8, a photographing lens 22 is disposed inside an imaging device 21, and a prism 31 is disposed on the optical axis of the photographing lens 22. A half mirror is formed on the bonding surface inside the prism 31, and the light image is divided into two directions, a reflection direction and a transmission direction. Image sensors 24a and 24b are arranged at positions where the image is received. These imaging elements 24a,
24b are arranged with a shift of 1/2 pixel in the vertical and horizontal directions. Further, since the Bayer array color filter array is attached to the imaging surfaces of the imaging devices 24a and 24b, each of the imaging devices can capture RGB color signals.

The output of the image sensor 24a is supplied to the A / D converter 2
The data is input to the frame memory 32a via the memory 5a. Further, the output of the frame memory 32a is input to the compression operation unit 33a, and the output of the compression operation unit 33a is once returned to the frame memory 32a and then input to the frame memory 35a via the connection terminal 34a. The output of the frame memory 35a is once returned to the frame memory 35a via the decompression operation unit 36a and then input to the interpolation operation unit 27a.

On the other hand, the output of the image sensor 24b is input to the frame memory 32b via the A / D converter 25b. The output of the frame memory 32b is output to the compression
The output of the compression operation unit 33b is once returned to the frame memory 32b and then input to the frame memory 35b via the connection terminal 34b. Frame memory 35b
Is returned to the frame memory 35b via the decompression operation unit 36b and then input to the interpolation operation unit 27b.

The outputs of the interpolation calculation units 27a and 27b are input to the frame memory 30 via the synthesis calculation unit 29.
Each function of the interpolation calculation units 27a and 27b, the synthesis calculation unit 29, the compression calculation units 33a and 33b, and the expansion calculation units 36a and 36b is realized by a single microprocessor, a DSP, or hardware having equivalent functions. .

The correspondence between the invention described in claim 1 and the second embodiment has the same correspondence as the correspondence between the invention described in claim 1 and the first embodiment. . Next, the operation of the second embodiment will be described with reference to FIGS. FIG. 9 is a diagram (1) illustrating the image processing according to the second embodiment, and FIG. 10 is a diagram (2) illustrating the image processing according to the second embodiment.

The light image transmitted through the taking lens 22 enters the prism 31. The light image is divided into two directions of reflected light and transmitted light by the reflection / transmission film inside the prism 31. Since a color filter array having a Bayer array is attached to the image sensor 24a, the output is such that G signals (indicated by a circle in the figure) are arranged in a checkered pattern as shown in FIG. The B signals are further arranged in a checkered pattern.

As the output of the image sensor 24a, either an R signal or a B signal is obtained for each scanning line. Therefore, A
In the / D converter 25a, the output of the image sensor 24a is separated into RGB signals by sampling each pixel while determining the R signal and the B signal for each scanning line. The RGB signals are stored and stored in the frame memory 32a for each signal, and an R image composed of an R signal, a G image composed of a G signal, and a B image composed of a B signal are generated.

Each of the R, G, and B images is compressed by a compression operation unit 33a.
Are compressed and encoded and stored again as compressed image data in the frame memory 32a. Further, R,
The G and B compressed image data are transferred to the decompression operation unit 36a via the connection terminal 34a and the frame memory 35a, and are decompressed and decoded by the decompression operation unit 36a.

Each of the expanded image data is supplied to an interpolation operation unit 2
Pixel interpolation is performed at 7a. The result is shown in FIG. 9C. In the figure, the mark ○ indicates the pixel output of the G signal of the image sensor 24a, and the mark □ indicates the pixel interpolated based on the pixel output. In addition, as an interpolation method, for example, a pixel output (○
The pixel interpolation is performed on a line (line a) having a mark (mark), and the pixel interpolation is performed on a line (line b) having no pixel output. Specifically, the pixel output of the mark “印” is internally divided, the pixels of the mark “□” on the same line are obtained, and the pixel interpolation of the line “a” is performed. Next, upper and lower average interpolation is applied to each pixel of the line a of the two columns in which the pixels are interpolated, thereby obtaining each pixel of the line b.

On the other hand, the same processing is performed for the pixel output of the G signal of the image pickup device 24b (marked by ● in FIG. 9B), and the pixel indicated by × is obtained by using the pixel output of mark ●, as shown in FIG. 9D. Create an interpolated image. Since the two images of FIGS. 9C and 9D in which the pixels are interpolated have the same spatial phase, the combining operation unit 29 sequentially adds the corresponding pixels (for example, adds the pixel A and the pixel B in the figure). To generate a pixel C) to create a composite image of the G signal (FIG. 9E).

The composite image of the G signal is stored in the frame memory 30.
Are sequentially stored and stored. The above-described pixel interpolation processing and addition processing are performed not only on G signal pixels but also on R and B signal pixels. The interpolation processing and the addition processing for the pixels of the R signal will be described with reference to FIG. The A / D converter 25a separates the pixel output of the R signal from the pixel output (FIG. 10A) of the image sensor 24a, and generates an R image. The interpolation calculation unit 27a performs pixel interpolation on the R image. The result is shown in FIG. 10C. “R” in FIG. 10C is a pixel output of the R signal, and a square indicates a pixel interpolated based on the pixel output.

As an interpolation method, for example, "R
Pixel interpolation is performed for a line having a pixel output of “line” (line a), and then pixel interpolation is performed for a line having no pixel output of “R” (line b). The output is internally divided, and the pixels marked with □ on the same line are interpolated to perform pixel interpolation on line a. The line b is interpolated using the pixel a as it is.

On the other hand, the interpolation operation unit 27b
Interpolation processing is similarly performed on the pixel output b (FIG. 10B). The result is shown in FIG. 10D. “R” in FIG. 10D is
This is the pixel output of the R signal from the image sensor 24b, and the crosses are pixels interpolated based on the pixel output. Since the two images of FIG. 10C and D in which the pixels are interpolated have the same spatial phase, the combining operation unit 29 sequentially adds the corresponding pixels (for example, adds the pixel D and the pixel E in the figure). ) To create a composite image of the R signal.

The composite image of the R signal is sequentially stored in the frame memory 30. Note that the B signal is also subjected to interpolation processing and synthesis processing in the same manner as the R signal, and a synthesized image of the B signal is created. As described above, in the imaging apparatus according to the second embodiment, the interpolation processing is performed before creating the combined images of RGB. At this time, the interpolation process is performed only with the pixel output of one image sensor, and the pixel output of another image sensor is not used. Therefore, it is possible to prevent the generation of a false signal at the edge portion in all of the RGB images, which is more suitable for capturing a color image for creating an RGB image.

In the second embodiment, the compression operation unit 3
3a and 33b and the decompression operation units 36a and 36b, images can be effectively stored and accumulated in the frame memory. Note that, as in the second embodiment, the configuration in which the compression and decompression are performed and then the interpolation calculation and the combination calculation are performed is the first configuration.
Naturally, the present invention can also be applied to a single-color image pickup device as in the embodiment.

Further, in the second embodiment, the connection terminals 3
4a and 34b, it can be separated into a camera unit and an image processing unit. In particular, image processing and the like may be performed by an external device such as a personal computer.

(Third Embodiment) In the third embodiment,
This is an imaging device in which the same color pixels are arranged differently on a plate. The feature of the configuration is that only a color filter to be attached to the imaging device 24a in FIG. 8 is different.
This is the same as the imaging device of the embodiment. A green color filter is attached to the imaging surface of the imaging element 24a. FIG. 11A shows the pixel output at this time. It should be noted that in the figure, a circle indicates the pixel output of the G signal.

The interpolation operation section 27a performs an interpolation process on this image. The interpolation process has been described in the first embodiment, and a description thereof will be omitted.
The result of the interpolation process is shown in FIG. In the figure, the squares are pixels interpolated based on the pixel outputs of the circles. On the other hand, a color filter array having a Bayer array is attached to the image sensor 24b, and FIG. 11B shows a pixel output at this time. In the drawing, the mark ● indicates the pixel output of the G signal.

The A / D converter 25b separates only the G signal to generate a G image, and the interpolation calculator 27b performs pixel interpolation. Note that the description of the interpolation processing has been described in the second embodiment, and a description thereof will be omitted here. FIG. 11D shows the result of the interpolation processing. Note that in the figure, the crosses are pixels interpolated based on the pixel outputs of the black circles.

Since the two G images of FIGS. 11C and 11D in which the pixels have been interpolated have the same spatial phase with each other,
No. 9 sequentially adds corresponding pixels (for example, adds a pixel A and a pixel B in the figure to generate a pixel C) and creates a composite image (FIG. 11E). The effects of the third embodiment are the same as those of the first and second embodiments.

In this embodiment, the image sensors are vertically and horizontally shifted from each other by 1/2 pixel. However, the present invention is not limited to a strict 1/2 pixel shift as long as error correction can be performed. Further, in the present embodiment, the description has been given of the two-chip type imaging device. However, the present invention is not limited thereto, and the present invention can also be applied to an imaging device which performs imaging by performing a 1/3 pixel shift in a three-plate type imaging device. .

The interpolation processing performed in this embodiment is an example, and the present invention is not limited to this. Further, the pixel shifting direction is also shifted in the vertical and horizontal directions, but is not limited thereto.
Direction. Further, the light splitting means is not limited to the prism of the present embodiment, and its shape and structure are not limited as long as the optical system has the characteristics of a half mirror.

Further, in the second and third embodiments, a color filter array having a Bayer array is used to capture a color image with a single plate. However, the present invention is not limited to this. A filter array may be used.

[0056]

According to the first aspect of the present invention, the pixel interpolation of the image information is performed only by the pixels of the image information. Therefore, since pixel interpolation is performed without using the pixel output of another image sensor, generation of a false signal at an edge portion can be avoided, and an image without mosaic distortion can be obtained.

Further, since all the pixels constituting the composite image are generated as a result of the addition processing, random noise such as thermal noise can be more reliably reduced at the time of pixel addition, and the S / N can be improved. Can be achieved. As described above, in the imaging apparatus to which the present invention is applied, the generation itself of the false signal at the edge portion can be prevented, so that the configuration for removing the false signal can be omitted, and a high-quality and high-definition imaging apparatus can be realized. can do.

[Brief description of the drawings]

FIG. 1 is a principle block diagram of the invention according to claim 1;

FIG. 2 is a diagram for explaining the operation of the invention described in claim 1;

FIG. 3 is a diagram illustrating a configuration of a first embodiment.

FIG. 4 is a flowchart illustrating the operation of the first embodiment.

FIG. 5 is a diagram (1) for explaining image processing according to the first embodiment;
It is.

FIG. 6 is a diagram (2) illustrating image processing according to the first embodiment;
It is.

FIG. 7 is a diagram illustrating an interpolation process.

FIG. 8 is a diagram illustrating a configuration of a second embodiment.

FIG. 9 is a view for explaining image processing according to the second embodiment (1).
It is.

FIG. 10 is a diagram (2) illustrating image processing according to the second embodiment.

FIG. 11 is a diagram illustrating image processing according to a third embodiment.

FIG. 12 is a diagram illustrating a conventional imaging device.

FIG. 13 is a diagram illustrating a spatial pixel shifting method.

FIG. 14 is a diagram (1) illustrating a problem;

FIG. 15 is a diagram (2) illustrating a problem.

[Explanation of symbols]

Reference Signs List 1 imaging means 2 interpolation means 3 image synthesizing means 21 imaging device 22, 51 imaging lens 23, 31 prism 24a, 24b imaging element 25a, 25b A / D conversion unit 26a, 26b, 28a, 28b, 30, 32a, 32
b, 35a, 35b Frame memory 27a, 27b Interpolation operation unit 29 Compositing operation unit 33a, 33b Compression operation unit 34a, 34b Connection terminal 36a, 36b Decompression operation unit 52 Dichroic prism 52a Color filter array 53a Image sensor for RB 53b, 53c G Image sensor 54 signal processing circuit

Claims (1)

[Claims] 1. An optical system according to claim 1, wherein the light images divided and formed on the respective light receiving surfaces are individually photoelectrically converted through an imaging optical system and a light splitting unit to generate image information, and mutually reciprocate in a direction along the light receiving surfaces. An image pickup apparatus having a plurality of image pickup units arranged in a shifted manner and performing image pickup by a spatial pixel shift method, wherein an interpolation unit that interpolates pixels of image information generated by each of the image pickup units and matches a spatial phase between the image information. And an image synthesizing means for adding a pixel output corresponding to each piece of image information interpolated by the interpolation means to create a synthesized image.