JPH06269010A - Color separation optical system, image pickup method and image pickup device - Google Patents

Abstract

PURPOSE:To obtain a picture signal with high resolution for high vision application by using an image sensor used widely at present. CONSTITUTION:For example, a half mirror 16 is provided to a G prism 14 and lights G1, G2 separated from a light G are made respectively incident image sensors DG1, DG2 comprising a PAL system CCD. A 16:9 image area including 1/2 horizontal scanning lines by the High Vision system, for example is set to the image sensors DG1, DG2 and they are arranged while being deviated in the vertical direction. Then the signals obtained from the two sensors are synthesized through scanning conversion, a horizontal scanning line number corresponding to that of the High Vision system is obtained and the deterioration in the resolution is prevented, then an overlapped part of an opening of adjacent horizontal picture element array when deviated in the vertical direction is masked.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a color separation optical system, an image pickup method, and an image pickup apparatus, and uses a conventional image sensor such as PAL to obtain a higher resolution than conventional such as HDTV. The present invention relates to improvements in a color separation optical system suitable for obtaining image information, an imaging method, and an imaging device.

[0002]

[Prior Art and Problems to be Solved by the Invention] Current N
On a TSC television, the screen display has an aspect ratio of 4: 3. However, recently, a horizontally long screen represented by a movie or the like, for example, a screen having an aspect ratio of 16: 9 such as high definition has been desired from the viewpoint of power. The HDTV standard is as shown in Table 1 below, and the screen definition is required to be 3 to 5 times higher than that of the conventional NTSC system.

[0003]

[Table 1]

As shown in Table 1, for example, the number of horizontal scanning lines is 525 in the NTSC system and 625 in the PAL system, while in the high-definition system, it is about twice as many as 1 in the horizontal line.
The number is 125.

By the way, such a high pixel density CCD which is an image sensor for high-definition is very expensive even with today's technology. In other words, because of the high resolution, high signal processing speed is required, power consumption is high, and peripheral devices used are also technically extremely sophisticated, which greatly affects the price. There is. Therefore, it is considered that it will take a considerable amount of time before it is widely put into practical use as an imaging means for high definition.

The present invention focuses on these points,
An object of the present invention is to provide a color separation optical system, an image pickup method, and an image pickup apparatus capable of obtaining a high-resolution image signal for high-definition by using an image sensor which is widely used at present.

[0007]

In order to achieve the above-mentioned object, the color separation optical system of the present invention has a first high resolution.
Corresponding to a second method (for example, PAL method) having a relatively low resolution different from the above method (for example, high-definition), and the first method using a horizontal pixel row having half the number of horizontal scanning lines in the first method. The image area having the aspect ratio of the method can be set, and the first and second image sensors in which the overlapping portions of the openings of the adjacent horizontal pixel rows when the reading area is shifted in the vertical direction are masked, Incident light splitting means for splitting the light is provided, and the image sensor is arranged with the read regions being shifted in the vertical direction on the split light optical axis of the incident light splitting means.

According to a second aspect of the present invention, in the first and second image sensors, an overlapping portion of openings of adjacent vertical pixel columns when the reading area is shifted in the horizontal direction is masked, and the divided light beam is divided. It is characterized in that the reading areas are arranged so as to be displaced in the horizontal direction on the axis. According to a third aspect of the present invention, at least a green image is picked up by the color separation optical system, scan conversion of the image signals read from the first and second image sensors is performed, and the vertical scanning of the first method is performed. An image pickup method is characterized in that a green image signal having a resolution is obtained.

A fourth aspect of the present invention is a read control means for simultaneously driving the first and second image sensors of the color separation optical system to read out a signal at a frequency half the horizontal scanning frequency of the first method. And memory means for temporarily storing these read signals and compressing and outputting in the time axis direction corresponding to the horizontal scanning frequency of the second method, and the signal output from the memory means for horizontal scanning of the first method. And an output control means for performing scan conversion so that the order corresponds to. In a fifth aspect of the present invention, the image pickup device includes image signal correction / interpolation means for correcting or interpolating an image signal of another color by using the green image signal output from the memory means. It is characterized by

[0010]

According to the present invention, for example, a PAL CCD is used. On this image sensor, for example, a 16: 9 image area including 1/2 horizontal scanning lines of a high-definition system is set, and reading areas are arranged in a vertically shifted manner. Light from the imaging target is split by a half mirror or the like and is incident on each image sensor. By scanning-converting and synthesizing the signals obtained from the two sensors, the number of horizontal scanning lines corresponding to high-definition can be obtained. In order to prevent the resolution from being lowered, the overlapping portion of the openings of the adjacent horizontal pixel columns when shifted in the vertical direction is masked.

[0011]

Embodiments of a color separation optical system, an image pickup method, and an image pickup apparatus according to the present invention will be described below in detail with reference to the accompanying drawings. <Color Separation Optical System of this Example> First, the configuration of the color separation optical system of this example will be described. FIG. 1 shows the configuration of the color separation optical system in this embodiment. In the figure, the light-incident side is B for extracting the B (blue) light component.
The prism 10 is arranged. An R prism 12 for extracting an R (red) light component is arranged on the light output side of the B prism 10. Further, a G prism 14 for extracting a G (green) light component is arranged on the light output side of the R prism 12. A minute air layer (not shown) is provided between the B prism 10 and the R prism 12. These prism configurations are similar to those of an image pickup device having a general three-plate configuration.

In this embodiment, the G prism 14 is further provided with a half mirror 16. That is, the G prism 14 includes the G first prism 16A and the G second prism 16A.
B and a half mirror 16 is provided on the joint surface thereof. Also, the half mirror 16
Is provided at an angle of 45 ° with respect to the incident optical axis as shown in FIG. 1, the image forming planes of the image sensors DG1 and DG2 are at an angle of 90 °, so that the prism fabrication and the mounting of the image sensor are accurate. Can be done well. In addition,
A half mirror 16 may be provided as shown by a dotted line in FIG. 1 and the image sensor DG1 may be arranged on the image sensor DR side.

Next, the operation of the above color separation optical system will be described with reference to FIG. First, as shown in FIG. 7A, the light in the B wavelength band included in the incident light incident from the imaging optical system (not shown) is dichroic of the B prism 10 that selectively reflects the B light. It is reflected by the film 10A. The reflected B light is further totally reflected by the incident surface 10B of the B prism 10 and is incident on the image forming surface of the B image sensor DB through the B trimming filter 10C to form an image.

Next, the R light contained in the incident light is transmitted through the dichroic film 10A of the B prism 10 and enters the R prism 12 as shown in FIG. The R prism 12 that selectively reflects the light in the R wavelength band
Is reflected by the dichroic film 12A. The reflected R light is totally reflected by the incident surface 12B of the R prism 12 that is in contact with the B prism 10 through the air layer, and is incident on the image forming surface of the R image sensor DR through the R trimming filter 12C. To do.

Further, the G light included in the incident light is transmitted through the dichroic film 12A of the R prism 12 and is incident on the G first prism 14 as shown in FIG. The above operation is similar to that of the conventional R, G, B three-color separation optical system. The G light incident on the G first prism 16A further enters the half mirror 16, and the half mirror 1
The G light reflected by 6 is the G trimming filter 14A.
It is emitted to the image formation plane of one element DG1 of the two G image sensors DG1 and DG2 via. Then, the G light transmitted through the half mirror 16 passes through the G trimming filter 14B and the other image sensor DG2.
The incident image is formed on the image plane of.

As described above, the color separation optical system of this embodiment is similar to the general one for B and R, but is divided into two for G and is incident on two image sensors. It has a structure of four plates as a whole. Since the image formed by the image sensor DG1 is a reflected image by the half mirror 16, the left and right are inverted with respect to the image formed by the other image sensors DB, DR, DG2. Therefore, with respect to the image sensor DG1, the right-and-left reverse reading from the CCD or the right-and-left reverse reading by the line memory or the frame memory is performed to obtain a normal image.

<Setting of high-definition image area> Next, the setting of an image area corresponding to a high-definition aspect ratio of 16: 9 on each image sensor will be described. Image sensor DB, DR, DG1, DG
2 is a CCD element for a PAL system, which is generally provided and has an image size of 1/3 inch, and which has been subjected to camera shake correction. Since the image stabilization is performed, the number of horizontal pixels (pixels) and vertical pixels is increased by about 25% in the entire area of the image area.

This is shown in FIG. 3, and the size of the image area is 3.6 mm in the vertical direction and 4.9 mm in the horizontal direction. And the pixel is 7 in the vertical direction.
26 pixels (hence 726 lines), horizontal 8
It has 58 pixels. Further, the prescribed area WS of the PAL system shown by diagonal lines has 575 pixels in the vertical direction and 669 pixels in the horizontal direction. That is, even if the image forming area on the image sensor moves up and down, left and right due to camera shake,
The image stabilization is performed by also moving the specified area WS from which the image signal is taken out in the moving direction.

Next, in this embodiment, a commercially available image sensor of such a PAL system is used for high-definition, and the number of effective scanning lines 103 per frame of high-definition is 103.
Since it is impossible to secure five, 1/2 of that,
That is, the number of horizontal effective scanning lines of 517.5 is secured. Then, the number of pixels in the horizontal direction is obtained from the pixels in the vertical direction 518 (517.5 is rounded up to 518), the dimensions of the image sensor shown in FIG. 3, and the aspect ratio of high-definition horizontal: vertical = 16: 9. And becomes 808 pixels.

More specifically, when the dimension between each pixel is obtained from the number and dimensions of pixels in the vertical and horizontal directions of the image sensor shown in FIG. 3, the vertical direction is 5 μm and the horizontal direction is 5. It becomes 7 μm. As described above, since there are 518 pixels in the vertical direction, the total size is 518 × 5 μm =
It becomes 2.59 mm. Therefore, the horizontal dimension with an aspect ratio of 16: 9 is 2.59 × (16/9) = 4.6.
mm, and the number of pixels included in this dimension is 4.6 mm
/5.7 μm = 808.

In FIG. 5, the prescribed region WW having the aspect ratio 16: 9 of high-definition thus obtained is shown by hatching. The position of the high-definition definition area WW in the horizontal and vertical directions on the image area area may be set appropriately. In the illustrated example, the high-definition prescribed area WW is set in the center of the image area area in the vertical direction and on the left side in the horizontal direction.

The image sensor DB, DR, shown in FIG.
The above 16: 9 defined area WW is set in DG1 and DG2. Then, in the present embodiment, the charge signal reading is performed on this high definition region WW. As for the method of sweeping away the charge signal in an unnecessary portion other than the specified area WW and the signal reading method in the specified area WW, there is, for example, the one filed as Japanese Patent Application No. 3-329942.

<Method for Obtaining Resolution for High-Definition in Vertical Direction> Next, a method for obtaining resolution corresponding to high-definition in the vertical direction will be described. The vertical resolution of the high-definition system is NT as described above.
It is about double that of the SC and PAL systems. However, as shown in FIG. 5, the number of horizontal effective scanning lines in the high-definition region WW of the image sensor in this embodiment is 1/2 that of high-definition.

Therefore, the simplest method for obtaining the number of horizontal effective scanning lines corresponding to high-definition is to perform pixel scanning interpolation processing within the image area of the prescribed area WW to obtain 1,135 horizontal effective scanning lines for high-definition. There is something to be gained. However, in this scanning interpolation method, the image information of the scanning lines adjacent in the vertical direction is the same, and the image resolution in the vertical direction is only 518 pixels, which is 1/2 of high definition.

Therefore, instead of such a scanning interpolation method, as described above, the two-plate (piece) image sensors DG1 and DG are used.
It is possible to use a combination of two and obtain a high-definition signal with 1035 horizontal effective scanning lines. In Figure 6,
The portions of the two image sensors DG1 and DG2 described above are shown enlarged. The light of G is incident on the half mirror 16 as shown by the arrow FA and is divided into two, and is incident on the image sensors DG1 and DG2, respectively. Here, as shown in FIG. 7A, the image sensors DG1 and DG2 are exactly the same light receiving position with respect to the incident optical axis, that is, the image sensors DG1 and D2 from the incident position through the half mirror 16.
When viewing G2, they are not arranged so that they overlap at all, but as shown in FIG.
It is arranged to be shifted by the pixel.

FIG. 8 is a plan view showing a part (part of FIG. 7B) of the pixel array of the image sensors DG1 and DG2 having such an arrangement. In the figure, n1 + m indicates the number of horizontal lines of the image sensor DG1, and n1 + 1
The area from n1 to 518 is the specified area WW for high-definition television shown in FIG. Further, n2 + m represents the number of horizontal lines of the image sensor DG2, and the part from n2 + 1 to n2 + 518 is the specified area WW for high-definition television shown in FIG. It should be noted that since both of them are arranged vertically offset by one pixel, n2 = n1 + 1.

In the high-definition system, as shown in Table 1 above, the interlace is 2: 1. Therefore, a total of 1036 image signals from the two image sensors DG1 and DG2 in the first and second fields 518 respectively. Shall be obtained. The two image sensors DG1 and DG2 are simultaneously driven together with the other image sensors DB and DR.
That is, when the charge signal reading of the (n1 + m) th line of the image sensor DG1 is performed, the image sensor DG
The charge signal reading of the second n2 + m-1 line is also performed at the same time.

The read signal of the arrow shown by the solid line constitutes the first field, and the read signal of the arrow shown by the dotted line constitutes the second field. That is, the n1 + 1th line of the image sensor DG1 corresponds to the first scanning line of the high-definition image, the n2 + 2th line of the image sensor DG2 corresponds to the second scanning line of the high-definition image, and so on. This relationship is summarized in Table 2 below.
In this way, the read signals are alternately taken out from the image sensors DG1 and DG2 to form the first and second fields.

[0029]

[Table 2]

FIG. 9 shows a time chart of the signal reading process. The image sensors DG1 and DG2 are arranged vertically offset by one pixel as described above, and are simultaneously driven. Therefore, FIG.
As shown in (B), n1 + 1 line and n2 line, n1
+3 line and n2 +2 line, n1 +5 line and n2 +4
Lines, and so on, are read simultaneously by the interlaced method. The read signal of each of these lines is read at a frequency of 1/2 of the horizontal synchronizing frequency fH of high-definition and stored in a line memory (not shown). Then, thereafter, the signal is read at a speed twice as high as that at the time of writing, that is, at the horizontal synchronizing frequency fH of high-definition and becomes a serial signal (see FIG. 7C). In this way, the high-definition image signal of the first field is obtained. The same applies to the second field.

However, in such a method, although the image sensors DG1 and DG2 are arranged shifted by one pixel in the direction perpendicular to the incident optical axis, the optical image receiving portion of each line is the first field and the second field. It is common in the field. For example, the light receiving image of the n1 + 1 line of the image sensor DG1 and the light receiving image of the n2 + 1 line of the image sensor DG2 are equal. That is, the information of the first scanning line in the first field and the information of the 519th scanning line in the second field are equal. The same applies to the other lines. Therefore, this method has the same result in terms of vertical resolving power as the method of obtaining the high-definition image signal by the above-described scanning interpolation, and only obtains the resolving power corresponding to 518 pixels included in the high-definition prescribed area WW of the image sensors DG1 and DG2. I can't.

As an improved method for such a method, for example, the image sensor DG2 is replaced by the image sensor DG.
A method of shifting by half a pixel in the direction perpendicular to 1 may be considered. FIG. 10 shows pixels of the image sensors DG1 and DG2 arranged in this way. According to such a method, although the vertical resolution can be expected to be improved, it cannot be said to be a preferable method because the line intervals are not uniform.

Therefore, in this embodiment, as shown in FIG. 11, an optical mask process is applied to a part of each pixel of the image sensors DG1 and DG2. For the pixels on a certain line, the lower half of the opening is masked, and for the pixels on the next line, the upper half of the opening is masked. Such masking is alternately repeated. Such masking is performed by the image sensor DG
1 and DG2 are performed in exactly the same manner, but as described above, they are arranged so as to be vertically displaced by one line on the optical axis, and as a result, masking alternates.

Specifically, the image sensor DG
In the n1 + 1 line of 1, the lower half of the pixel opening is masked. On the n1 + 2 line, on the contrary, substantially the upper half of the pixel opening is masked. In the same manner, the lower half of the n1 + 3 line, the upper half of the n1 + 4 line, and so on.

On the other hand, in the image sensor DG2,
The masking is reversed because the lines are arranged vertically offset from each other. That is, in the n2 + 1 line, the upper half of the pixel opening is masked. n2 + 2
On the contrary, in the line, the lower half of the opening of the pixel is masked. In the same manner, the upper half of the n2 + 3 line, the lower half of the n2 + 4 line, and so on.

If the masking portion in each pixel is narrowed and the opening is enlarged, the light receiving area is increased and the sensitivity is improved, but the image sensors DG1 and DG2.
Therefore, the pixel openings of the corresponding lines are overlapped with each other, and the vertical resolution (vertical image resolution) is reduced. Therefore, masking is performed by the image sensor DG1,
So that the combined openings of DG2 do not overlap each other,
Moreover, the area of the opening is set to be the largest.

When the optical mask is applied in this way, the composite of the pixels of each line becomes an array similar to the vertical pixel array of the image sensor having the number of pixels corresponding to the high-definition system. As a result, the pixel information of adjacent lines when forming a high-definition image becomes different, and as a result, a high-definition resolution of 1035 lines, which is twice 518, can be obtained.

The signal reading of each line of the image sensors DG1 and DG2 is as described above. That is, since the two image sensors DG1 and DG2 are driven simultaneously at the same time, when the charge signal of the (n1 + m) th line of the image sensor DG1 is read out, the image sensor DG
The charge signal reading of the second n2 + m-1 line is also performed at the same time.

The read signal of the arrow shown by the solid line in FIG. 11 constitutes the first field, and the read signal of the arrow shown by the dotted line constitutes the second field. That is, the (n1 + 1) th line of the image sensor DG1 is the first scan line of the high-definition image and the (n) th line of the image sensor DG2 is
The 2 + 2 line corresponds to the second scanning line of the high-definition image, and so on. This relationship is as shown in Table 2 above. In this way, the read signals are alternately taken out from the image sensors DG1 and DG2 to form the first and second fields.

As shown in FIGS. 9A and 9B, n1 +
One line and n2 line, n1 + 3 line and n2 + 2 line, n1 + 5 line and n2 + 4 line, and so on are read simultaneously by the interlaced method. The read signal of each of these lines is read at a frequency of 1/2 of the high-definition horizontal sync frequency fH and stored in the line memory, and at the same speed as that at the time of writing, ie, at the high-definition horizontal sync frequency fH. The signal is read out and becomes a serial signal (see FIG. 7C). In this way, the high-definition image signal of the first field is obtained. The same applies to the second field.

In this case, in this embodiment, as shown in FIG.
Since the masking is performed as described above, the optical image receiving portion of each line is different between the first field and the second field. For example, n1 of the image sensor DG1
Due to masking, the +1 line (first scanning line) received light image and the n2 + 1 line (519th scanning line) received image of the image sensor DG2 adjacent thereto are different from each other due to masking. That is, the information of the first scanning line in the first field and the information of the 519th scanning line in the second field are different. The same applies to the other lines.

FIG. 12A shows a state in which the apertures of the pixels in the specified region WW of the image sensors DG1 and DG2 as described above are combined on the optical axis to correspond to the high-definition scanning line. Has been done. In the figure, the circled pixels are the first
The image of the field is formed, and the pixels of the square marks form the image of the second field. In addition, white marks indicate pixels of the image sensor DG1, and black marks indicate pixels of the image sensor DG2. Further, the numbers in the figure are scanning line numbers in high definition.

For B and R, the signals read from the image sensors DB and DR are modified to obtain the number of scanning lines corresponding to high definition. FIG.
Shows the situation, and signal reading is performed every other line based on the interlacing method as shown in FIG. Then, as shown in FIG.
The read signals are read out from the memory at high speed, and the correction data H (n + m) is added. The read signal may be repeatedly interpolated. This will be described in detail later.

As described above, according to this embodiment, two commercially available PAL image sensors with similar masking are used for G, which corresponds to high-definition 1
An image signal having a resolution of 035 horizontal effective scanning lines can be obtained.

<Method for Improving Horizontal Resolution> Next, a method for improving the horizontal resolution will be described. In the high-definition standard, as shown in Table 1, the number of pixels in the horizontal direction is not specified. Therefore, in the horizontal direction, basically, the number of horizontal pixels 808 included in the defined area WW shown in FIG. However, since there is a high-definition system for the purpose of obtaining a higher definition image than the current NTSC system, it is preferable that the resolution is higher in the horizontal direction, that is, the number of pixels is larger.

The easiest way to increase the resolution in the horizontal direction is to arrange the image sensors DG1 and DG2 by shifting them by a half pixel pitch in the horizontal direction. However, in this method, the apertures of the pixels of the image sensor DG1 and the apertures of the pixels of the image sensor DG2 that are located on the same line partially overlap with each other, and it is impossible to expect an effective improvement in resolution.

Therefore, in the present embodiment, the optical mask processing is performed also in the horizontal direction so that the overlapping of the openings does not occur. This is shown in FIG. The image sensor DG2 is arranged horizontally offset from the image sensor DG1 by (1/2) P which is half the pixel pitch P on the optical axis. In the figure, the shaded area SA is a masking area for increasing the resolution in the vertical direction described above.

Further, in this embodiment, the image sensor DG1
The pixel PS and the pixel PS of the image sensor DG2 are masked in the overlapping portion in the horizontal direction, and are shown by the shaded area SB. In addition, horizontal masking S
From the standpoint of sensitivity, B is preferably applied so that the openings between the pixels of the image sensors DG1 and DG2 do not overlap and the area of the openings is the largest.

When such an image sensor arrangement and horizontal masking are performed, the pixel position of the image sensor DG2 is shifted by 1/2 pixel pitch in the horizontal direction. That is, in each pixel array on the high-definition scanning line shown in FIG.
The G2 pixel, that is, the black pixel, is ½ in the horizontal direction.
The pixel pitch shifts as shown in FIG. Thus, for example, the pixel opening on the first scan line in the first field and the 51st pixel in the adjacent second field
The pixel openings on the 9th scanning line are located between pixels, and when viewed in the image on the high-definition screen space,
Those pixels are combined. As a result, in the specified area WW of the image sensors DG1 and DG2, the horizontal movement of 80
Although it has 8 pixels, a horizontal resolution equivalent to twice that on a high-definition image can be obtained.

<Reading out and converting charge signal> Next, the image sensors DB, DR, DG1 and DG2 as described above.
The charge signal reading from the digital camera and the conversion processing of those into a high-definition image signal will be further described. In FIG.
An example of a signal read processing circuit is shown. In addition, B, R
Have the same configuration, only one is shown. In the figure, the above-mentioned image sensors DG1, DG2, DB
The (DR) is provided with a read control unit 50 that controls the signal reading by driving them at the same time. The charge signal output sides of the image sensors DG1, DG2, DB (DR) are connected to preamplifiers (shown by PA) 52, 54, 56, A.
It is connected to the line memories 64, 66 and 68 via the / D converters 58, 60 and 62, respectively.

Of these, the line memories 64,
The output side of 66 is connected to the line memories 70 and 72, respectively, and the output side of the line memory 68 is connected to the line memories 74 and 76, respectively. The output sides of the line memories 70 and 72 are connected to the switching input side of the scan conversion changeover switch 78, and the output sides of the line memories 74 and 76 are changed over to the data interpolation processing changeover switch 80. Each is connected to the input side.
These line memories 64-76, changeover switches 78, 8
The operation control of 0 is performed by the output control unit 82.

Of these, the signal read from the image sensor DB (DR) by the read control unit 50 includes one horizontal pixel column reading method performed every other line and two horizontal pixel column reading method reading every two lines. . Either of them may be used, but in this example, the one horizontal pixel column readout method is used for easy understanding.

FIG. 16 shows the sequence of signal reading from the image sensors DG1, DG2, DB, DR by the one horizontal pixel column reading method. The numbers in the figure are the numbers of the scanning lines, but for DG1 and DG2, they show the order of the scanning lines in high-definition. As shown in this figure, for B and R, signal reading is performed every other line. The reading of signals from the image sensors DG1 and DG2 by the reading control unit 50 is as shown in FIG. In the figure, the signal read out as indicated by the solid line arrow constitutes the first field, and the signal read out as indicated by the dotted line arrow constitutes the second field. It should be noted that whether or not the configuration for increasing the resolution in the horizontal direction shown in FIG. 14 may be appropriately determined as necessary.

Image sensors DG1, DG2, DB (D
R) from scan reading to line memories 64, 66, 6
The frequency until the signal is stored for 8 is 1/2 (16.875 KHz) of the frequency fH in the case of the high-definition system. As described above, the charge signal read scanning frequency of the PAL type CCD is 15.625 KHz, but since this value is approximately the same as fH / 2, it can be used without taking any special measures. However, the line memories 70 and 7
After reading the signals from 2, 74, and 76, the frequency is fH in the case of the high-definition system.
The charge reading scanning method in this example is the same as the normal method except for the reading frequency.

Next, the line memories 64, 66 and 68 are
For storing the signals read from the image sensors DG1, DG2, DB (DR), the line memories 70, 72,
74 and 76 are output in parallel at high speed. Then, the line memories 70, 72, 74, 7
The signal stored in 6 is output at a high-definition frequency, that is, at a speed twice as fast as when read from the image sensor.

Next, the changeover switch 78 selectively and alternately outputs the signals stored in the line memories 70 and 72 in the order of high-definition scanning shown in FIG. 16 to perform scanning conversion. It is a thing. The changeover switch 80 is for alternately outputting the signals stored in the line memories 74 and 76 to interpolate the signal data. The operation of the line memory and the changeover switch is controlled by the output control unit 82.

Next, the operation of this apparatus will be described with reference to FIG. Image sensor DG1, DG2, D
From B (DR), the read control unit 50 performs signal reading by interlaced scanning at a frequency of 1/2 of the horizontal scanning frequency fH of high-definition, or in other words, a cycle of 2 / fH. These read signals are shown in FIGS. 17 (A) to (D). At the time of reading this signal, the selection of the signal is performed by utilizing the method patented as Japanese Patent Application No. 3-329942.

First, the charge signal of the pixel line of the first line is read from the image sensor DB (DR). This is B1 in FIG. 17 (A) and R1 in FIG. 17 (B). On the other hand, as shown in FIG. 16, the charge signal of the pixel line on the first line is read from the image sensor DG1.
This is G11 in FIG. 17 (C). Image sensor D
From G2, the charge signal of the pixel column on the line immediately above the first line (not shown) shown in FIG. 16 is read (see FIG. 11). This is G20 in FIG. 17 (D). These signals G11, G20, B1 (R1) are supplied to the preamplifier 5
2, 54, 56 amplification, A / D converters 58, 60,
After A / D conversion by 62, line memories 64, 66,
68, respectively. Further, these data are written in parallel to the line memories 70, 72, 74 and 76 at high speed.

At the next time, this time the image sensor DG
Signals B3, R3, G13, G22 of the third line are obtained by interlaced scanning in 1, DG2, DB (DR), and similarly stored in the line memories 64, 66, 68, respectively. At this time, the signal B1 (R1) is alternately read from the line memories 70 and 72 at twice the speed at the time of storage, that is, at the horizontal scanning cycle of high-definition. As a result, a signal having twice the horizontal scanning cycle of high-definition is time-compressed and interpolated into the horizontal scanning cycle of high-definition (see FIGS. 17E and 17F). Similarly, the signals G11 and G20 are read from the line memories 70 and 72 in the horizontal scanning period of high-definition, but the output control unit 82 controls the changeover switch 78 to first output G2.
0, and then G11 is read (see (G) in the figure).

The above-described operation is repeated, and B, R, and G image signals compatible with high-definition are obtained as shown in FIGS. In addition, since the same signal is repeated for B and R, the actual resolution is 1/2 of that of high definition. The first signal G20 in FIG. 7E is unnecessary and is discarded. As described above, according to the apparatus of FIG. 15, the number of vertical scanning lines is the same as that of high-definition for R, G, and B. The vertical resolution of G is high-definition, and the resolution of B and R is 1/2 of that. The image signal of

As described above, the image sensor DG
Since the image formation of 1 is a reflection image by the half mirror 16,
The left and right are inverted with respect to the image formation of the other image sensors DB, DR and DG2. For this reason, when the image sensor DG1 that is capable of left-right inversion reading is used,
The read control unit 50 is configured to perform left-right inverted read. Alternatively, the line memory 64 by the output control unit 82
Alternatively, the left-right inversion reading may be performed when the signal is output from 70. The same applies to the device of FIG. 18 described later.

Next, referring to FIG. 18 and FIG.
Another signal read conversion processing device will be described. In the device of FIG. 15 described above, the same signal is continuously repeated twice for B and R, but in this example, the data of G is corrected using the signal of G. The same components as those in FIG. 15 are designated by the same reference numerals.

In FIG. 18, the image sensor DG1,
DSPs 84 and 86 are connected to the output sides of the A / D converters 58 and 60 on the output side of the DG 2, respectively. Further, the A / D converter 6 on the output side of the image sensor B (R)
DSPs 88 and 90 are respectively connected to the output side of 2. Further, the DSP 88, 90 has a multiplier 9
2, 94 are provided respectively.

The multiplier 92 includes A / D converters 58 and 62.
The output including the coefficient K is multiplied by the coefficient K, and the multiplier 94 performs the multiplication including the coefficient K by the outputs of the A / D converters 60 and 62.
The output sides of the DSPs 84, 86, 88, 90 are connected to the line memories 70, 72, 96, 98, respectively,
The line memories 96, 98 further include the line memories 74,
Connected to 76. The line memories 96 and 98
The output control unit 82 also performs the operation control of.

Of the above, the DSPs 84, 86, 88, 9
0 is for performing signal processing such as so-called γ correction and white clipping on the input signal. Multiplier 9
2, 94 are B (R) supplied from the A / D converter 62
Is a circuit for multiplying the G read signal supplied from the A / D converters 58 and 60 by the G read signal and the coefficient K to correct the read signal. The line memories 96 and 9
8 has the same operation as the line memories 70 and 72.

Next, regarding the operation of the above apparatus,
This will be described with reference to the time chart of FIG. The order of reading data from each of the image sensors DG1, DG2, B (R) is the same as in the case described above, and FIGS.
(D) is the same as (A) to (D) of FIG. Of these, the read signals of the image sensors DG1 and DG2 are the same as in the case of FIG. 15 except that the .gamma. Correction is performed by the DSP, and the output of the changeover switch 78 is shown in FIG. ).

On the other hand, the read signals of the image sensor DB (DR) are multiplied by the multipliers 92 and 94. Then, the multiplication results are processed by the DSPs 88 and 90, and the processed signals are stored in the line memories 96 and 90.
It is converted into a serial data string by the action of 98, 74 and 76. As a result, the output of the changeover switch 80 becomes as shown in (F) and (G) of FIG. Here, as is clear by comparing (F) and (G) in FIG. 19 with (E) and (F) in FIG. 17, in this example, the B (R) signal is the G signal and It is modified by the factor K.

For example, in the multiplier 94, the image sensor D
The signal B1 is input from the B side, and the image sensor D
If the signal G20 is input from G2, the multiplier 9
The output of No. 4 is B1 * G20 * K (see (F) in the figure).
In addition, "*" represents multiplication. Further, if the signal B1 is input to the multiplier 92 from the image sensor DB side and the signal G11 is input from the image sensor DG1, the output of the multiplier 92 is B1 * G11 * K (FIG. (See (F)). These are B1 * G20 * K and B1 * G11
* K is stored in the line memories 96 and 98,
The data is written in the line memories 74 and 76 at high speed and is read out alternately via the changeover switch 80. Then, the output of the changeover switch 80 becomes as shown in FIG.
The same applies to R (see (G) in the same figure).

The above operation is repeated, and a G image signal compatible with high-definition is obtained as shown in FIG.
For B and R, these G image signal and coefficient K
B image signal and R image signal corrected by
They are obtained as shown in FIGS. In this way, the B and R image signals are corrected by using the G image signals with high vertical resolution obtained from the image sensors DG1 and DG2. The G image component in the input image is
In the case of high-definition, the ratio in the luminance signal is high as shown by the following expression, and contains more information than the B and R images. Y = 0.212R + 0.701G + 0.087B

Therefore, rather than using two image sensors for B and R, two image sensors for G are used to obtain an image signal with high vertical resolution, and this is used to correct B and R. By doing so, it is possible to increase the vertical resolution very efficiently while being simple.

Further, according to this example, the processes such as γ correction and white clip are performed at the low frequency stage. As described above, when various kinds of signal processing are performed on the read signal from the image sensor, it is advantageous that the frequency is low. However, it does not prevent performing various signal processing after the high-definition scanning frequency is reached.

As described above, according to this embodiment, two image sensors that have been subjected to an optical mask process are prepared for picking up an image of G, and these are arranged by shifting one pixel in the vertical direction. Since the readout signal of the image sensor is subjected to the multiplex and scan conversion processing to obtain the high resolution image signal for high-definition, there are the following advantages.

(1) As an image sensor, inexpensive 1 /
Since a 3-inch PAL camera is used, a very inexpensive and realistic high-definition video camera can be provided. (2) Since both B and R have a single-plate structure, they are inferior in terms of resolution, but are very effective in reducing size, weight, and cost. Regarding the resolution,
The improvement can be achieved by correcting and interpolating the signal by using a high resolution G image signal containing more information than B and R.

(3) High-definition signals require high-speed processing, but since the PAL image sensor is used in this embodiment, no special high-speed processing technology is required, and peripheral devices are used. The general thing currently used as can be used. In particular, a DSP currently popular for signal processing such as multiplex and scan conversion can be used, and it is very advantageous in that cost can be reduced by using general-purpose components. Further, the optical mask can be realized only by adding a simple optical mask treatment step.

<Other Embodiments> The present invention is not limited to the above embodiments, and includes the following, for example. (1) In the above-described embodiment, only G has two plates and B and R have one plate. However, all R, G, and B have two plates, and a total of six plates may be used. (2) Correction and interpolation for the B and R image signals may be performed as necessary. Further, in the above-described embodiment, the correction and the interpolation are performed by multiplying the G image signal by the coefficient K, but the method may be appropriately changed as necessary. For example, the G image signal is multiplied by a constant and added to the B and R image signals.

(3) In the above embodiment, the image sensor is arranged so as to be shifted by one pixel in the vertical direction. However, this arrangement itself is at the same position in the vertical direction with respect to the incident optical axis, and the signal reading is performed by one. The same effect can be obtained by shifting the lines. The present invention also includes the case where the signal read area is shifted between the two image sensors in this way. In this case, one of the two image sensors is driven by being shifted by one line in the vertical direction with respect to the other. However, in general, it is more convenient for the R, G, and B image sensors to be driven simultaneously. Therefore, the image sensors are arranged so as to be displaced with respect to the incident optical axis as in the above embodiment. The same applies to the horizontal direction.

(4) For processing the signal read from the image sensor, for example, the signal is converted into a digital signal and D
You may make it utilize SP etc. (5) In the above-described embodiment, the case where an image with a high-definition aspect ratio of 16: 9 is obtained has been described. However, the present invention is not limited to this, and the ratio may be set appropriately.

(6) The above embodiment is applied to a three-plate type image pickup device, but is also applicable to a single-plate type color image pickup device and a monochrome image pickup device. (7) Further, in the above-mentioned embodiment, the required high-definition image area WW is set at the substantially center of the image sensor, but it may be set at a position shifted vertically and horizontally.

(8) In the above-described embodiment, the image pickup devices are arranged with a one-pixel pitch in the vertical direction and a 1 / 2-pixel pitch in the horizontal direction, but how much the two image pickup devices are displaced is as follows.
After all, it is related to how to perform masking. In short, it suffices that the center lines of the apertures of the pixel rows arranged by masking be equidistant when viewed on the screen after the scan conversion to high definition.

(9) As described above, the image sensor D
Since the left and right of the image of G1 is reversed with respect to the other image sensors DB, DR, and DG2, normal imaging is obtained by the following method. A CCD is used that has the capability of reading out the signal with left-right inversion. A signal is read from the image sensor DG1 while being left-right inverted, and the signal is stored in a memory means such as a line memory or a frame memory. When read out, the signal is left-right inverted to obtain a normal vertical image signal.

However, as shown in FIG. 20, the right and left may be optically reversed to obtain a normal image. First, in the example of FIG. 3A, the G first prism 100 is provided via the R prism 12 and a minute air layer (not shown). A half mirror 104 is provided between the G first prism 100 and the G second prism 102.

The G light included in the incident light is the R prism 1
The light passes through 2 and enters the G first prism 100. G first
The G light incident on the prism 100 is further reflected by the half mirror 1.
The G light that is incident on 04 and is reflected and split by the half mirror 104 is in contact with the R prism 12 via the air layer.
The light is totally reflected by the incident surface 106 of the first prism 100, and the image sensor DG1 passes through the G trimming filter 14A.
The incident image is formed on the image plane of. And the half mirror 104
The G light that has passed through is incident on the image forming surface of the other image sensor DG2 via the G trimming filter 14B and forms an image.

Next, in the example of FIG.
2 and the dummy prism 110 are in contact with each other, and the G first prism 112 is provided via the dummy prism 110 and a minute air layer (not shown). Also, this G first
A half mirror 116 is provided between the prism 112 and the G second prism 114.

The G light included in the incident light is the R prism 1
2, G first prism 1 through the dummy prism 110
It is incident on 12. The G light incident on the G first prism 112 is further incident on the half mirror 116, and the G light reflected and split by the half mirror 116 is the dummy prism 1.
10 is totally reflected by the incident surface 118 of the G first prism 112 which is in contact with the G first trimming filter 14A through the air layer.
The incident image is formed on the image forming surface of the image sensor DG1 via. Then, the G light transmitted through the half mirror 116 is
The incident image is formed on the image forming surface of the other image sensor DG2 through the G trimming filter 14B.

In any of the above, since the light incident on the image sensor DG1 is reflected twice like B and R, a similar erect image can be obtained. Therefore, there is no need for left-right inversion, and four image sensors D
B, DR, DG1, and DG2 having the same performance can be used. (10) In the above-mentioned embodiment, the image sensor compatible with the PAL system is used, but any other system may be used as long as it is equivalent thereto.

[0086]

As described above, the color separation optical system, the image pickup method, and the image pickup apparatus according to the present invention have the following effects. (1) Since a high-resolution image signal such as high-definition can be obtained by using an image sensor having a relatively low vertical resolution such as the PAL system, a very inexpensive and realistic high-resolution video camera can be obtained. it can. (2) By correcting and interpolating the image signal, it is possible to improve the resolution of the image signals of other colors based on the image signal of the color with high resolution. (3) Further, general general-purpose components can be used including peripheral devices, and the degree of technical difficulty is low. The optical mask can also be realized by adding a simple optical mask treatment step.

[Brief description of drawings]

FIG. 1 is a configuration diagram showing an embodiment of a color separation optical system according to the present invention.

FIG. 2 is an explanatory diagram showing an operation of the color separation optical system.

FIG. 3 is an explanatory diagram showing an image area of a PAL type image sensor that constitutes the color separation optical system.

FIG. 4 is an explanatory diagram showing a pixel interval of the image sensor.

FIG. 5 is an explanatory diagram showing a high-definition prescribed area set on the image sensor.

FIG. 6 is an explanatory diagram showing how light is incident on the image sensors DG1 and DG2.

FIG. 7 is an explanatory diagram showing a positional relationship between image sensors DG1 and DG2.

FIG. 8 is an image sensor D without masking.
It is explanatory drawing which shows the order of the signal read-out from G1, DG2.

FIG. 9 is a time chart showing the sequence of signal reading from the image sensors DG1 and DG2 when the masking is not performed.

FIG. 10 is an explanatory diagram showing scanning positions when the image sensors DG1 and DG2 are arranged so as to be displaced by 1/2 pixel.

FIG. 11 is an image sensor D when masking is performed.
It is explanatory drawing which shows the order of the signal read-out from G1, DG2.

FIG. 12 is an explanatory diagram showing a correspondence between pixels read from the image sensors DG1 and DG2 and a high-definition screen.

FIG. 13 is a time chart showing the sequence of signal reading from the image sensors DG1 and DG2 when the masking is performed.

FIG. 14 is an explanatory diagram showing masking for increasing the resolution in the horizontal direction and an arrangement state of image sensors DG1 and DG2.

FIG. 15 is a block diagram showing an example of a circuit that performs signal reading and scan conversion processing from each of the R, G, and B image sensors.

FIG. 16 is an explanatory diagram showing a signal reading order in each of the R, G, and B image sensors by the circuit.

FIG. 17 is a time chart showing the sequence of signal reading in each of the R, G, and B image sensors by the circuit.

FIG. 18 is a block diagram showing an example of a circuit that performs signal reading from each of R, G, and B image sensors, scan conversion, and correction processing.

FIG. 19 is a time chart showing the order of signal reading and correction in each of the R, G, and B image sensors by the circuit.

FIG. 20 is a configuration diagram showing another embodiment of the color separation optical system.

[Explanation of symbols]

10 ... B prism, 12 ... R prism, 14,100,
102, 112, 114 ... G prism, 16, 104,
116 ... Half mirror (incident light splitting means), 50 ... Read control section (read control means), 52, 54, 56 ... Preamplifier, 58, 60, 62 ... A / D converter, 64, 66, 6
8, 70, 72, 74, 76, 96, 98 ... Line memory (memory means), 78 ... Changeover switch, 80 ... Changeover switch (image signal correction / interpolation means), 82 ... Output control section (output control means), 92, 94 ... Multiplier (image signal correction / interpolation means), DB, DR ... Image sensor, DG1,
DG2 ... First and second image sensors, P ... Horizontal pixel pitch, PS ... Pixels, SA, SB ... Masking area, WW ... Hi-vision image area.

Front page continued (72) Inventor Naotake Nakata 3-12 Moriya-cho, Kanagawa-ku, Yokohama, Japan Victor Company of Japan, Ltd. (72) Masao Nozaki 3-12 Moriya-cho, Kanagawa-ku, Yokohama, Japan Victor Co., Ltd.

Claims (5)

[Claims] 1. A horizontal pixel column corresponding to a second system having a relatively low resolution different from the first system having a relatively high resolution, and having a horizontal pixel line number of 1/2 of the number of horizontal scanning lines of the first system. The first and second image sensors in which the image area having the aspect ratio of the first method can be set, and the overlapping portions of the openings of the adjacent horizontal pixel rows when the reading area is shifted in the vertical direction are masked, Incident light splitting means for splitting light to be imaged, and the image sensors are arranged with their reading areas shifted in the vertical direction on the split light optical axis of the incident light splitting means. Optical system. 2. The first and second image sensors according to claim 1, wherein the overlapping portions of the openings of the adjacent vertical pixel rows when the reading area is shifted in the horizontal direction are masked, and the divided light beams are divided. A color-separation optical system, wherein the reading areas are arranged so as to be horizontally displaced on an axis. 3. The method of claim 1, at least for green images.
Alternatively, the first and second images are captured by the color separation optical system described in 2.
The image pickup method is characterized in that the image signal read from the image sensor is subjected to scan conversion to obtain a green image signal having a vertical resolution of the first method. 4. A read method for simultaneously driving the first and second image sensors of the color separation optical system according to claim 1 to read out a signal at a frequency half the horizontal scanning frequency of the first method. The control means, the memory means for temporarily storing these read signals, and compressing and outputting in the time axis direction corresponding to the horizontal scanning frequency of the second method, and the signal output from the memory means of the first method. An image pickup apparatus, comprising: an output control unit that performs scan conversion so that an order corresponding to horizontal scanning is obtained. 5. The image signal correcting / interpolating means according to claim 4, wherein the image signal correcting / interpolating means for correcting or interpolating an image signal of another color using the green image signal output from said memory means. An image pickup apparatus comprising: