JP2635917B2 - Electronic still camera - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic still camera for recording an image on a recording medium.

[0002]

2. Description of the Related Art There are two types of electronic still cameras: analog cameras using a floppy disk, and digital cameras using a semiconductor memory (RAM, etc.). Compared to analog recording, digital recording has various advantages, such as being free from image quality deterioration due to copying and suitable for miniaturization because there is no rotating mechanism for recording. A solid-state image sensor generally used for consumer use has about 400,000 pixels in view of the price of the solid-state image sensor.

[0003] A solid-state image pickup device having such a number of pixels can reproduce an image which is almost satisfactory for viewing on a current monitor, but has a high definition (HD).
It is not enough for output to a monitor or hard copy. Further, when a high-definition solid-state imaging device (equivalent to 2 million pixels) is used for high definition, the cost becomes too high and is not suitable for a consumer electronic still camera.

Furthermore, there is a still image input device which uses a low-cost solid-state image sensor having a few hundreds of thousands of pixels and moves the solid-state image sensor to achieve higher resolution. According to this device, first, a red (R) filter of a color filter disk is placed in front of the solid-state imaging device, and an image signal is read from the solid-state imaging device. This signal is subjected to predetermined processing by a preprocessing circuit, and then converted into a digital value by an A / D converter. Further, a process of converting the image signal into a recording format is performed by a digital signal processing circuit, and the image signal is recorded on a recording medium.

[0005] When one frame of R data is recorded in this way, the solid-state imaging device is slightly moved for higher resolution, and the same recording is performed to obtain a plurality of R frame data. When the processing of R is completed, the color filter disk rotates, and a G filter element is placed on the front surface of the solid-state imaging device.
The processing in this case is performed and recorded in the same manner as the processing of R.
Similar processing is performed on the B filter element.

Although a high-resolution still image is obtained as described above, the solid-state image pickup device using this system is a monochrome device, and the camera body is large because of a rotation filter for taking in color information in a frame sequential manner. Since only a stationary object can be photographed, it is not suitable for an electronic still camera for photographing a still image of a moving object.

Further, when an electronic camera having a general solid-state imaging device of about 400,000 pixels captures an image of a black-and-white subject such as a character, color moiré due to a color filter array occurs.
There is a problem that a clear image cannot be obtained. In addition, in order to reduce the color moiré, a method of providing a quartz optical filter is generally adopted, but this method is not desirable because the resolution is deteriorated.

[0008] In addition, several companies have announced or released digital electronic camera systems that record still images using digital signals using a card-shaped semiconductor memory as a recording medium. The semiconductor memory card is detachable from the camera body. Semiconductor memory still has a high unit price per bit, and its storage capacity may not be enough.
Magneto-optical disks, hard disks, magnetic tapes, magnetic bubble memories, and the like are used.

These electronic cameras can transfer image digital data to a monitor of a personal computer or a work station and display the image. However, an analog signal to an RF terminal or a video terminal is connected to a normal TV standard monitor. In general, an image is output in the form of an input. The reproduced image at this time is in a range formed in the effective imaging area of the imaging element at the time of shooting, and this range is fixed.

[0010] In addition, photographing data can be recorded for each still image on a secondary recording medium such as the aforementioned semiconductor memory card and disk tape.

The digital system such as a memory card and the above-mentioned video floppy system are different from conventional photographic prints in that their contents cannot be understood from the appearance. The only choice is to load the recording medium into the playback device and play it back to check it, or to write down the contents on the package.

Japanese Patent Application No. 2-234492 discloses a technique for improving the efficiency by using conditions at the time of photographing as a method of image retrieval. The photographing conditions include photographing date and time, white balance, incident light amount, focus, aperture, zoom, nonuse of flash, humidity, air pressure, photographing camera ID, lens type data, and the like.
In the reproducing unit, the shooting condition is prepared in a question format, and the searcher answers the answer or the searcher inputs shooting conditions by himself. For example, questions may be taken outdoors or indoors, and whether the subject is a person, a landscape, or a printed matter. The reproducing unit reads the photographing condition data necessary for these questions, performs an arithmetic operation, and reproduces an image having a high possibility.

Further, as an electronic still camera, a camera that digitizes analog image information obtained from an image pickup device, compresses data, and then records data has already been commercialized.

On the other hand, the zoom function is widely used in home video movie cameras and compact cameras.
As a method of zooming a home video movie camera,
There are two types, one using a lens and the other an electronic zoom that performs arithmetic processing on the image signal output from the image sensor. With the lens zoom, a good image can be obtained at the time of zooming, but a large lens is required with an increase in magnification, and it is difficult to make the camera compact. On the other hand, electronic enlargement processing can provide a small and lightweight video movie camera or the like without changing the camera lens.

When performing an electronic image enlargement process, since the image is spatially sampled in the image sensor, it is necessary to compensate for the sample pixels in accordance with the enlargement ratio. In such a case, a general method is to use the actual pixel data obtained by the image pickup device and create data of supplementary pixels by interpolation processing. When performing pixel interpolation processing in real time, a two-point interpolation method that requires a small circuit scale is widely used.

As an example of the two-point interpolation method, an IEEE Transact used in a movie camera is used.
ions on Consumer Electron
ics Vol. 37, no. 3, August 19
91 "An Electronic Zoom Vid
eo Camera Usage Image Sca
However, the enlargement process according to this method impairs the bandwidth of the original signal and causes a reduction in resolution. Also, current video movie cameras convert image information into an analog signal. The digital image data was not output or recorded in a compressed form as digital image data.

On the other hand, with respect to image compression, compression using block coding is being adopted as an international standard for moving images and still images, and products such as high-definition still image storage devices, TV telephones, TV conference systems, etc. Application is considered. When digital zooming of these stored or coded digital images is performed on the reproduction side, there is a problem that block distortion due to block coding is perceived by human eyes and image quality deteriorates.

[0018]

As described in detail above, (1) In a conventional electronic still camera, when the image is output to an HDTV (high-definition TV) due to a lack of the number of pixels of the solid-state image sensor, the image is viewed. In addition, there has been a problem that a satisfactory image cannot be obtained when a hard copy is made.

Further, a still image input device that takes in a color image in a frame sequential manner using a black-and-white image pickup device is not suitable for an electronic still camera because the camera body is large and can shoot only a stationary object.

Therefore, a first object of the present invention is to use a color solid-state image pickup device conventionally used for consumer use, and
An object of the present invention is to provide a high-quality electronic still camera that can be viewed on a DTV and that can obtain a satisfactory image even when hard copied.

(2) In a conventional electronic still camera, a color solid-state image sensor is used. Therefore, when a black-and-white subject such as a character is imaged, there are problems such as occurrence of moire and insufficient resolution.

A second object of the present invention is to obtain a high-quality image of a black-and-white subject such as a character with high resolution and no generation of color moiré by using a color solid-state imaging device conventionally used for consumer use. An electronic still camera is provided.

(3) The conventional electronic still camera has a problem that the output range of the reproduced image is fixed.

A third object of the present invention is to provide an electronic still camera having a new reproduction method in which the output range of a reproduced image is wider than that of a standard TV monitor and an operator can change the output range.

(4) To search for a recorded image,
Adding a title or a keyword method to each image as described above is a very time-consuming task. Further, even if the title or keyword can be clearly expressed in the document, it may be difficult to clearly express the title or keyword in the captured image. The aforementioned "digital electronic still camera system"
Was invented for the purpose of solving the above-mentioned problem, and was to provide a digital still camera system capable of efficiently searching for a desired image from recorded images.

A fourth object of the present invention is to provide an electronic still camera with further improved efficiency. (5) As described above, both the electronic zoom and the block coding basically involve image quality deterioration. When electronic zoom and block coding are adopted to construct an image pickup apparatus that is compact, has a zoom function, and performs data compression, image quality degradation becomes a problem.

A fifth object of the present invention is to provide an electronic zoom function for reducing the size of an image pickup apparatus and a function for efficiently compressing image information for storing a large number of images on a small-capacity recording medium. Provided is an imaging device having an electronic zoom function and an image compression function in which image quality deterioration is suppressed despite using an image deterioration process such as electronic zoom and high-efficiency image compression. Is to do.

If the square of the zoom magnification ratio at the time of selecting the zoom is lower than the image compression ratio, it is called a high magnification, and if the square is higher than the image compression ratio, it is called a low magnification. At this high magnification, the image quality deteriorates. It is an object of the present invention to provide an imaging device capable of storing a large number of images on a small-capacity recording medium without using the same.

[0029]

According to the present invention, there is provided a color solid-state imaging device having a color filter and a color solid-state imaging device having a color filter. Signal processing means for performing signal processing on the output signal to generate a color image signal of a still image, recording means for recording the color image signal of the still image generated by the signal processing means, and Moving means for moving in a scanning direction, a vertical scanning direction, and an oblique direction.

Here, the color filters of the fixed color image pickup device have two rows and two columns of color filter elements as filter units, and these filter units are periodically arranged in a horizontal scanning direction and a vertical scanning direction orthogonal thereto. And the first
The color filter elements in a row include a green filter element and a red or blue filter element, and the color filter elements in a second row include a green filter element and a blue or red filter element. One still image is obtained by moving the pixel pitch (Px) and the distances of (1 / 2Px + 1 / 2Py) and (-1 / 2Px + 1 / 2Py) in the oblique direction and taking in data at a total of four positions. It is characterized by the following.

Alternatively, the color filters of the color solid-state imaging device include a red filter element, a green filter element, and a blue filter element arranged in a three-row cycle in the vertical scanning direction. One still image is obtained by moving a distance of 1/2 Py in the vertical scanning direction and (1.5 Py + 1/2 Py) in the oblique direction and taking in data at a total of four positions. Alternatively, 1.5Px in the horizontal scanning direction, (−3 / 4Px + ／ Py) and (3 / 4Px + 1 /
2Py) is moved, and one still image is obtained by taking in data at a total of four positions.

Further, the color solid-state imaging device has a charge sweeping function, and performs a charge sweeping operation during a moving period by the moving means. (2) In order to achieve the second object, the present invention generates a color image signal of a still image by performing signal processing on a color solid-state imaging device having a color filter and an output signal from the color solid-state imaging device. A color image generation signal processing unit, a binary image generation signal processing unit that performs a binarization process on an output signal from the color solid-state imaging device to generate a binary image signal, and the color image signal generation signal processing unit. Recording means for recording the obtained color image signal or the binary image signal obtained by the binary image generation signal processing means.

Alternatively, a first imaging module including at least an optical lens and a color solid-state imaging device;
A color image generation signal processing unit that performs signal processing on an output signal from the imaging module to generate a color image signal of a still image; a second imaging module including at least an optical lens and a monochrome solid-state imaging device; Black-and-white image generation signal processing means for performing signal processing on an output signal from the imaging module to generate a black-and-white image signal of a still image, and a color image signal or the black-and-white image generation signal obtained by the color image generation signal processing means Recording means for recording the black and white image signal obtained by the processing means;
The image pickup module and the second image pickup module are detachable.

(3) According to the present invention, in order to achieve the third object, a signal is read out once from the image pickup device, and in addition to the normal photographing mode in which the reproduction output range is fixed, the image pickup device By moving the subject image-forming position, reading out the signal a plurality of times, and performing signal processing suitable for this, the output range during reproduction is made variable.

That is, an electronic still camera according to the present invention comprises: a solid-state imaging device on which a subject image is formed; moving means for moving a subject image formation position on the solid-state imaging device;
Signal reading means for reading a still image signal from the solid-state image pickup device a plurality of times in accordance with the movement of the image forming position of the subject by the moving means; and a predetermined designated one of the still image signals read by the signal reading means. Recording means for recording the still image signal of the area on a recording medium, and reproducing means for reproducing the still image signal recorded on the recording medium,
Display means for displaying a still image signal reproduced by the reproduction means.

Further, the reproduction output range may be widened by using an element wider than the output range of the reproduction monitor as the image sensor.

(4) In order to achieve the fourth object, the present invention utilizes a method using photographing condition data described in the already-filed "Digital Electronic Still Camera System". Is added.

That is, the present invention provides a camera unit for recording an image signal photographed by using a solid-state image pickup device on a first recording medium, the first recording medium or an image signal recorded on the first recording medium. And a reproducing unit for reproducing an image by mounting a second recording medium on which a plurality of images are recorded, wherein the camera unit is configured to record the image on the first recording medium at that time. The reproducing unit reads out the photographing condition data recorded on the first or second recording medium, and retrieves the photographing condition data recorded on the first or second recording medium. And an image output means for determining an image likely to correspond to the question data from the search question means from the photographing condition data and outputting an image from the image having a high possibility. To.

(5) In order to achieve the fifth object of the present invention, an image pickup apparatus comprising an interpolation processing means for electronically enlarging and displaying a predetermined area of a digital image and an image compression means by block coding. The apparatus performs the interpolation processing before performing the image compression processing. Further, as described above, the apparatus is provided with a means for storing only the area enlarged at the time of high magnification in the recording medium.

That is, an image pickup apparatus according to the present invention comprises: a solid-state image pickup device; interpolation processing means for performing interpolation processing for image enlargement on a still image signal output from the solid-state image pickup device; Compression processing means for performing compression processing on a still image signal interpolated by the above, recording means for recording the still image signal compressed by the compression processing means on a recording medium, and still image data recorded on the recording medium. It is characterized by comprising reproduction means for reproducing an image signal and display means for displaying a still image signal reproduced by the reproduction means.

[0041]

(1) In the present invention, a color solid-state image sensor is used, and in the HD (high definition) mode, the solid-state image sensor is moved in a horizontal scanning direction, a vertical scanning direction, and an oblique direction, respectively, in accordance with a color filter array. In order to obtain one still image by taking in data of a total of 4 positions,
High resolution can be achieved.

Further, since no charge is accumulated during the movement period, blurring due to the movement is eliminated, and a clear image can be obtained.

Further, since it is not necessary to add a signal processing circuit for the HD mode by devising the signal processing, the size of the electronic still camera can be reduced.

(2) In the present invention, a color solid-state imaging device is used, and when an image requiring only binary data such as a character is photographed, a binarization signal processing circuit is used to perform a binarization process emphasizing resolution. Thus, the amount of data can be efficiently reduced and a high-resolution binary image can be obtained.

Further, by making the image pickup module detachable, it is possible to perform photographing in accordance with the purpose.

(3) In the present invention, by moving the position of the image pickup device or moving the optical path between the lens and the image pickup device, the image of the subject on the image pickup device is moved in the vertical direction, and each position is moved. The vertical scanning twice as large as the normal scanning is obtained by reading out the imaging signal. Half of the vertical scanning image signal is signal-processed as an effective image portion, and is recorded on a recording medium. The angle of view in the horizontal direction of an image recorded on a recording medium is twice as large as that of a normally recorded image.

When this is reproduced by the reproducing circuit, the output range of the reproduced image can be moved in the horizontal direction by changing the horizontal reading position of the reading frame memory. If an image of a subject in the horizontal direction is moved at the time of image input, the resolution is increased.

Further, using an image sensor having a wide horizontal angle of view,
In the case of normal shooting, the pixel at the center is used. In the case of another shooting mode, by using all the pixels, the horizontal reproduction image range can be expanded.

(4) In the present invention, position information generated outside the electronic camera or inside the electronic camera is recorded on a recording medium together with image data at the time of shooting together with other shooting condition data. The image reproducing unit reads out the position information of the shooting condition data recorded together with these image signals, and outputs the images in order from the image having the position information closest to the answer to the question asking the shooting location of the search. Go.

(5) The image pickup apparatus according to the present invention has a standard image pickup mode and an enlarged image pickup mode. In the standard image pickup mode, a signal from the image pickup apparatus is subjected to analog signal processing and analog / digital conversion to perform zoom processing. Output or recording is performed after digital signal processing not included.

In the enlarged image pickup mode, the signal from the image pickup apparatus is subjected to analog signal processing and analog-to-digital conversion, and after performing digital signal processing including zoom processing, output or recording is performed. The zoom processing unit in the enlarged imaging mode selects different processing at the time of the low magnification and at the time of the high magnification, respectively.

At a low magnification, by performing image compression after performing the interpolation processing, deterioration of image quality due to image compression can be minimized. By outputting only the area to be enlarged to the recording medium without compressing it at high magnification,
It is not affected by image quality deterioration due to image compression.

Therefore, at both low magnification and high magnification, deterioration of the image quality due to image compression is minimized, and the amount of image data output to the recording medium is reduced, so that a large number of images can be recorded on a small capacity recording medium. can do.

[0054]

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of an electronic still camera according to the first embodiment of the present invention. In this figure, a color solid-state imaging element, for example, a CCD 1 is moved by a piezoelectric element 11 driven by a driving circuit 2. The output terminal of the CCD 1 is connected to a buffer memory 6 via a preprocessing circuit 4 and an A / D conversion circuit 5 which are controlled by a synchronization signal from a synchronization signal generation circuit 3 in series. The buffer memory 6 is controlled by a memory control circuit 7 which receives a synchronization signal from the synchronization signal generation circuit 3. The output terminal of the buffer memory 6 is connected via a signal processing circuit 8 to a recording medium 9 such as a semiconductor memory. An output terminal of the mode switching circuit 10 is connected to the synchronization signal generating circuit 3.

First, normal processing of the electronic still camera will be described.

The normal photographing mode is selected by the mode switching circuit 10, and the driving circuit 2 operates so that the synchronizing signal pulse outputted from the synchronizing signal generating circuit 3 reads out the image signal for one frame from the CCD 1. The drive circuit 2 sends a read drive signal to the color CCD 1 and a drive signal to move the CCD 1 to the piezoelectric element 11.
The image signal read from the color CCD 1 is input to a pre-processing circuit 4, where it is subjected to predetermined processing such as pre-amplification, white balance, and gamma correction, and then converted into a digital signal by an A / D converter 5. Are temporarily stored in the buffer memory 6. When one frame of image data is stored in the buffer memory 6, it is processed in a non-interlaced manner.
Even if the reading of the color CCD 1 is interlaced, the image signal can be processed in a non-interlaced manner through the buffer memory 6. The image signal read from the buffer memory 6 is input to the signal processing circuit 8. The signal processing circuit 8 generates a luminance signal and a color difference signal from the image signal, performs data compression, and records the compressed data on a recording medium 9 such as a semiconductor memory.

FIG. 5A shows a color filter element of the color CCD 1.
The signal processing in the case of arrangement as shown in FIG. 6 will be described using the luminance / color difference signal generation circuit of FIG.

The signal read from the buffer memory 6 is converted by the matrix circuit 13 into a high-frequency luminance signal Y _H , a low-frequency luminance signal Y _L , and color difference signals RY and BY. The arithmetic expression of the matrix is as follows in the case of G _{2 (Y) and 3 (X)} pixels in FIG. 5A.

[0060] _{Y H = 0.25 (G 1,3 +} G 2,3 + R 1,2 + B 2,2) (1) Y L = 0.30 (R 1,2 -G 1,3) +0.11 (B 2, ₂₋ G _2,3 ) + G _2,3 (2) RY = 0.70 (R _1,2 −G _1,3 ) −0.11 (B _2,2 −G _2,3 ) (3) BY = −0.30 (R _1,2 −G _1,3 ) +0.89 (B _2,2 −G _2,3 ) (4) In the case of G _3,3 pixels, the following matrix is obtained.

Y _H = 0.25 (G _2,3 + G _3,3 + R _3,2 + B _2,2 ) (5) Y _L = 0.30 (R _3,2- G _3,3 ) +0.11 (B _{2, 2−} G _2,3 ) + G _3,3 (6) RY = 0.70 (R _3,2 −G _3,3 ) −0.11 (B _2,2 −G _2,3 ) (7) BY = −0.30 (R _3,2 −G _3,3 ) +0.89 (B _2,2 −G _2,3 ) (8) After passing through the matrix circuit 13 that performs the above processing, the color difference signal RY And BY are low-pass filters (L
PF) 17 and 18. Luminance signal Y
The signal Y _L -Y _H is band-limited by the LPF 15, is generated by adding the Y _H. The luminance signal and chrominance signal thus produced are subjected to data compression by a compression circuit (not shown), and are recorded on the recording medium 9.

Next, the HD recording mode will be described.

FIG. 5B shows a color CCD in the HD recording mode.
The state of the shift of 1 is shown. First, the mode switching circuit 1
It is assumed that the HD mode is selected by 0 and the synchronization signal generator 3 outputs synchronization signal pulses for four frames. First, in the normal state (non-shift state (1)), color CC
An image signal is read from D1 and is taken into the buffer memory 6. When the image signal for one frame has been captured, the color CCD 1 is in the state of (2) (P in the horizontal direction).
x).

Here, the means for shifting the color CCD 1 where Px and Py are the pixel pitches in the horizontal and vertical directions include shifting the color CCD 1 itself with the piezoelectric element 11 or the like, or using the second embodiment shown in FIG. Like color CC
Various means are conceivable, such as providing a parallel plate 12 on the front surface of D1 and changing the angle to change the optical path and equivalently shift. Such a shift mechanism is disclosed in, for example, JP-A-58-130677 and JP-A-60-54576.

After shifting from the position (1) to the position (2) by the above method, an image signal is read from the color CCD 1 and taken into the buffer memory 6. Similarly, also at the position (3) (− ／ Px + ／ Py) and the position (4) (（Px + ／ Py),
An image signal is read from the color CCD 1 and taken into the buffer memory 6.

Here, the relationship between the signal reading operation of the color CCD 1 and the pixel shift will be described in detail.

An explanation will be given using an interline transfer type CCD (IT-CCD) capable of full-line reading as shown in FIG.
In the D array, a vertical transfer unit 402 is provided adjacent to a column of the photodiode photoelectric conversion elements 401. The electric charge of each photoelectric conversion element 401 is transferred to the corresponding vertical transfer unit 402 by the field shift pulse φV1, and transferred to the horizontal transfer unit 403 line by line. The pulse φH outputs the output terminal (OUT) from the output circuit 404 as an electric signal. Is output to The other end of the vertical transfer unit 402 has a charge sweeping unit 4
05 is provided.

The state of the charge of the color solid-state imaging device 1 will be described with reference to FIGS. 8A to 8E. When the processing at the positions (1), (2), (3), and (4) of the color CCD 1 as shown in FIG.
A. Here, when the color CCD 1 is shifted, for example, the CCD 1 is moved from the position (1) to the position (2).
Is shifted, it takes time to move as shown in FIG. 8B, and the charge accumulation at the position (2) becomes inaccurate, resulting in a blurred image. Therefore, in order to solve this problem, charges are swept out during the movement period as shown in FIG. 8C. In this case, the accumulation of the charges after the sweeping of the charges is performed during the period excluding the movement period as shown in FIG. 8D, and therefore, the phenomenon that the image is blurred is eliminated. The accumulated charges are transferred to the vertical transfer unit 402, and are read out over one frame as shown in FIG. 8E.

As another example of the shift, FIGS.
As shown in C, there is a method of processing an image signal at one-frame intervals. In this case, the charge accumulation period and the movement period can be shifted, so that the phenomenon of image blurring does not occur.

In the above embodiment, the present invention has been described as being applied to an IT-CCD capable of full-line reading. However, an IT-CCD of interlaced reading and a frame interline transfer type CCD (FIT-CC) have been described.
D) and the like can be applied to all solid-state imaging devices having a function of discharging electric charges.

FIGS. 10A and 10B show equivalent pixel arrangements when the above-described four positions (1) to (4) are fetched. As can be seen from the figure, the G color filter elements are arranged in a checkered pattern at a pixel pitch of 1 / 2Px in the horizontal direction and a pixel pitch of 1 / 2Py in the vertical direction, and the R and B color filter elements are arranged in the vertical direction. Are alternately arranged every two lines at a pixel pitch of 1/2 Py. By optimally selecting the arrangement of the color filter elements and the moving direction in this manner, it is possible to regularly arrange the shifted equivalent pixel arrangement,
HD (high definition) can be achieved.

Next, signal processing in this arrangement will be described. When the pixel arrangement shown in FIGS. 10A and 10B is compared with a normal pixel arrangement, these are completely different pixel arrangements, so that signal processing is also different from each other, and another signal processing circuit is required. In such a case, the circuit scale becomes large, which is not desirable particularly when downsizing the electronic still camera. Therefore, a method of using the signal processing circuit in the normal mode for the solid-state imaging device of FIGS. 10A and 10B will be described.

First, the frame data at the four positions is fetched into the buffer memory 6, but considering the two-dimensional address, the frame data is stored in the buffer memory 6 by the memory control circuit 7 in the order shown in FIG.
Is written to. When the arrangement of the two-dimensional addresses in the buffer memory 6 is compared with the arrangement of the color filter elements of the CCD 1 (array A), the arrangement of the color filter elements is equivalent to the arrangement of the two-dimensional addresses arranged vertically by two lines. .
The signal processing in this case is performed as follows.

For the vertical addresses of y numbers 1, 3, 5,..., The horizontal addresses of x numbers 2, 4, 6,.
X addresses 1, 3, 5,
.. Are processed for the horizontal address. G
The calculation of the _{3 (y) and 6 (x)} pixels is as follows.

Y _H = 0.25 (G _1,6 + G _3,6 + R _1,6 + B _3,6 ) (9) Y _L = 0.30 (R _1,6- G _1,6 ) +0.11 (B _{3, 6−} G _3,6 ) + G _3,6 (10) RY = 0.70 (R _1,6 −G _1,6 ) −0.11 (B _3,6 −G _3,6 ) (11) BY = −0.30 (R _1,6- G _1,6 ) +0.89 (B _3,6- G _3,6 ) (12) In addition, the calculation of the G _{4 (Y) and 5 (X)} pixels is a matrix of the following equation. Becomes

Y _H = 0.25 (G _2,5 + G _4,5 + R _2,5 + B _4,5 ) (13) Y _L = 0.30 (R _2,5- G _2,5 ) +0.11 (B _{4, 5−} G _4,5 ) + G _4,5 (14) RY = 0.70 (R _2,5 −G _2,5 ) −0.11 (B _4,5 −G _4,5 ) (15) BY = −0.30 (R _2,5 −G _2,5 ) +0.89 (B _4,5 −G _4,5 ) (16) Thus, when the vertical address (y) is an N line, that line and (N -2) An operation is performed using line data and data of the same horizontal address (x). By doing so, it is possible to generate a luminance signal at a pitch of 1 / 2Px in the horizontal direction, and it is possible to obtain a resolution that is twice or more the normal resolution. Further, the resolution in the vertical direction is twice as large as that of a normal color object because only the signal of the vertical line is replaced.

By performing the above processing, the HD
An image quality comparable to a TV can be obtained. In addition, as can be seen from the comparison with the equations (1) to (8), the signal processing in this embodiment performs the same operation as in the normal mode. Therefore, the same signal processing circuit 8 can be used except that the reading from the buffer memory 6 is changed. .

In the above description, the array A shown in FIG. 5A has been described, but the same processing can be performed in the array B shown in FIG. 12A and the array C shown in FIG. 14A. For these sequences, FIG. 12B and FIG.
4B, Px in the horizontal scanning direction and -1 in the oblique direction.
FIG. 13 shows an equivalent color filter element arrangement when shifted by ＰPx + ／ Py and ＰPx + ／ Py.
A and 13B and FIGS. 15A and 15B, it can be seen that they are exactly the same as in the arrangement of FIG. 5A. Not limited to these arrangements, the above-described embodiment can be realized by arranging the G color filters in every line in the vertical direction and the R and B color filters in every other line in the vertical direction.

Next, the array D (RGB stripe filter array) shown in FIG. 16A will be described. The shift of the color CCD 1 in this case is performed as shown in FIG. 16B. That is, (2) is 1.5Px, (3) is 1 / 2Py,
(4) shifts by 1.5Px + ／ Py. As can be seen from the equivalent array of color filter elements obtained when shifting (FIG. 17), since the RGB stripe array is used, the signal processing circuit 8 is exactly the same as the previous embodiment, and the resolution is horizontal. Both the direction and the vertical direction are twice as large as the normal one.

A method for further increasing the horizontal resolution in the array D shown in FIG. 18A will be described. As shown in FIG. 18B, 1.5Px (2) in the horizontal scanning direction, −3 / 4Px + ／ Py (3), and 3 / 4Px + 1 in the oblique direction.
/ 2Py (4). As can be seen from the equivalent arrangement (FIG. 19) obtained when moving as shown in FIG. 18B, this embodiment can be expected to further increase the horizontal resolution.

The following system can be considered by providing the buffer memory 6. Since the capacity of the buffer memory 6 is required to be four times as large as the number of pixels of the color CCD 1 for the HD mode, the electronic still camera of the above embodiment operates at high speed during normal shooting in order to make effective use of the buffer memory 6. Can be used for continuous shooting.

According to the third embodiment shown in FIG. 3, continuous image signals from the color CCD 1 are continuously stored in the memory areas BM1, BM2, BM3 and BM4 of the buffer memory 6. 4 stored in the buffer memory 6
The continuous image data of a sheet is read from the BM 1, subjected to digital signal processing, and recorded on the recording medium 9.

Similarly, the memory areas BM2, BM3 and B
The image data stored in M4 also undergoes the same processing as the image data stored in memory area BM1. That is, 1
The recording time of a sheet image is limited by the signal processing time, the recording time on the recording medium 9, and the like, so that high-speed continuous shooting is difficult. Therefore, several images are fetched into the buffer memory 6 at a high speed, and then signal processing is performed to write them on the recording medium 9. By doing so, there is no restriction on the signal processing time and the like, and the speed of continuous shooting can be increased.

In the third embodiment, the case where the buffer memory 6 is provided for the HD mode has been described. However, as another embodiment, a system without the buffer memory 6 can be considered.

A system as described above will be described as a fourth embodiment with reference to FIG.

First, image signals are read out with the position of the color CCD 1 at (1). The read signal is subjected to predetermined processing by the pre-processing circuit 4, and A / A
The data is converted into a digital value by the D converter 5. The signal converted into the digital value is recorded on the recording medium 9 without passing through the digital signal processing circuit 8 and in the state in which the color filter elements are arranged as they are. Similar processing is performed in positions (2), (3), and (4). When the four positions of the imaging data are recorded on the recording medium 9, a video signal is generated by the reproducing device. Thus, the recording in the HD mode can be performed without providing the buffer memory 6.

Next, another example of signal processing in the embodiment of FIG. 2 will be described with reference to the signal processing circuit of FIG. 6, the filter element arrangement patterns of FIGS. 20 to 22, and FIGS. 23 to 26.

In the digital signal processing circuit shown in FIG. 6, the signals read from the buffer memory 8 are converted into a high-frequency luminance signal Y _H and a low-frequency luminance signal Y by the matrix circuit 13.
_L , and are converted into color difference signals RY and BY. The matrix operation expression in the matrix circuit 13 is expressed as follows for the pixels W2 _{(Y) and 3 (x)} shown in FIG. Y _H = (2 / 3S W _2,3 + 2 / 3S W _1,3 + SYe _1,2 + SCy _2,2 ) /8=0.29R+0.42G+0.29B (17) Y _L = α ₁ ( SYe _1,2 -2 / 3S W _1,3 ) + β ₁ (SCy _2,2 -2 / 3S W _2,3 ) + 1 / 3S W _2,3 (18) RY = α ₂ (SYe _{1 , 2} -2 / 3S W _1,3 ) + β ₂ (SCy _2,2 -2 / 3S W _2,3 ) (19) BY = α ₃ (SYe _1,2 -2 / 3S W _1,3 ) + β ₃ (SCy _2,2 −2 / 3S W _2,3 ) (20) Further, the matrix operation expression is expressed as follows for the pixels W _{3 (Y) and 3 (X)} . Y _H = (2 / 3S W _3,3 + 2 / 3S W _2,3 + SYe _3,2 + SCy _2,2 ) /8=0.29R+0.42G+0.29B (21) Y _L = α ₁ ( SYe _3,2 -2 / 3S W _1,3 ) + β ₁ (SCy _2,2 -2 / 3S W _2,3 ) + 1 / 3S W _2,3 (22) RY = α ₂ (SYe _{3 , 2} -2 / 3S W _1,3 ) + β ₂ (SCy _2,2 -2 / 3S W _2,3 ) (23) BY = α ₃ (SYe _3,2 -2 / 3S W _{1,3 ) + β 3 (SCy 2,2 -2} / 3S W 2,3) (24) _{where, S W y, x, SYe} y, x, SCy y, x is the horizontal address (X) and vertical address (Y ), And S _w
= R + G + B, S _ye = R + G, S _cy = G + B,
R, G, and B are shown normalized to one. Also,
α ₁ , α ₂ , α ₃ , β ₁ , β ₂ and β ₃ are matrix coefficients.

After passing through the matrix circuit 13 as described above, the color difference signals RY and BY are band-limited by low-pass filters (LPF) 17 and 18. The luminance signal Y is band-limited by LPF15 signals Y _L -Y _H, is generated by adding the Y _H in the signal.
The luminance signal and the color difference signal generated in this way are data-compressed by a compression circuit (not shown) and recorded on the recording medium 9.

Next, the HD recording mode will be described.

FIG. 24 shows a color CCD in the HD recording mode.
1 shows a shift state. First, when the mode switching unit 10 selects the HD mode, the synchronization signal generator 3 outputs synchronization signal pulses for four frames. First, in the normal state (non-shift state (1)), the color C
An image signal is read from the CD 1 and written to the buffer memory 6. When an image signal for one frame is written into the buffer memory 6, the color CCD 1 is in the state (2).
(Px in the horizontal direction). Note that Px and P
y represents the pixel pitch in the horizontal and vertical directions. The means for shifting is the color C as described in the first embodiment.
This can be realized by shifting the CD 1 itself or changing the optical path.

When a pixel is shifted to the position (2) by the above method, an image signal is read from the color CCD 1 and written to the buffer memory 6. Similarly, at the position (−3Px + ／ Py) at (3) and the position (（Px + ／ Py) at (4), the color CCD 1
Are read out from the memory and written into the buffer memory 6. The image signals obtained at the four positions in this way are equivalently signals obtained from the pixels arranged as shown in FIGS. 26A and 26B. This method is illustrated in FIG.
FIGS. 25A and 25B show equivalent arrays when applied to the color filter shown in FIG. That is, FIG.
A shows an equivalent arrangement of the filter element F1, and FIG. 25B shows an equivalent arrangement of the filter elements F2 and F3.

As can be seen from the figure, all the color filter elements are arranged at a pixel pitch of 1/2 Py in the horizontal direction, and W is arranged at a pixel pitch of 1/2 Py in the vertical direction. The Ye and Cy components are alternately arranged every two lines at a pixel pitch of 1/2 Py.

Next, an example of signal processing in the above arrangement will be described.

At the vertical addresses of y numbers 1, 3, 5,..., The pixel signals at the horizontal addresses of x numbers 1, 3, 5,. , The pixel signals at the horizontal addresses of x numbers 2, 4, 6,... Are processed.

For example, the calculation of the pixels W _{5 (Y) and 3 (X)} is performed as follows. Y _H = (2 / 3S W _5,3 + 2 / 3S W _3,3 + SYe _5,3 + SCy _3,3 ) /8=0.29R+0.42G+0.29B (25) Y _L = α ₁ ( SYe _5,3 -2 / 3S W _5,3 ) + β ₁ (SCy _3,3 -2 / 3S W _3,3 ) + 1 / 3S W _5,3 (26) RY = α ₂ (SYe _{5 , 3} -2 / 3S W _5,3 ) + β ₂ (Scy _3,3 -2 / 3S W _3,3 ) (27) BY = α ₃ (SYe _5,3 -2 / 3S W _5,3 ) + β ₃ (SCy _3,3 −2 / 3S W _3,3 ) (28) Further, the matrix operation formula is expressed as follows for the pixels W _{4 (Y) and 4 (X)} . _{Y H = (2 / 3S W} 4,4 + 2 / 3S W 2,4 + SYe 2,4 + SCy 4,4) /8=0.29R+0.42G+0.29B (29) Y L = α 1 ( SYe _2,4 -2 / 3S W _2,4 ) + β ₁ (SCy _4,4 -2 / 3S W _4,4 ) + 1 / 3S W _4,4 (30) RY = α ₂ (SYe _{2 , 4} -2 / 3S W _2,4 ) + β ₂ (SCy _4,4 -2 / 3S W _4,4 ) (31) BY = α ₃ (SYe _2,4 -2 / 3S W _{2,4 ) + β 3 (SCy 4,4 -2} / 3S W 4,4) (32) _{where, S W y, x, SYe} y, x, SCy y, x is the horizontal address of the equivalent array after shift (x ), The output signal of the vertical address (y).

As described above, when the vertical address (y) is the Nth line, it is the data of the line and the (N-2) th line.
The operation is performed using the data of the same horizontal address (x). By doing so, in the horizontal direction, 1 / 2Px
A luminance signal can be generated at a pitch, and a resolution twice or more that of a normal signal can be obtained. Further, the resolution in the vertical direction is twice as large as that of a normal one because an achromatic subject is replaced only by the signal of the vertical line.

In the case of a chromatic object, red (R),
Since white (W) including all components of green (G) and blue (B) is arranged on all lines, the generation of a false color signal is smaller than that of the primary color filter array.

When the color filters of the filter element array shown in FIG. 21 are moved in accordance with the shift pattern shown in FIG. 24, the equivalent arrays shown in FIGS. 25A and 25B are obtained. Since this equivalent arrangement is exactly the same as the color filter of FIG. 20, a high-definition color image can be obtained similarly to the color filter of FIG.

In the case of the color filter element arrangement shown in FIG. 22, Py in the vertical direction, (1 / 2Px-1 / 2Py) and (1 / 2Px + 1 / 2Py) in the oblique direction according to the shift pattern shown in FIG. A shift is made. The equivalent arrangement obtained by this shift is shown in FIGS. 28A and 28B.
Is shown in In this arrangement, the color filter elements F1 are arranged horizontally and vertically in a checkered pattern at a half pixel pitch, and the filter elements F2 and F3 are arranged vertically on all lines at a half pixel pitch and horizontally in two lines. The shape is arranged alternately for each column. That is, this arrangement is shown in FIGS.
This is the same as the arrangement when the arrangement shown in FIG. 5B is laid down.
Therefore, if the signal processing is performed by exchanging the horizontal and vertical directions, a high-definition color image can be obtained with the same matrix as in FIGS. 25A and 25B. Note that the shift pattern shown in FIG. 24 can also be applied to the color filter element arrangement of FIG.

In the above description, the color filter element array of W, Ye, and Cy is described. However, the present invention is not limited to this array but can be applied to a color filter having the following array as an array including a complementary color filter. .

That is, (F) as shown in FIG.
1: G (green), F2: R (red), F3: Cy (cyan)), as shown in FIG.
(Green), F2: Ye (yellow), F3: Cy (cyan)) and (F1: Ye) as shown in FIG.
The present invention is applicable to color filters of (luminance), F2: R (red), and F3: B (blue).

Incidentally, Y (luminance) indicates a filter element having a spectral transmission characteristic of luminance.

Even with the arrangement including the complementary color filters as described above, a high-definition color image can be obtained by optimizing the shift pattern.

As another embodiment, as shown in FIG. 32, four colors (Y
e (yellow), Cy (cyan), Mg (magenta),
A description will be given of high definition when a color filter element of G (green) is used. First, the luminance signal (Y) and the color difference signals (RY, BY) in the normal mode are generated based on the following calculation, for example.

That is, the pixels Mg _{2 (Y), 3 (X)} are generated based on the following equation.

Y = (2SG _2,2 + SMg _2,3 ) /4=0.25R+0.5 G + 0.25B (33) RY = α ₁ (Scy _1,2- SYe _1,3 ) + β ₁ (SMg _{2,3 -2S G 2,2) (34)} B-Y = α 2 (SCy 1,2 -SYe 1,3) + β 2 (SMg 2,3 -2S G 2,2) (35) Further, the pixel Ye _{3 (Y), 3 (X)} is generated based on the following equation.

Y = (SYe _3,3 + SCy _3,2 ) /4=0.25R+0.5 G + 0.25B (36) RY = α ₁ (SCy _3,2- SYe _3,3 ) + β ₁ (SMg _{2 , 3} -2SG _2,2 ) (37) BY = α ₂ (SCy _3,2 -SYe _3,3 ) + β ₂ (SMg _2,3 -2 SG _2,2 ) (38) S _{YeY, X} , S _{CyY, X} , S _{MgY, X} and S _{GY, X} represent output signals corresponding to the horizontal address (X) and the vertical address (Y), and S _Ye = R + G, S _cy = G + B, and S _Mg = R + B. Assuming, R, G, B are normalized to 1 and shown. Α _1, α ₂ , β ₁ and β ₂ are matrix coefficients.

Next, the HD recording mode will be described.

FIG. 33 shows a shift state of the color solid-state imaging device 1 in the HD recording mode. First, when the mode switching unit 10 selects the HD mode, the synchronization signal generator 3 outputs synchronization signal pulses for four frames. First, an image signal is read from the color CCD 1 in the normal state (the state (1) where no shift is performed), and is written to the buffer memory 6. When an image signal for one frame is written into the buffer memory 6, the position (2) (Px in the horizontal direction) and the position (3) (-1 / 2Px + 1 / 2P)
In the shift to the positions (（Px + ／ Py) of (y) and (4), the image signals are read at the respective positions and written to the buffer memory 6.

The image signals obtained at the above four positions are equivalent to the signals obtained from the pixels arranged as shown in FIG. In FIG. 34, Fa corresponds to Ye (yellow) and Cy (cyan) color filter elements, and Fb corresponds to G (green) and Mg (magenta) color filter elements. As can be seen from the figure, all color filter elements are arranged at a pixel pitch of 1 / 2Px in the horizontal direction, and lines of Ye and Cy and lines of G and Mg are arranged at a pixel pitch of 1 / 2Py in the vertical direction. They are alternately arranged every two lines. Considering the R, G, and B components, all components are included in Fa and Fb, so that all components are arranged at a half pixel pitch both horizontally and vertically. Therefore, high definition can be achieved.

Next, an example of signal processing in the above arrangement will be described.

At the vertical addresses of y numbers 1, 3, 5,..., The pixel signals at the horizontal addresses of x numbers 1, 3, 5,. , The pixel signals at the horizontal addresses of x numbers 2, 4, 6,... Are processed.

The calculation of the pixels _{Fa5 (Y) and 3 (X)} is performed as follows.

Y = (SYe _5,3 + SCy _5,3 ) /4=0.25R+0.5 G + 0.25B (39) RY = α ₁ (Scy _5,3 −SYe _5,3 ) + β ₁ (SMg _{3 , 3 -2S G 3,3) (40} ) B-Y = α 2 (SCy 5,3 -SYe 5,3) + β 2 (SMg 3,3 -2S G 3,3) (41) Further, the pixel F The calculation of _{b4 (Y), 4 (X)} is performed based on the following equation.

[0116] _{Y = (2S G 4.4 + SMg} 4.4) /4=0.25R+0.5 G + 0.25B (42) R-Y = α 1 (SCy 2,4 -SYe 2,4) + β 1 (SMg 4,4 - 2S G _4,4 ) (43) BY = α ₂ (SCy _2,4- SYe _2,4 ) + β ₂ (SMg _4,4 -2 S G _4,4 ) (44) Luminance signal generation as described above In (2), the horizontal and vertical resolution can be improved to about twice the normal resolution since the generation can be performed in one line and at a pitch of 1 / 2Px. However, as can be seen from the equation, if (S _Cy + S _Ye ) and (2S _G + S _Mg ) are not equal in the vertical low-frequency component,
Since line crawl of the luminance signal may occur, correction is necessary. This correction is performed as follows. That is, the signals (S _Ye , S _Cy , 2
S _G, when taking the white balance of S _Mg), line crawl for formula (39) and (42) equals an arithmetic expression of the luminance signal when the achromatic subject is not generated. The occurrence of line crawl of the luminance signal is considered when a chromatic subject is photographed, that is, (R−Y ≠ 0 or B
−Y ≠ 0). Therefore, for example, a correction table is provided, the correction amount between the lines of the luminance signal is determined based on the magnitude of the color difference signal (RY, BY), and the luminance signal is corrected to generate the line scroll. Is suppressed.

Next, an embodiment in which a shift as shown in FIG. 35 is performed will be described.

In the same manner as in the previous embodiment, the horizontal direction Px
Shift (2), a shift (−3) by − ／ Px + Py in the oblique direction, and a shift (4) of ＰPx + Py. FIG. 36 shows an equivalent color filter element arrangement at that time. Cy and Ye pixels and G and Mg pixels are arranged in a checkerboard pattern at a half pixel pitch in the horizontal direction and one pixel pitch in the vertical direction. The signal processing in this case, that is, the low-frequency luminance signal Y _L , the high-frequency luminance signal Y _H, and the color difference signal R
The generation of -Y and BY is performed, for example, as follows.

The calculation of the pixels _{Fa5 (Y) and 3 (X)} is performed as follows.

Y _L = (SYe _5,3 + SCy _5,3 + SMg _5,2 + 2SG _5,2 ) /8=0.25R+0.5G+0.25B (45) Y _H = (SYe _5,3 + SCy _5,3 + SMg _3,3 + 2SG _3,3 ) /8=0.25R+0.5G+0.25B (46) R−Y = α ₁ (SCy _5,3 −SYe _5,3 ) + β ₁ (SMg _{3,3 -2S G 3,3) (47)} B-Y = α 2 (SCy 5,3 -SYe 5,3) + β 2 (SMg 3,3 -2S G 3,3) (48) Further, the pixel The calculation of F _{b5 (Y), 4 (X)} is performed based on the following equation.

Y _L = (SYe _5,3 + SCy _5,3 + SMg _5,4 + 2SG _5,4 ) /8=0.25R+0.5G+0.25B (49) Y _H = (SYe _3,4 + SCy _3,4 + SMg _5,4 + 2SG _5,4 ) /8=0.25R+0.5G+0.25B (50) R−Y = α ₁ (SCy _3,4 −SYe _3,4 ) + β ₁ (SMg from _{5,4 -2S G 5,4) (51)} B-Y = α 2 (SCy 3,4 -SYe 3,4) + β 2 (SMg 5,4 -2S G 5,4) (52) the above equation high-frequency luminance signal in the horizontal direction Y _H becomes, 1/2
Since a luminance signal can be generated at a Px pitch, a resolution approximately twice as large as a normal resolution can be obtained. Low-frequency luminance signal in the vertical direction is represented by the formula Y _L, but is generated in one line, not the resolution improvement effect of the more horizontal for a Py pitch as equivalent sequence is shown in Figure 36. However, since line crawl does not occur, it is not necessary to correct the luminance signal between lines.

As described above, even in the single-chip complementary color system, high definition can be achieved by optimizing the shift pattern.

In the above description of the embodiment, the color filter element arrangement shown in FIG. 32 has been described. However, the arrangement is not limited to this arrangement, and the arrangement shown in FIGS. 37 to 39 can be obtained by shifting the previous embodiment. An equivalent array that is exactly the same as the equivalent array is obtained.

In the embodiments described above, high definition is achieved by taking in data at four positions. An embodiment of a system for achieving higher definition will be described below.

In the method described in the previous embodiment, for example, FIG.
2 per 4 pixels as in the equivalent arrangement shown in 5A and 25B
The color filter elements F1 to which the pixels are assigned can be arranged in a checkered pattern at a half pixel pitch in the horizontal and vertical directions, but the color filter elements F2 and F3 to which one pixel is assigned have two lines in the vertical direction (the shifted line ), The insensitive pixels (insensitive pixels)
xel? ) Line occurs. This causes a problem that a false color signal is generated. In order to solve such a problem, this embodiment shifts the pattern as shown in FIG.
In FIG. 40, the capture of the image signal at the positions (1), (2), (3) and (4) is the same as that of the previous embodiment.
It is the same as signal acquisition at one position. In this case, the shift axis has four positions in two axes: a shift position only by Px in the horizontal direction (the position of (2)) and a shift position only by -1 / 2Px + 1 / 2Py in the oblique direction (the position of (3)). Can be realized. The position (4) can be realized by combining the two axes. In this embodiment, one axis is added to the above two axes. In this embodiment, a shift axis for the position shifted by Py in the vertical direction, that is, the position of (5) is added. Additional shift pattern by adding this axis (5)
Py, (6) Py + (Px), (7) Py + (− ／)
Px + ／ Py) and (8) Py + (１ ／ Px + 1)
/ 2Py) can be realized. FIG. 42 shows an equivalent arrangement obtained when shifting is performed at eight positions in this manner. The color filters F of this equivalent arrangement are F1, F2 and F
3 represents all the color filter elements. Although overlapping pixels occur in the color filter element F1, overlapping data may or may not be captured. As can be seen from the figure, all three colors (F1, F2, and F3) are arranged horizontally and vertically in a checkerboard pattern at a 1/2 pixel pitch, so that an operation for color generation can be realized in pixel units, and processing can be performed. A simple and high-definition color image with few occurrences of color false signals can be obtained by a 4-position swing. The equivalent arrangement shown in FIG. 42 is the same as the arrangement obtained when the arrangement of the color filter elements shown in FIGS. 20 and 21 is shifted in accordance with the shift pattern of FIG. 40, but is not limited to these arrangements. , The sequence (F
Assuming that 1 is G (green), F2 is R (red) and F3 is B (blue), two rows like a G stripe-R / G complete checkerboard pattern which is considered to have the best image quality in an electronic still camera. This embodiment is applicable to all arrangements having four rows as a basic unit.

Similarly, in the arrangement shown in FIG. 22 (FIG. 21), the equivalent arrangement shown in FIG. 42 is obtained by swinging the filter element to eight positions as shown in FIG. Also in this case, the present embodiment is not limited to this arrangement, and this embodiment can be applied to all arrangements having a basic unit of 4 rows and 2 columns as shown in FIG. Further, this embodiment is not limited to the color filter array composed of three colors, but can be applied to the Ye, Cy, Mg, and G mosaic arrays shown in FIG.

As described above, if the restriction on the image data loading time is small, higher definition can be realized by shifting to eight positions.

The embodiment shown in FIG. 45 is an electronic still camera capable of obtaining a high-quality image of a black-and-white subject such as a character with high resolution and no color moiré. According to this embodiment, the color solid-state imaging device (CCD) 31 is
The output terminal of the CCD 31 is connected to signal processing circuits 36A and 36A via a preprocessing circuit 34 and an A / D converter 35.
6B is selectively connected. Signal processing circuits 36A and 3
The output terminal of 6B is coupled to a recording medium 37 such as a semiconductor memory. The crystal optical filters 38A and 38B are driven by the drive circuit 32 and are selectively disposed in front of the CCD 31. The synchronization signal generation circuit (SG) 33 receives the mode selection signal from the mode switching unit 39,
The synchronization signal is supplied to the preprocessing circuit 34 and the A / D converter 35.

First, normal processing of the electronic still camera shown in FIG. 45 will be described. When the normal shooting mode is selected by the mode switching unit 39, a quartz optical filter 38A, which is a filter for reducing color moire caused by the color filter array, is set on the front surface of the color CCD 31. The signal read from the color CCD 31 is subjected to predetermined processing such as pre-amplification, white balance, and gamma correction by a pre-processing circuit 34, and then is converted into a digital signal by an A / D converter 35. 36A. The signal processing circuit 36A generates a luminance / color difference signal from the image signal, performs data compression, and sends the compressed data to a recording medium 37 such as a memory card.

Next, the monochrome photographing mode will be described.
When the black and white shooting mode is selected by mode switching,
When the color solid-state imaging device 31 is regarded as a black-and-white imaging device, a quartz optical filter 38B, which is a filter for suppressing aliasing noise due to sampling of pixels, is set on the front surface of the color CCD 31.

The signal read out from the color solid-state imaging device 31 is subjected to predetermined processing by a pre-processing circuit 34, converted into a digital signal by an A / D converter 35, and input to a signal processing circuit 36B. You. Here, a binarization process is performed. Assume that the color filter array of the color CCD 1 is the array E shown in FIG. This color CCD1
FIG. 47 shows an equivalent arrangement when is used in the black and white mode. Here, WG, WR, WB are color filters R, G,
This means that B is regarded as a white pixel. If the subject is black and white and the white balance is obtained in the pre-processing circuit 34, the signal levels of WG, WR and WB are equal before the A / D conversion, so that the threshold value of the binarization processing in the signal processing circuit 36B is used. The level may be the same value for R, G and B.

Next, when the R, G, and B white balances are not obtained, or when binarization is performed without white balance, the values of R, G, and B are checked to determine the threshold value of the binarization processing. Levels are set separately.

Next, another embodiment related to FIG. 45 will be described with reference to FIG. According to this embodiment, the imaging signal output from the first imaging module including the optical lens 40, the quartz optical filter 41, and the color solid-state imaging device 42 is subjected to processing such as preamplification, white balance, and gamma correction by the preprocessing circuit 34. Is performed, the data is converted into a digital value by the A / D converter 35, and the signal processing circuit 36
A generates a luminance signal and a color-difference signal and compresses the data, and the result is recorded on a recording medium 37 such as a semiconductor memory.

Here, the first imaging module is detachable from the electronic still camera, and can be replaced with the second imaging module. For example, the image may be black and white, and the second image pickup module is used in the case of photographing where importance is placed on the resolution. The second imaging module includes an optical lens 43, a quartz optical filter 44, and a black-and-white solid-state imaging device 45.
The image signal output from the imaging module is output to the preprocessing circuit 3
After being subjected to a predetermined process by 4, the digital value is converted into a digital value by the A / D converter 35, black-and-white processing is performed by the signal processing circuit 36 B, and is recorded on the recording medium 37.
If the number of pixels of the color solid-state imaging device 42 of the first imaging module is equal to the number of pixels of the monochrome solid-state imaging device 45 of the second imaging module, the drive circuit 32 and the synchronization signal generator 33 may be exactly the same circuit.

[0135] By making the imaging module detachable from the electronic still camera in this way, it is possible to perform imaging in accordance with the purpose. In addition, by including the driving circuit 32 and the signal generator 33 in the imaging module, the imaging module can use a solid-state imaging device (for example, H
The solid-state imaging device for D) can be replaced with an imaging module.

Next, with reference to FIG. 49, a method of using an electronic still camera system capable of reproducing a reproduced image in a wider range than a standard TV monitor and the concept of the system will be described below.

FIG. 14 shows a digital electronic still camera 50 having a reproducing function using the memory card 51 as a recording medium and a state in which a photographed subject is reproduced on a normal TV monitor 54. The output range of the conventional electronic camera system is fixed, but in the system of the present invention, the portion indicated by the dotted line is the range of the reproduced image that can be output. The operator operates the play button 52 and the direction instruction button 53,
Within this possible output range, the output range can be changed in the horizontal direction. Hereinafter, this image output method is referred to as a panorama mode.

Next, a description will be given of a photographing method which makes it possible to reproduce a panorama. By reading a signal at a position corresponding to a half of the vertical pixel pitch in the image formed by the subject using the method as shown in FIG. 1 or FIG. The number of pixels is doubled.

An example of the photographed image at this time is shown in FIG. Normally, the number of effective vertical pixels of the TV monitor 54 is Nv
FIG. 51 has a vertical pixel number of 2Nv. Signal processing is performed using an arbitrary half in the vertical direction in FIG. 51 as an effective image, and is recorded on a recording medium, for example, a memory card 51. In the example of FIG. 51, a portion that is not a diagonal line is selected. In this way, it is possible to obtain a recorded image having an area where the image output range extends in the horizontal direction as shown in FIG.

Next, the reproducing means will be described. FIG.
3 is a block diagram of a reproducing unit 70 of the electronic camera 50. The still image data recorded in the memory card 51 is transferred to a still image reproducing unit of the electronic camera 50 via an interface (IF) 71 between the card and the camera. Data is,
The data is recorded in the image output memory circuit 73 via the digital reproduction signal processing circuit 72. The output data of the image output memory circuit 73 is converted into an analog signal by the D / A converter 74, and is output to the monitor 54 via the analog signal processing unit 75.

Here, it is assumed that the output rate when outputting a normal monitor image is fs, and the number of output pixels in the horizontal direction at this time is Nh as shown in FIG. The horizontal effective image required for the panorama mode image is twice Nh. The horizontal input pixel at the time of panorama shooting is Nh, and if this is output at a rate of fs, a correct output image cannot be obtained. In order to output a correct image having an aspect, output and reproduction are performed at a rate of fs / 2 or the output memory circuit 7 shown in FIG.
As shown in FIG. 3, a horizontal interpolation circuit 79 is provided at the subsequent stage of the image output memory 78 to supplement the effective data by interpolation and output the data at a rate of fs. FIG. 59 shows an example of the address space of the image output memory 78 at this time, and the image output memory 78 needs a capacity of Nh × Nv pixels. The hatched portion in FIG. 59 shows an example of the memory portion which is selected and outputs an image.

In this example, the image portion substantially at the center of the image output possible range is selected, and the operator selects the left / right direction of the direction instruction button 53 shown in FIG. 49 to move the image output range. be able to. The selection of the image output direction by the operator is transmitted to the horizontal address counter 76 which generates the address of the image output memory circuit 73 in FIG. 55, and can be changed by changing the start address of the horizontal address of the image output memory 78. You.

FIG. 56 shows an example in which a horizontal interpolation circuit 79 is provided in front of the image output memory 78. At this time, the memory capacity of the image output memory 78 needs to be twice as large in the horizontal direction as shown in FIG.

The embodiment in which the number of pixels in the vertical direction is doubled has been described above.

Next, the case where the number of pixels in the horizontal direction is increased in the same manner as in the vertical direction will be described. This is HD
A recording mode may be used. In the example of FIG. 51, the vertical is 2N
In the present embodiment, the input is 2Nv pixels in the vertical direction and 2Nh pixels in the horizontal direction. Of these, half the pixels in the vertical direction are selected, subjected to signal processing, and stored in a recording medium. The image data on the recording medium is reproduced by the reproducing unit having the image output memory circuit 73 shown in FIG. The address space of the image output memory 78 at this time is shown in FIG.
Same as 0.

Next, FIG. 61 shows another example of the image output memory address space in which the capacity of the image output memory 78 can be reduced. The image output memory 78 has a memory space for Nv pixels vertically and Nh + Na pixels horizontally. Here, 0 ≦ Na <Hh. In addition to the Nh pixels for the side selected for image output, the side
A memory for a pixels is provided, and image data in the direction designated for the Na pixels is newly supplied from the recording medium through the signal processing circuit, and is supplementarily recorded. Thereafter, the address start position in the horizontal direction is moved. Then, the output image is moved.
For this reason, the horizontal address counter selects the head address next to the last address in the horizontal direction so that the address follows. A block diagram of the image output memory circuit 73 at this time is shown in FIG.

As described above, the amount of the image output memory 78 can be reduced by supplementing the reproduced image data as needed. When a block coding technique such as a still image cosine transform is used for signal processing, the memory can be used more efficiently if Na is set to the horizontal blocking length or a multiple thereof.

In the above, the embodiment in which the image forming position of the subject on the image sensor 63 is moved and the signal reading is performed a plurality of times has been described. Next, another embodiment in which the imaging position is not moved will be described.

FIG. 62 shows an example in which an image sensor having a wide angle of view in the horizontal direction is used. The pixel used in normal shooting is the shaded area at the center, and the entire pixel is used in the panorama mode. The number Nb of pixels in the horizontal direction is not particularly limited, and may be Nb> Nh.

FIG. 63 shows whether the playback screen is a normal photographed image or not.
An image in which a mark is added to a panoramic playback image is shown in order to distinguish the image captured in the panorama mode. “P” at the upper right of the reproduced image indicates that it is a panoramic image. When reading the data from the memory card, the playback unit of the electronic camera reads whether the data is a normal image or a panoramic image from the image control data recorded in the memory card without reading the data. Select the signal processing. As one process, when the output image is a panoramic image, a mark indicating this is shown in the reproduced image.

FIG. 64 shows directions in which images can be output. In this case, images can be output in both directions. In FIG. 65, there is no more image to the right,
The direction mark on the screen indicates that the user can move only to the left. The panorama mark and the direction mark are generated by a mark generation circuit 81 composed of a digital circuit as shown in FIG.
The signal is generated by a mark generation circuit 83 composed of an analog circuit and mixed by an analog mixing circuit 84. FIG. 68 shows an example in which a display unit is provided on the operation unit of the electronic camera to indicate the possible image output direction to the operator. In this example, the direction instruction button 53a
And 53b are provided with a light emitting section, and the light emission allows the operator to know the image output possible direction. In this figure, only the left direction is lit, and it can be seen that only the left direction movement is possible.

In order to notify the photographer by displaying the normal photographing mode or the panoramic mode at the time of photographing, the photographing is displayed on the liquid crystal display panel of the camera body or in the viewfinder as shown in FIG. “P” at the upper right of the display in the viewfinder of FIG. 69 indicates the panorama mode.
“P” may be located in the viewfinder image display. In normal photographing, the entire viewfinder is an effective image, and in the panorama mode, the effective image area is the central portion excluding the upper and lower hatched portions. As a means for distinguishing the shaded portion, a shielding cover can be considered in the real image finder. In the electronic finder,
By raising or lowering the luminance signal level, or by raising or lowering the level of some color signals, eg, one of RGB signals, or two types of signal,
It can be distinguished from the central part. FIG. 70 shows a display in the viewfinder when the image sensor having a wide horizontal angle of view shown in FIG. 62 is used.

In the above description, the panorama function in the horizontal direction has been described. In the example of up, down, left, and right, half the vertical effective range in FIG.
FIG. 71 is a block diagram of an image output memory unit 73 according to an embodiment in which an image sensor having Nv pixels in the vertical direction and Nh pixels in the horizontal direction is provided at two positions in each of the vertical and horizontal directions, and the apparent number of pixels is quadrupled by changing the imaging position.
FIG. 72 shows an example of image output. As shown in FIG. 71, the vertical address counter 77 also has a horizontal address counter 76.
Direction information is input as shown. In the example of FIG. 72,
This indicates that it can be moved in any direction, up, down, left and right. In this case, an element having a longer horizontal direction and having a larger number of pixels than an image pickup element for a normal TV may be used.

As such an element, for example, a high-vision image pickup element having a number of vertical pixels approximately twice as large as a normal number can be considered.

The embodiment shown in FIG. 73 is an electronic still camera capable of adding position information indicating a photographing place to photographing condition data. This electronic still camera includes a solid-state imaging device (CCD) 91, an imaging signal processing circuit 92, a digital signal processing circuit 93, an imaging condition data recording circuit 94 including position information, a memory card interface 95, and a memory card 96. .

The position data indicating the photographing place is generated inside the camera, or the position data is inputted from the outside, and is recorded together with the image data on the recording medium through the photographing condition data recording circuit 94. The embodiment shown in FIG. 73 is an example of a digital electronic camera system for recording image signals digitally. As a recording medium, a memory card 96 is used. The following example is also described using a digital electronic still camera as an example.

As a means for generating position data, a global positioning system (global positioning system,
GPS (GPS) location data, mobile phone relay station location data, FM broadcasts, AM broadcasts, and other radio broadcasts and TV broadcasts that can be received at the shooting location, keyboard input Or a method using a pen input board.

FIG. 74 shows a state in which the GPS device and the electronic camera are connected and position data is being input. GP at the same time as shooting or after shooting
The data is read from the S device and recorded in the memory card in a manner linked with the image data. FIG. 75 shows a playback unit of the electronic camera;
Alternatively, it is a block diagram of a playback unit of an electronic album having a medium for storing and storing a large amount of image data captured by an electronic camera. The playback unit includes an image data storage unit 101, an image data storage unit interface 102, and a playback signal processing circuit 10.
3, image display unit 104, CPU 105, input unit 106,
It comprises a search question display section 107 and a search data storage section 108.

As the medium for storing a large amount, an optical disk, a magneto-optical disk, a hard disk, a magnetic tape, a bubble memory and the like can be considered. In this reproducing unit, a person searching for an image inputs the name of the shooting location by input means, pulls out the position data from the search data storage unit that stores the position data of the location, and stores the position data and the recorded image data. After comparing the position data, an image with a small error is displayed as a high possibility. This image display method enables highly efficient image retrieval.

FIG. 76 shows a method of connecting a portable telephone and an electronic still camera and inputting position data. A line is connected to a relay station closer to the mobile phone, and data for identifying the relay station is input. By having the position information of the identification data station in the reproducing unit, the approximate shooting location can be known. By using the identification data of a plurality of relay stations, the shooting location can be specified more.

In the examples of FIGS. 74 and 76, the electronic still camera inputs position data from an external system.
This function may be provided in the camera.

FIG. 77 shows an example in which a radio wave and / or a television wave are used as the input means of the position data. The tuning frequency of the internal or external tuner 98 is selected from a plurality of predetermined frequencies in the frequency data memory 97, and when a tuning output is generated, the output frequency is converted into data and passed through the photographing condition data circuit 94. The recorded data is stored in the memory card 96.

The search data storage section of the playback section stores frequency data of broadcast stations and relay stations. By comparing the frequency data with the tuning frequency data in the memory card 96, it is possible to infer the location where the image was taken. Can be.

The present invention is based on Japanese Patent Application No. Hei.
2 It is also possible to use together with the method using many photographing condition data described in “Digital electronic still camera system”. FIG. 78 is an example of a search screen in the playback unit, in which position data is used together with other shooting condition data. The searcher answers the questions on the screen, and the reproducing unit calculates the possibility values of each image based on the answers, compares these values, and outputs the image with the highest possibility value. FIG. 79 shows the state of the CPU 105 performing the calculation. In response to the question, the photographing condition data of each image required for the calculation is read and substituted into a predetermined calculation expression.
Each calculation result is put together by a comprehensive operation, and becomes a possible numerical value of each image. This numerical value becomes a numerical value for comparison.

In the above embodiments, a digital system has been described, but the present invention can be similarly applied to an analog system.

The embodiment shown in FIG. 80 is an electronic camera system having an enlargement function and a compression function. The control flow of this system is shown by the flowchart in FIG. In FIG. 80, an imaging unit 201 includes an optical system including a lens, a shutter, and the like, and a photoelectric conversion element such as a CCD that photoelectrically converts an image formed by the optical system. The image signal output from the image sensor 201 is input to the signal processing unit 202 and is subjected to γ correction, white balance adjustment, and the like. The output of the signal processing circuit 202 is A
The signal is subjected to analog-digital conversion by the / D converter 203 and input as digital image information to a digital signal processing circuit 204 including an interpolation processing unit.

When the photographer selects the enlarged image pickup mode at the time of photographing, the image is electronically enlarged by interpolation processing such as linear interpolation in the interpolation processing section. The image is input to the enlarged image compression circuit 205. As a compression method,
A coding method for a color still image, which is a type of block coding, such as JPEG, may be considered. The video compressed by this method is recorded on a video pack 206 such as a memory pack, for example, an IC card, which is detachably stored and connected to the electronic camera body.

As described above, in the present electronic zoom system, by performing image compression after performing interpolation processing, image quality deterioration due to enlargement of block distortion due to block coding is eliminated. Since the high-frequency component is suppressed, the occurrence of block distortion due to image compression is suppressed.

FIGS. 82, 83A to 83F show an example of JPEG which is a block data compression system.
This will be described with reference to FIG. In JPEG, FIG.
A is divided into 8 × 8 pixel blocks as shown in FIG. 83B, and discrete cosine transform (D
CT). High-efficiency compression is realized by reducing high-frequency components by using a quantization table that roughly quantizes high-frequency frequencies of DCT coefficients obtained by DCT. But,
For this reason, in the decoded image, as shown in FIGS. 83C and 83D, discontinuity of a frequency component occurs at a block boundary, and block distortion occurs. For this reason, if the interpolation processing is performed after the compression, the block distortion due to the block coding is enlarged as shown in FIGS. 83E to 83F, so that it becomes easy for human eyes to notice, and the image quality is remarkably deteriorated.

On the other hand, in the interpolation processing, as shown in FIG. 84, a low-pass characteristic is used.
and the amount of information in the high-frequency area at the time of image compression is reduced. Also, as described above, the blocked data compression method is realized by reducing information in a high frequency region. Since the information amount of the high-frequency components has already been reduced by the interpolation process, the influence of the reduction of the high-frequency components in the compression process is hardly affected, so that the occurrence of block distortion is suppressed.

As described above, according to the method of the present invention, it is hard to be affected by block distortion due to block coding, and deterioration of image quality can be suppressed.

[0172] As a matter of course, regarding the zoom, a combination of the optical system zoom and the electronic zoom may be considered.

FIG. 85 shows an embodiment provided with a function of changing the compression ratio according to the zoom magnification and a function of confirming the captured image. The electronic still camera of this embodiment has a controller 210 as a user interface, and the photographer can select a function of the camera system, for example, a zoom function or an image confirmation function. The selected processing content is displayed on the display unit 211 so that the photographer can confirm it.

The zoom magnification is selected by the photographer by the controller 210. When the zoom magnification is increased, in the case of electronic zoom, information of high frequency components on the spatial frequency decreases. Therefore, even if the compression rate of block coding for roughly quantizing high frequency components is increased, the effect on image quality is small. The zoom ratio at the time of shooting is input to the compression ratio selector 212, and as the zoom ratio increases, a higher compression ratio is selected. Thereby, the storage memory can be saved without deteriorating the image quality.

When the function of confirming the image captured by the controller 210 is selected, the compression unit processing circuit 205
Alternatively, image information is passed from the recording medium 206 to the decoder 213, and the decoded image is stored in the frame memory 214.
Is accumulated in This image information is output to an image output monitor 215 or an external monitor terminal 216 provided in the present imaging apparatus. This makes it possible to check the quality of the captured image before recording or storing it on an optical disk, magnetic disk, digital tape medium, or the like, or before printing out.

This embodiment is an example of an electronic camera that handles digital still images. However, if a digital moving image camera uses compression accompanied by deterioration due to block coding, an effective method can be used. It can be said that

FIG. 86 shows an image pickup apparatus according to an embodiment having a function of selecting processing suitable for each of low magnification and high magnification in the enlarged imaging mode and processing suitable for each. 6 is a flowchart for controlling the photographing device.

When the photographer selects the enlarged image pickup mode at the time of photographing, the compression / cutout processing selecting section 221 judges whether or not to perform compression.
Image compression processing circuit 2 without performing compression processing by
22 outputs are selected. The processing selection unit 221 adaptively selects a processing from the compression ratio of the compression processing unit and the zoom magnification.

As the zoom magnification is increased, the number of actual pixels in the area of the zoom image becomes very small. Therefore, if only this area is recorded, the amount of data can be reduced and the storage memory can be saved. it can. Further, the interpolation image is not degraded due to distortion on the image caused by the compression. According to the above-described method, the storage memory can be saved, and the image quality of the zoom image is improved.

FIG. 89 shows a compression / cutout processing selection unit 2
21 illustrates the operation principle. The input from the user is input to the process selection unit 221 according to the compression ratio and the zoom ratio. The processing selection unit 221 has a function f (x) therein, and when the compression ratio is x and the zoom ratio is y, if y ≦ f (x), the zoom ratio is lower than the compression ratio. The compression process is selected. If y> f (x), the clipping process is selected because the zoom ratio is higher than the compression ratio. An example of the internal function f (x) is f (x) = 1/2,
This function is given to the system in advance. With this device, adaptive processing can be performed, and the best image can always be obtained within the selected conditions.

FIG. 88 shows an example of an image format.
Shown in The header portion has additional information such as the size of the image and the shooting date and time, and also has a flag indicating whether the image is compressed or cut out and needs to be zoomed.

On the other hand, as shown in FIG.
The third process selection unit 224 selects a reproduction process based on the flag. If it is compressed, it is decoded by the decoder 213 and the result is stored in the frame memory 214 and output to the monitor 215 or the like. If it has been cut out, the electronic zoom processing circuit 225 performs zooming on the image data stored in the recording medium by interpolation processing such as the above-described two-point interpolation or higher-order interpolation.

Although the embodiments of FIGS. 53 to 59 have described the invention for application to a still image system, the application to a moving image system will be described below.

FIG. 91 is a block diagram showing a schematic configuration of a digital camera system according to an embodiment for digitally recording a moving image using the electronic zoom system. Figure 91
, The imaging unit 201 includes a photoelectric conversion element such as a CCD for photoelectrically converting an image formed by an optical system including a lens and the like. The image signal output from the image sensor 201 is passed to a signal processing circuit 202, where γ correction and white balance adjustment are performed. This signal processing circuit 20
The output of 2 is subjected to analog-to-digital conversion by the A / D converter 203 and input as digital image information to a digital signal processing circuit 204 including an interpolation processing unit. When the photographer selects the magnified imaging mode at the time of photographing, the electronic zoom processing circuit 207 electronically magnifies the image by interpolation processing such as linear interpolation. The enlarged video is input to the image compression circuit 205.

As a compression method, MPEG, which is a moving image coding method which is one type of block coding, can be considered. The video compressed by this method is recorded via a recording signal processing circuit 301 and an amplifier 302 on a video recording medium such as a digital tape 303 or an optical disk which is detachably housed and connected to the camera system main body.

As described above, in the present electronic zoom system, since the image compression is performed after the interpolation processing by the electronic zoom processing circuit 207, the image quality deterioration due to the enlargement of the block distortion due to the block coding is prevented. In addition, since the interpolation process suppresses the high frequency components of the image, the occurrence of block distortion due to the image compression is suppressed.

Further, as shown in FIG. 92, the present camera system is provided with a communication interface 309 so that the communication path 311 and the codec 31
2. The present invention can be used for a TV conference system and a TV telephone that output an image to a TV monitor 314 installed in a remote place via a communication interface 313.

As described above, when the present electronic zoom system is used in a moving image system, image quality deterioration due to block coding is suppressed, and a compression process saves a storage memory in storage, and in communication, Communication volume can be reduced.

[0189]

(1) As described in detail above, according to the electronic still camera of the present invention, as compared with the conventional electronic still camera, a detailed mode (referred to as HD mode) by the moving means is provided as an additional function. With such an arrangement, it is possible to obtain image quality that can be output to an HD monitor, and it is possible to obtain a hard copy with higher image quality. Further, size reduction can be realized as compared with a high-resolution still image input device.

(2) According to the electronic still camera of the present invention, a binarization mode is provided as an additional function as compared with a conventional electronic still camera, so that only binary data such as photographing characters is required. Thus, a high-resolution binary image can be obtained with a small amount of data.

Further, by using an electronic still camera in which the image pickup module can be replaced, it is possible to perform image pickup in accordance with the purpose.

(3) According to the present invention, a new function can be added to the playback function of an electronic camera for inputting a still image, and the electronic camera can be enjoyed and used in various ways.

(4) According to the present invention, a desired image can be easily and efficiently output without attaching a title or a key forward to each of a large number of images recorded on a recording medium.

(5) According to the present invention, it is possible to realize an imaging apparatus capable of performing interpolation processing and image compression with little deterioration in image quality. Further, according to the present invention, it is possible to realize an imaging apparatus which does not cause image quality deterioration and can save storage memory.

[Brief description of the drawings]

FIG. 1 is a block diagram of an electronic still camera according to a first embodiment of the present invention.

FIG. 2 is a block diagram of an electronic still camera according to a second embodiment of the present invention.

FIG. 3 is a block diagram of an electronic still camera according to a third embodiment of the present invention.

FIG. 4 is a block diagram of an electronic still camera according to a fourth embodiment of the present invention.

FIG. 5 is a diagram for explaining a color filter array used in the present invention and a moving state thereof.

FIG. 6 is a block diagram showing a digital signal processing circuit in the electronic still camera of the present invention.

FIG. 7 is a diagram showing a schematic configuration of a solid-state imaging device.

FIG. 8 is a diagram for explaining a state of charge accumulation when the solid-state imaging device of FIG. 7 is used in an HD mode.

FIG. 9 is a diagram for explaining another state of charge accumulation when the fixed imaging device in FIG. 7 is used in the HD mode.

10 is a diagram showing an equivalent color filter array when the solid-state imaging device having the color filter array shown in FIG. 5A is used in the HD mode.

FIG. 11 is a diagram for explaining a two-dimensional address of a buffer memory in the case of an equivalent color filter array in the HD mode shown in FIGS. 10A and 10B.

FIG. 12 is a diagram for explaining another color filter array used in the present invention and its moving state.

13 is a diagram showing an equivalent color filter array when the solid-state imaging device having the color filter array shown in FIG. 12A is used in the HD mode.

FIG. 14 is a diagram for explaining another color filter array used in the present invention and its moving state.

FIG. 15 is a diagram showing an equivalent color filter array when the solid-state imaging device having the color filter array shown in FIG. 14A is used in the HD mode.

FIG. 16 is a diagram for explaining another color filter array used in the present invention and its moving state.

FIG. 17 is a diagram showing an equivalent color filter array when the solid-state imaging device having the color filter array shown in FIG. 16A is used in the HD mode.

FIG. 18 is a diagram for explaining the same color filter array and its moving state as in FIG. 16A.

FIG. 19 shows another HD with the color filter array shown in FIG. 18A.
FIG. 7 is a diagram showing an equivalent color filter array when used in a mode.

FIG. 20 is a diagram showing a color filter array of a color solid-state imaging device used in the electronic still camera according to the present invention.

FIG. 21 is a diagram showing another color filter array of a color solid-state imaging device used in the electronic still camera according to the present invention.

FIG. 22 is a diagram showing another color filter array of a color solid-state imaging device used in the electronic still camera according to the present invention.

23. F1 of the color filter array shown in FIG.
FIG. 6 is a diagram showing a color filter array when color filters of Ye (yellow), F2 and Cy (cyan) are used for F2.

FIG. 24 is a diagram showing a state of a swing of the color solid-state imaging device in the HD mode according to the present invention.

25 is a diagram showing an equivalent color filter array when the color solid-state imaging device having the color filter arrays shown in FIGS. 20 and 21 is used in the HD mode shown in FIG. 24;

26 is a diagram showing an equivalent color filter array when the color solid-state imaging device having the color filter array shown in FIG. 23 is used in the HD mode in FIG. 24;

FIG. 27 is a diagram showing another swing state of the color solid-state imaging device in the HD mode according to the present invention.

FIG. 28 is a diagram showing an equivalent color filter array when the color solid-state imaging device having the color filter arrays shown in FIGS. 21 and 22 is used in the HD mode shown in FIG. 27;

29 shows a color filter array applicable to the present invention, wherein F1 of the color filter array shown in FIG. 20 is G (green);
FIG. 9 is a diagram showing a color filter array when F2 is an R (red) color filter element and F3 is a Cy (cyan) color filter element.

30 shows a color filter array applicable to the present invention, wherein F1 of the color filter array shown in FIG. 20 is G (green);
The figure which shows the color filter arrangement | sequence when F2 is Ye (yellow) and F3 is a color filter element of Cy (cyan).

31 is a color filter array applicable to the present invention. In the color filter array shown in FIG. 20, F1 is Y (a color filter having a spectral transmission characteristic of a luminance signal), F2 is R (red), and F3 is B The figure which shows the color filter arrangement | sequence when it is set as a (blue) color filter element.

FIG. 32 is a diagram showing a color filter array of a color solid-state imaging device for explaining another embodiment of the present invention.

FIG. 33 is a diagram showing a state in which the color solid-state imaging device having the color filter array shown in FIG. 32 swings in the HD mode.

34 is a diagram showing an equivalent color filter array when the color solid-state imaging device having the color filter array shown in FIG. 32 is used in the HD mode shown in FIG.

FIG. 35 is a diagram showing a state in which the color solid-state imaging device having the color filter array shown in FIG. 32 swings in the HD mode.

FIG. 36 is a diagram showing an equivalent color filter array when the color solid-state imaging device having the color filter array shown in FIG. 32 is used in the HD mode shown in FIG. 35;

FIG. 37 is a diagram showing another color filter array applicable to the present invention.

FIG. 38 is a diagram showing another color filter array applicable to the present invention.

FIG. 39 is a diagram showing another color filter array applicable to the present invention.

FIG. 40 is a diagram showing a state of a swing of eight positions in the HD mode according to the present invention.

FIG. 41 is a diagram showing a state of a swing of another eight positions in the HD mode of the present invention.

FIG. 42 is a diagram showing an equivalent color filter array obtained when the swing shown in FIGS. 40 and 41 is performed.

FIG. 43 is a diagram showing another color filter array applicable to the swing shown in FIG. 40;

FIG. 44 is a diagram showing another color filter array applicable to the swing shown in FIG. 41;

FIG. 45 is a block diagram of an electronic still camera according to another embodiment, in which color moiré is prevented.

FIG. 46 is a diagram showing a color filter array for explaining the embodiment in FIG. 45;

FIG. 47 is a diagram showing an equivalent color filter array when the solid-state imaging device having the color filter array of FIG. 46 is regarded as a black and white device.

FIG. 48 is a block diagram of an electronic still camera according to another embodiment related to the embodiment of FIG. 45;

FIG. 49 is a view for explaining functions of an electronic still camera according to another embodiment capable of changing an image output range.

FIG. 50 is a diagram showing a mechanism for moving a subject image forming position on a solid-state image sensor in the embodiment of FIG. 49;

FIG. 51 is a view for explaining an image capturing range in the embodiment.

FIG. 52 is a view showing another example of the subject image forming position moving mechanism.

FIG. 53 is a block diagram showing a reproduction signal processing system in the embodiment.

FIG. 54 is an exemplary view for explaining the number of pixels required for reproduction on a monitor in the embodiment.

FIG. 55 is a block diagram of a first example of the image output memory unit in FIG. 53;

FIG. 56 is a block diagram of a second example of the image output memory unit in FIG. 53;

FIG. 57 is a block diagram of a third example of the image output memory unit in FIG. 53;

FIG. 58 is a block diagram of a fourth example of the image output memory unit in FIG. 53;

FIG. 59 is a view showing a first example of an address space of an image output memory;

FIG. 60 is a view showing a second example of the address space of the image output memory.

FIG. 61 is a view showing a third example of the address space of the image output memory.

FIG. 62 is a diagram of an image sensor having a longer field angle than a normal image sensor.

FIG. 63 is a view showing an example of a playback screen indicating a panorama mode.

FIG. 64 is a view showing an example of a playback screen indicating that the playback range can be moved in both the left and right directions.

FIG. 65 is a view showing an example of a playback screen indicating that the playback range can be moved only in the left direction.

FIG. 66 shows a first example of a reproducing unit incorporating a mark generation circuit.
Block diagram of an example.

FIG. 67 shows a second example of the reproducing unit incorporating the mark generation circuit.
Block diagram of an example.

FIG. 68 is a diagram showing an example of an electronic still camera having a display unit capable of moving a reproduction range in an operation unit.

FIG. 69 is a view showing an example of display in a viewfinder of an electronic still camera having a panoramic mode.

FIG. 70 is a view showing another example of the display in the viewfinder of the electronic still camera having the panorama mode.

FIG. 71 is a block diagram of an image output memory section of an electronic still camera having an image output range of up, down, left, and right.

FIG. 72 is a diagram showing an example of an output image indicating that the image has an upper, lower, left, and right image output range;

FIG. 73 is a block diagram of a recording unit of an electronic still camera having a search function according to another embodiment.

FIG. 74 is an exemplary view showing an example in which the electronic still camera of the embodiment is connected to a GPS receiver and used;

FIG. 75 is a block diagram of a playback unit of the electronic still camera according to the embodiment or a playback unit of an electronic album having a medium for recording image data captured by the electronic still camera.

FIG. 76 is a view showing an example in which the electronic still camera according to the embodiment is used as a mobile phone.

77 is a block diagram of the electronic still camera according to the embodiment related to FIG. 73.

FIG. 78 is a view showing an example of a question screen for retrieving image data shot by the electronic still camera of the present invention.

FIG. 79 is a block diagram for performing an image search operation.

FIG. 80 is a block diagram of an electronic still camera having a zoom function and a compression function according to another embodiment.

FIG. 81 is a flowchart showing a flow of control in the embodiment.

FIG. 82 is an exemplary view showing a block image data compression method used in the embodiment.

FIG. 83 is an exemplary view for describing features of the block image data compression method;

FIG. 84 is a view showing a video signal band characteristic and a band characteristic of a signal obtained by a two-point interpolation method.

FIG. 85 is a block diagram of an imaging apparatus according to another embodiment related to FIG. 80;

FIG. 86 is a block diagram of an imaging apparatus according to another embodiment.

FIG. 87 is a flowchart showing the flow of control in the embodiment.

FIG. 88 is a view showing an image format.

FIG. 89 is a view showing the operation principle of the compression / cutout selection processing unit in FIG. 86;

FIG. 90 is a block diagram of an imaging apparatus according to another embodiment related to FIG. 80;

FIG. 91 is a block diagram of an imaging apparatus according to another embodiment related to FIG. 80;

FIG. 92 is a block diagram of an imaging apparatus according to another embodiment related to FIG. 80;

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Color solid-state image sensor 2 ... Image sensor drive circuit 5 ...
A / D converter 6 ... Buffer memory 7 ... Memory control circuit 8,36,204 ... Digital signal processing circuit 9,3
7, 206 recording medium 11 piezoelectric element 12 refraction plate 31, 42 color solid-state image sensor 32 image sensor drive circuit 34 pre-processing circuit 35 A
/ D converters 38, 41, 44 ... Quartz optical filters 40, 43 ... Optical lens 45 ... Monochrome solid-state image sensor 61 ... Lens 62 ... Transparent refraction plate 63, 91 ... Solid-state image sensor 64 ... Image sensor drive mechanism 94 ... Photographing conditions Data recording circuit 105 CPU 10
7. Search question display section 201 Imaging section 205 Compression processing circuit 207 Electronic zoom processing circuit 221 Processing selection section 222 Cutout processing circuit

 ────────────────────────────────────────────────── ─── Continuing from the front page (72) Yoshitomo Tagami, Inventor 1 Tokoba, Komukai Toshiba-cho, Kawasaki-shi, Kanagawa Prefecture Toshiba Research & Development Center Co., Ltd. No. 1 Toshiba R & D Center (72) Inventor Kazuhiro Takashima No. 1 Toshiba R & D Center (72) Inventor Mitsuo Nagareishi Komukai Koyuki, Kawasaki City, Kanagawa Prefecture No. 1 Toshiba-cho Toshiba Corporation Research and Development Center (56) References JP-A-3-231589 (JP, A) JP-A-62-226789 (JP, A) JP-A-3-78388 (JP, A)

Claims (19)

(57) [Claims] 1. A plurality of colors arranged in a horizontal and vertical axis direction.
A filter, a solid-state imaging device having pixels at positions corresponding to the respective color filters , and outputting an image signal corresponding to the incident optical image, and an optical image moving from the first position without movement and from the first position. A second position moved by one pixel in one direction of the horizontal or vertical axis direction, a third position moved by half a pixel in both the horizontal and vertical axis directions from the first position, and a first position;
Moving means for moving between a second position and a fourth position forming four vertices of a parallelogram together with the third position, an image signal at each of the first, second, third and fourth positions; An electronic still camera, comprising: 2. The apparatus according to claim 1, further comprising signal processing means for generating a still image based on the image signals at the first, second, third, and fourth positions read by said reading means. 2. The electronic still camera according to 1. 3. The moving means moves an optical image from a first position to one of second, third, and fourth positions,
3. The electronic still camera according to claim 2, wherein the electronic still camera is moved to one of the remaining two positions and then moved to the remaining position. 4. The electronic still camera according to claim 3, wherein said moving means returns the optical image to a first position after moving. 5. The solid-state imaging device according to claim 1, wherein said moving means moves an optical image, and said solid-state imaging device stores electric charges in a charge storage section of said solid-state imaging device.
Electronic still camera according to claim 1, further comprising a means for out sweep. 6. The solid-state image pickup device according to claim 1, wherein the charge accumulated in said solid-state image pickup device is transferred while said moving means moves an optical image.
Electronic still camera according to claim 1, further comprising a means for out sweep. 7. The electronic still camera according to claim 1, wherein said moving means has a parallel flat plate disposed in front of a solid-state image sensor for changing a path of an optical image. 8. The color filter according to claim 1, wherein the color filter is 2 rows and 2 columns or 4 rows and 2 columns.
A mosaic color filter having a unit of 2 columns or 2 rows and 4 columns is sequentially arranged in the horizontal and vertical axis directions.
2. The electronic still camera according to claim 1, comprising a color filter of four colors. 9. A color obtained by the solid-state imaging device at the first, second, third, and fourth positions, wherein a green color or a color containing a green component is water.
To be arranged in a checkered pattern every half pixel in the vertical direction,
The electronic still camera according to claim 8, wherein the color filters are arranged . 10. A plurality of light emitting devices arranged in horizontal and vertical axis directions.
A color filter, a solid-state imaging device having pixels at positions corresponding to the respective color filters , and outputting an image signal corresponding to an incident optical image; a first position where the solid-state imaging device does not move;
, A second position shifted by one pixel in one direction in the horizontal or vertical axis direction, a third position shifted by half a pixel in both the horizontal and vertical axis directions from the first position, and a first position. Moving means for moving between a position, a second position, and a fourth position forming four vertices of a parallelogram with the third position; and a moving means for moving the first, second, third, and fourth positions. An electronic still camera, comprising: means for reading an image signal. 11. A signal processing means for generating a still image based on image signals at the first, second, third and fourth positions read by said reading means. 10. The electronic still camera according to 10. 12. The moving means comprises: a solid-state image sensor;
From the second position to one of the second, third, and fourth positions, then to one of the remaining two other positions, and then to the remaining position. Claim 1
2. The electronic still camera according to 1. 13. The electronic still camera according to claim 12, wherein said moving means returns the solid-state imaging device to a first position after moving. 14. The solid-state imaging device, while said moving means moves the solid-state imaging device according to claim 10, characterized in that it comprises a means for out sweeping charge accumulation portion <br/> of charge in the solid-state imaging device An electronic still camera as described. 15. The solid-state image pickup device according to claim 1, wherein said solid-state image pickup device is stored in said solid-state image pickup device while said moving means moves said solid-state image pickup device.
Electronic still camera according to claim 10, wherein further comprising means for out sweeping charges were. 16. An electronic still camera according to claim 10, wherein said moving means electrically moves said solid-state imaging device. 17. The electronic still camera according to claim 16, wherein said moving means moves said solid-state image pickup device using a piezoelectric element. 18. The color filter includes a mosaic color filter having a unit of 2 rows and 2 columns, 4 rows and 2 columns, or 2 rows and 4 columns, which are sequentially arranged in the horizontal and vertical axis directions. 11. The electronic still camera according to claim 10, comprising a four-color filter. 19. The first, second, third, colors including green or green color components obtained by the solid-state image pickup device in the fourth position is
It is arranged in a checkered pattern every half pixel in the horizontal and vertical directions.
19. The electronic still camera according to claim 18 , wherein the color filters are arranged .