JP2002209100A - Image processor and image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processor and an image processing method specific to so-called a honeycomb layout which enable maintenance of high image quality even when an image size is reduced. SOLUTION: A digital camera 10 gives pixel data corresponding to a G square lattice pattern and RB complete checkered patterns to a signal processing section 30, a reduction correction section 46 of the signal processing section 30 generates pixel data in desired colors by regarding object pixels as real pixels or virtual pixels, uses the pixel data of the object pixels for representative data of surrounding pixel data and obtains the data having the same substantial spatial frequency distribution by considering green pixel layout configuring the structural image while reducing the number of pixels in each colors used as a whole and a broadband processing section 30c applies frequency broadband processing to the pixel data obtained by the reduction correction section 46 to maintain the image quality of the obtained image.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing apparatus and an image processing method, and more particularly to inputting, for example, millions or more of pixel data, performing image processing on the obtained pixel data, and outputting the image data. It is suitable for an image input device such as a digital camera and an image output device such as a printer.

[0002]

2. Description of the Related Art As of 2000, digital still cameras with a high number of pixels in the consumer market are often configured to pick up images at 1 to 6 million pixels using an image sensor as the number of pixels. At present, an image obtained by imaging is expressed by the above-described number of pixels at the maximum size. There is also a camera in which the number of pixels equal to or more than the actual number of pixels of the image sensor is output by interpolating pixel data between pixels using captured pixel data, thereby increasing the resolution of the obtained image.

[0003] As shown by these, the increase in the number of pixels is designed for the purpose of improving the resolution in photographic printing, which is one of the final output forms, for example.

When an obtained image is displayed on a device, the display device may not be able to display the entire image in many cases because it is not actually configured to display such a large number of pixels. In order to display the entire image, the image may be displayed in a reduced size, or may be reduced during recording on a medium.

There are two methods for reducing an image, a method of reducing the obtained image data by signal processing, and a method of performing addition mixing or thinning-out reading at the stage of reading from the imaging unit. The latter method is, for example, an automatic focusing control (AF) performed by an automatic processing system.
l) and automatic exposure adjustment (AE: Automatic Exposure co)
This is used when high-speed reading is required, such as in a movie mode for displaying a moving image.

[0006]

By the way, the above-mentioned 2
One of the image reduction methods may originally lower the image quality compared to the quality of the generated image. However, there is a method of obtaining a good image quality when reducing the image by making use of the feature of the so-called honeycomb arrangement, and it is possible to improve the marking problem.

An object of the present invention is to provide a so-called honeycomb-array-specific image processing apparatus and image processing method capable of solving the drawbacks of the prior art and maintaining high image quality even when the image size is reduced. Aim.

[0008]

According to the present invention, in order to solve the above-mentioned problems, a plurality of image sensors for photoelectrically converting color-separated incident light are arranged with respect to a geometric center of an adjacent image sensor. And / or an image processing apparatus that generates image data that provides a high-quality image by using pixel data of each color obtained by shifting the pixel data by a half pitch in the column direction. The pixel data obtained in the form of a square lattice is a color related to the generation of luminance data included in the image data, and the pixel data at one diagonal position sandwiching the color related to the generation of the luminance data is a color related to the generation of the luminance data. The same color or a different color as one of the different colors, and the pixel data at the other diagonal position sandwiching the color related to the generation of the luminance data is represented by the color related to the generation of the luminance data and the other color different from the one color Then, pixel data supplied in the same color or a different color is input, and the supplied pixel data is used as data from an existing real pixel, and the center of an area surrounded by the existing pixels is set as a geometric center. Virtual pixels,
Reduction processing means for setting a target pixel used for reduced display to a real pixel or a virtual pixel in the supplied pixel data and generating pixel data of a desired color from surrounding pixel data surrounding the target pixel; Means for broadening the frequency of the pixel data obtained as (1).

According to the image processing apparatus of the present invention, the pixel data of the target pixel is generated by the reduction operation processing unit by using the target pixel as a real pixel or a virtual pixel to generate pixel data of a desired color. As a whole, the number of pixels of each color used is reduced, and it is determined in consideration of the arrangement of green pixels constituting a structural image. Since the image processing is performed, the image quality of the image is further improved.

Here, the color of the supplied pixel data may be of a primary color type or a complementary color type. The spatial structure of the obtained pixel data is a so-called honeycomb structure in which pixels are shifted. The relationship related to the spatial frequency distribution of each color in this honeycomb structure is as follows. First, the colors related to the generation of the luminance data included in the image data are obtained in a two-dimensional square lattice, such as green, magenta or white. I do. As pixel data supplied with pixel data of a color related to generation of luminance data, two colors used in a primary color system and a complementary color system are supplied in a checkered or complete checkered pattern. In the case of primary colors, for example, one color is red and the other color is blue.

In order to solve the above-mentioned problem, the present invention separates incident light from a subject field into three colors, and an image pickup device for photoelectrically converting the obtained three colors of incident light is used for an adjacent image pickup device. A plurality of pixels are arranged at a half pitch in the row and / or column direction with respect to the geometric center, the image sensor is used as an actual pixel, and each of image signals obtained from the image sensor is digitally converted using the image sensor. In the image processing method of calculating the pixel data and generating a virtual pixel that newly forms a space surrounded by the real pixels and generating luminance and color difference data corresponding to each pixel by calculating based on the supplied pixel data, According to this method, the color of the pixel data sandwiches the pixel data obtained two-dimensionally in a square lattice between the color related to the generation of the luminance data included in the image data and the color related to the generation of the luminance data. The pixel data at the diagonal position is the same color or a different color as one color different from the color related to the generation of the luminance data, and the pixel data at the other diagonal position sandwiching the color related to the generation of the luminance data is related to the generation of the luminance data. Using pixel data supplied in the same color or different color as a color and another color different from one color, the supplied pixel data is regarded as data from a real pixel, and the center of an area surrounded by the real pixel is used. Is a virtual pixel having a geometric center, and of the supplied pixel data, a target pixel used for reduced display is a real pixel or a virtual pixel, and pixel data of a desired color is obtained from surrounding pixel data surrounding the target pixel. , And a second step of frequency-banding the pixel data obtained as the target pixel in frequency.

According to the image processing method of the present invention, a target pixel is converted into a real pixel or a virtual pixel by using image data including pixel data in which a pixel having a square lattice arrangement is assigned a color related to luminance data. By generating pixel data of a plurality of desired colors from the surrounding pixel data surrounding the target pixel and performing a frequency band widening process on the generated pixel data of the desired color, the pixel data of the target pixel is obtained. As a representative value of the surrounding pixel data, while reducing the number of pixels of each color to be used as a whole, while taking into account the arrangement of green pixels when constructing the image, the resolution of the structural image is improved, Further, the image data is improved in quality because the processing of widening the band is performed on the pixel data obtained by the widening means.

[0013]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Next, an embodiment of an image processing apparatus according to the present invention will be described in detail with reference to the accompanying drawings.

In this embodiment, a digital camera 10 to which the present invention is applied will be described. In the present embodiment, illustration and description of parts not directly related to the present invention are omitted. Here, the reference numerals of the signals are represented by the reference numbers of the connecting lines in which they appear.

The digital camera 10 has an optical lens system 1
2, operation unit 14, system control unit 18, timing signal generation unit 20, driver unit 22, mechanical shutter 24, imaging unit 2
6, pre-processing unit 28, signal processing unit 30, compression / decompression processing unit 32,
Storage unit 34, monitor 36, printer 38 and I / F unit 40
Is provided. These components will be described sequentially.

The optical lens system 12 is constituted by combining a plurality of optical lenses, for example. Optical lens system
In FIG. 12, although not shown, the positions of these optical lenses are adjusted to adjust the angle of view of the screen to an operation signal 14a from the operation unit 14.
It includes a zoom mechanism that adjusts according to the distance and an AF (Automatic Focus) adjustment mechanism that adjusts the focus according to the distance between the subject and the camera 10. As described later, the operation signal 14a is supplied to the system control unit 18 via the system bus 16 and the data bus 18a. The system control unit 18 supplies a control signal 18b to the system bus 16. Control signal 18c
Is supplied from the system bus 16 to the timing signal generator 20 and the driver 22 as a control signal 18c. The timing signal generating section 20 and the driver section 22 operate in response to the supplied control signal 18c to drive the optical lens system 12 with the drive signal 22a.
Is output.

The operation unit 14 has a release shutter button (not shown) and a selection function for displaying various items on, for example, a monitor screen and selecting the displayed items using a cursor. The operation unit 14 includes a still image / moving image setting unit among these various function selections. The still image / moving image setting unit outputs the set result as another operation signal (not shown in this embodiment). The operation selected by the operation unit 14 is, as described above, the operation signal 14a
Is notified to the system control unit 18 via the.

The system control unit 18 includes, for example, a CPU (Cent
ral Processing Unit).
The system control unit 18 has a ROM (Read Only Memory) in which the operation procedure of the digital camera 10 is written. The system control unit 18 includes a data signal line 18
a, for example, a control signal 18 for controlling the operation of each unit using information supplied from the operation unit 14 in response to a user's operation, that is, an operation signal 14a and information stored in the ROM.
Generate b. The system control unit 18 generates the control signal
18b via the system bus 16 to the optical lens system 12 described above.
In addition to the driving of the signal, the signal is supplied to the timing signal generation unit 20 and the driver unit 22.
8, signal processing unit 30, compression / decompression processing unit 32, storage unit 3
4. It is also supplied to the monitor 36, printer 38 and I / F unit 40.

The timing signal generator 20 is a basic clock (system clock) for operating the digital camera 10.
(Not shown) that generates the For example, a VCO (Voltage Controlled Oscillator) method or the like is used for the oscillator. The timing signal generator 20 supplies the basic clock to almost all necessary blocks such as the system controller 18 and the signal processor 30, and also generates various signals by dividing the frequency of the basic clock. In particular, the timing signal generator 20 includes a circuit that uses the basic clock to generate timing signals 20a and 20b for adjusting the operation of each unit based on the control signal 18b. The timing signal generator 20 supplies the generated timing signal 20a to the driver 22. The timing signal generator 20 also generates and supplies a timing signal 20b so as to supply it as operation timing of the preprocessor 28 and the like. In addition, although not shown, various timing signals are supplied to each unit as shown in FIG.

The driver section 22 uses the supplied timing signal 20a to generate desired drive signals 22a, 22b, and 22c.
Are generated and supplied to the optical lens system 12, the mechanical shutter 24, and the imaging unit 26, respectively. The drive signal 22c is
It supports modes for still images and moving images.

The mechanical shutter 24 operates in response to a pressing operation of a release button of the operation unit 14. The operation order is
By pressing the release shutter button, for example,
The operation signal 14a is transmitted via the system bus 16 to the system controller.
The control signal 18b is supplied to the driver 18 via the system bus 16 and the signal line 18c.
Supplied to 22. The mechanical shutter 24 is the driver section 22
It operates according to the drive signal 22b supplied from the controller. The mechanical shutter 24 operates and is controlled in such an order.

The imaging section 26 is a single-chip color CCD sensor in which a color filter 26a is integrally provided on the incident light side of a solid-state imaging device (Charge Coupled Device: CCD) 26b. this is,
The optical image formed by the optical lens system 12 is a solid-state image sensor 26b.
Output signal according to the amount of light reaching each image sensor of the light receiving unit
Outputs 27. The imaging unit 26 has a major feature in which an optical low-pass filter is not provided. The color filter 26a and the solid-state imaging device 26b are formed integrally in this order from the side of the incident light. The color filter 26a is a single plate.
Color filter segment 260 of color filter 26a and image sensor
262 is in a one-to-one correspondence.

The signal processing will be described on the assumption that the image sensor 262 is a real pixel in the subsequent stage, and a virtual pixel is located at the center of a region not corresponding to the image sensor 262, that is, a geometrical area surrounded by the image sensor 262. I do.

In the color filter 26a, for example, a color filter segment 260 of three primary colors RGB as shown in FIG. 2 is arranged. The arrangement pattern of the color filter segments 260 is a pattern in which the colors G are arranged in a square lattice, and the same color R or B is arranged at a diagonal position across the color G on a complete checkered pattern.

This color pattern is hereinafter referred to as a honeycomb type G square lattice RB perfect checkerboard pattern. The number of pixels shown in FIG. 2 is six out of fourteen, the original color G is six, and the colors R and B are four each. The color G mentioned above
The square lattice shape does not indicate the shape of the pixel but indicates the arrangement shape of the pixel. The shape of the pixel is a polygon, for example, a square, a hexagon, an octagon, or the like.

The imaging device includes a CCD and a MOS (Met)
al Oxide Semiconductor: A solid-state imaging device of the metal oxide semiconductor type is applied. In the imaging unit 26, the signal charge obtained by the photoelectric conversion in accordance with the supplied drive signal 22c is read as a predetermined timing, for example, by a field shift in a vertical blanking period by a field shift to a vertical transfer path. Is supplied to the horizontal transfer path. The signal charges that have passed through the horizontal transfer path are converted into analog voltage signals 27 by charge / voltage conversion by an output circuit (not shown) and output to the preprocessing unit 28. In the case of the CCD type, the solid-state imaging device 26b uses thinning-out reading and all-pixel reading depending on the signal charge reading mode.

Although not shown, the preprocessing unit 28 includes a correlated double sampling unit (hereinafter referred to as CDS).
) And an A / D converter. The CDS unit removes noise by contributing to reduction of 1 / f noise and reset noise included in the analog voltage signal. Further, the pre-processing unit 28 may perform gamma correction here. The output signal from which the noise component has been removed is sent to the A / D converter. The A / D converter has an A / D converter that quantizes the signal level of the supplied analog signal with a predetermined quantization level and converts the signal level into a digital signal 29. The A / D converter converts the digital signal 29 based on the timing signal 20 b such as a conversion clock supplied from the timing signal generator 20.
Is output to the signal processing unit 30. The pre-processing unit 28 performs each processing by setting the sampling frequency used for the analog image signal obtained from the imaging unit 26 high enough to prevent aliasing distortion even without the optical LPF.

The signal processing section 30 includes a data correction section 30a having a frame memory function, an interpolation processing section 30b, and a broadband processing section 30c. Further, the data correction unit 30a
Although not shown, a frame memory and a correction processing unit are included as buffer functions. Although not explicitly shown, the correction processing unit also has a function of performing gamma correction, white balance adjustment, and the like. The gamma correction here also makes it possible to reduce the number of bits for signal processing by reducing power consumption and circuit scale. If the gamma correction process is already performed by the pre-processing unit 28, the description is omitted. The image data 29 digitized by the A / D conversion is supplied to the frame memory of the data correction unit 30a and stored therein. Since the frame memory repeatedly reads pixel data while shifting the read area, it is advantageous to use a non-destructive memory. Further, the frame memory stores signals related to control of a write / read enable signal, a clock signal, and the like included in the control signal 18b from the system control unit 18 via the control signal 18 via the system bus 16.
supplied as d. As a process, the data correction unit 30a also performs gain adjustment for each color filter, for example, in order to adjust the level of image data corresponding to each color filter to a level suitable for subsequent signal processing. Further, the data correction unit 30a outputs the stored image data to the interpolation processing unit 30b as image data 42 in a predetermined order.

The interpolation processing unit 30b will be described using real pixels and virtual pixels defined in the description of the imaging unit 26. In the present embodiment, the interpolation processing unit 30b has a function of performing still image interpolation and moving image interpolation, and switches the supply destination of the supplied image data 42 according to a user's request and supplies the data to each unit. Among them, a configuration for performing still image interpolation will be described. The interpolation processing unit 30b includes a normal correction unit 44 and a reduction correction unit 46 corresponding to still image interpolation. Interpolation processing unit 30b
The normal correction unit 44 is controlled according to the control from the system control unit 18.
And the reduction correcting section 46 operates. However, the interpolation processing unit 30
The output as b is made from the selected one.

As shown in FIG. 3, the normal correcting section 44 includes a luminance interpolation developing section 44a and a color interpolation developing section 44b. Usually correcting unit 44 has a function for interpolating generates a high-frequency luminance data Y _H and the three primary colors data including high-frequency component in the virtual pixel located in the middle of the actual pixel and existing pixels. The luminance interpolation developing section 44a and the color interpolation developing section 44b include
Image data 42 is supplied from the data correction unit 30a. The image data 42 is pixel data from a real pixel existing in the image sensor.

The luminance interpolation / expansion unit 44a includes a high-frequency luminance data creation unit 440 and a luminance data interpolation / expansion unit 442. The high-frequency luminance data generation unit 440 generates high-frequency luminance data Y _H (444) including a high-frequency component at the position of a real pixel or a virtual pixel using the supplied pixel data. Frequency luminance data generator 440 of the present embodiment performs calculation of high-frequency luminance data Y _H at the location of the existing pixels. This calculation is
For example, the supplied pixel data 42 is used to detect the direction of the correlation with the supplied color, and high-frequency luminance data Y _H (444) corresponding to the detection result is generated.

The luminance data interpolation / expansion unit 442 generates high frequency luminance data Y _H at the position of the virtual pixel using the supplied high frequency luminance data Y _H (444). Luminance interpolation developing unit 44
a outputs the high-frequency luminance data Y _H (446) at the positions of all the real pixels and the virtual pixels to the broadband processing unit 30c by generating and interpolating the high-frequency luminance data Y _H in this manner.

[0033] Luminance interpolation expansion unit 44a, the high-frequency luminance data to create a high-frequency luminance data Y _H corresponding to the position of the virtual pixel, corresponding to the positions of existing pixels by using the high-frequency luminance data Y _H of the virtual pixel Y _H may be created.

The color interpolation developing section 44b has a function of performing an interpolation process on a color that does not correspond to a virtual pixel and a real pixel in consideration of the color arrangement of the real pixel, and generating three primary color data of the entire screen. Therefore, the color interpolation developing unit 44b includes:
There is an interpolation developing unit for each color. This is the R interpolation expansion unit 448
, A G interpolation developing section 450 and a B interpolation developing section 452. Each of the interpolation developing sections 448, 450, and 452 outputs color data 42a, 42b, 42c for each color of the supplied image data 42.
Is input, and interpolation and expansion processing is performed on the input color data to generate color data at all positions of the real pixels and the virtual pixels. In this way, all three primary colors RGB are aligned, and the RGB primary processing is performed to execute the three primary color data.
56 and 458 are output to the broadband processing section 30c.

The reduction correction section 46 supplies the supplied pixel data 42
Is a functional unit that treats a real pixel or a virtual pixel as a target pixel to which a representative value of reduction correction is assigned, and consequently reduces the number of pixels. This embodiment shows a configuration in which the target pixel is set as a real pixel (see FIG. 4). The reduction correction unit 46 includes a G reduction operation unit 46a, an R reduction operation unit 46b,
A B reduction operation unit 46c is included.

The G reduction operation unit 46a is an operation unit that calculates the pixel data of the color G by performing averaging using the pixel data of four colors G surrounding the target pixel with the position of the color R or the color B as the target pixel. is there. Here, it is assumed that the pixel data has the pixel arrangement relationship shown in FIGS. 5 and 6. The octagonal solid line represents a real pixel, and the octagonal dashed line represents a virtual pixel. Symbols R, G, and B written inside the octagon indicate the color generated by each pixel.
Subscripts added to the symbols R, G, and B indicate the positions of pixels in a matrix display.

[0037] G reduction operation unit 46a in G square lattice RB completely checkered pattern as shown in FIG. 5, when calculating a color G _22, G ₂₄ corresponding to the position of the color R _{2 2} and color B _24, color G ₁₁ , G ₁₃ , G ₃₁ ,
G ₃₃ and colors G ₁₃ , G ₁₅ , G ₃₃ , and G ₃₅ are used as a set of each and averaged and obtained. G reduction operation unit 46a
Not only is the pixel data 462 of the aggregated color G output, but also the pixel data 460 of the color G _H , that is, the luminance data Y _H is output to the broadband processing unit 30c.

The R reduction operation unit 46b performs averaging using the pixel data of four different colors R surrounding the target pixel with the position of the color B as the target pixel, and calculates the pixel data of the color R at the target pixel (color B). This is a calculation unit for calculating. When the target pixel is the color R, the pixel data at this position is used as it is. R reduction calculation section 46b in the pattern of Figure 6, the color corresponds to the position of the color B ₂₄ R
When calculating (same color) pixel data, the colors R ₀₄ , R ₂₂ , R
₂₆ and R ₄₄ as a set (solid line) and averaged to obtain the color R
Calculate ₂₄ . The position of the color R _22, pixel data of the original color R is used as it is. The R reduction operation unit 46b outputs the calculated pixel data 464 of the color R for the target pixel to the broadband processing unit 30c.

The B-reduction calculating unit 46c performs averaging using four different-color B pixel data surrounding the target pixel with the position of the color R as the target pixel and calculating the pixel of the color B at the target pixel (color R). This is an operation unit that calculates data. If the target pixel is color B, the pixel data at this position is used as it is. B reduction calculation section 46c in the pattern of FIG. 6, the case of calculating the pixel data of the color B (the same color) corresponding to the position of the color R _22, color B _02,
B _20, B _24, B ₄₂ a pair and averaged with a (dashed line), and calculates the color B _22. At the position of color B ₂₄ , the original color
The B pixel data is used as it is. B Reduction operation unit 46c
Outputs the calculated pixel data 466 of color B for the target pixel to the broadband processing unit 30c. In this embodiment, the number of pixels is reduced to the number of pixel data at the position indicated by the arrow in FIG.

[0040] The wideband processing unit 30c, and the high-frequency luminance data Y _H and plain RGB data supplied from either the normal correction portion 44 and the reduced correction unit 46 is supplied usually correcting unit 44 and the reduced correction unit Processing units corresponding to each of the 46 are prepared. Of these, in this embodiment, the configuration for reduced display is represented as a broadband processing unit 30c. The broadband processing unit 30c includes a high-pass filter circuit 48, a color difference matrix unit 50, an adder 52, a distortion prevention processing unit 54, an aperture adjustment unit 56, and color difference gain adjustment units 58 and 60 (see FIG. 7). ). The high-pass filter circuit (hereinafter referred to as HPF) 48 is supplied with high-frequency luminance data 446 or color data.
This is a filter for extracting high-frequency components from the pixel data 460 of _GH . HPF 48 is the extracted high frequency component Y _h (or G _h )
Is output to the one end 52a side of the adder 52.

The color difference matrix unit 50 uses the three primary color data 454, 456, 458 of the normal correction unit 44 or the three primary color data 464, 462, 466 of the reduction correction unit 46 to generate luminance data Y and color difference data C _r , C _b . are doing. The color difference matrix unit 50 uses the conventional calculation formula used so far for the matrix calculation performed here. Color difference matrix unit 50, the generated luminance data Y (50a) is supplied to the other end 52b of the adder 52, the color difference data C _r (50b), and outputs the C _b (50c) in anti-distortion processing unit 54.

The adder 52 generates a high-frequency component corresponding to the primary color G.
To widen the frequency band of the three primary color data by adding the Y _h. Broadbanding is performed for the color G in an out-of-green manner. The adder 52 supplies the luminance data 52a (Y) with the wide band to the distortion prevention processing unit 54.

The adder 52 is not limited to this arrangement position. Although not shown, the adder 52 is provided in front of the color difference matrix unit 50 when the number of pixels is increased by performing normal correction signal processing. Pixel data and high-frequency components generated for each color
Adder may respectively arranged to allow the addition of Y _h. In this case, the correction in the signal processing normally correcting unit 44 performs, for generating respective pixel data of each primary color of the resulting plane to expand as the high-frequency luminance data Y _H and color interpolation described above. HPF circuit extracts the high frequency luminance component Y _h through a high-frequency luminance data Y _H. Broadband is the pixel data of the extracted high-frequency luminance component Y _h and plain each primary color, are added in respective adders (not shown).

The distortion prevention processing unit 54 includes low-pass filters (hereinafter, referred to as LPFs) 540, 542, and 540 having a band up to a high frequency so as not to generate aliasing distortion without impairing the bands of the three supplied signals. There are 544. Among them LP
F 540 has a characteristic of passing the highest frequency according to the luminance data Y. In addition, the distortion prevention processing unit 54 also performs processing for limiting the overlapping frequency band in one direction with respect to the region where the horizontal and vertical frequency bands overlap when normal correction is performed. This avoids image quality degradation due to overlapping frequencies. The thus treated luminance data Y (54a) and color difference data C _r (54b),
C _b (54c) is supplied to the aperture adjustment unit 56 and the color difference gain adjustment units 58 and 60, respectively.

The aperture adjusting unit 56 processes and outputs the low-frequency component by LPF processing, for example, so as not to lower it. As a result, the same effect as when the outline (edge) enhancement processing is performed on the image is obtained. The color difference gain adjusters 58 and 60 supply the color difference data C _r (54b) and C _b (5
The gain is adjusted for 4c), and the level is adjusted to a predetermined level. Such signal processing section 30 and, the generated luminance data Y (60a), the color difference data C _r (60b), compressing the C _b (60c) /
It is supplied to the decompression processing unit 32.

[0046] Figure 1 in back compression / decompression processing unit 32, a frame memory for temporarily storing the luminance data Y and color difference data C _r for one frame supplied from the signal processing unit 30, a C _b, e.g. , JPEG (Joint Photogra
A circuit for performing compression according to the phic experts group) standard and a circuit for decompressing the compressed image to original data again. Here, the frame memory may also be used as the frame memory of the signal processing unit 30. Here, possessing a plurality of frame memories is advantageous in recording moving images, that is, in processing such as continuous shooting. Further, the compression is not limited to JPEG, but may be MPEG (Moving Picture coding Experts Group).
p) and a compression method such as motion JPEG which applies JPEG to a single frame of a moving image.

The compression / decompression processing unit 32 includes a system control unit 18
During recording by the control of
The data is supplied to the storage section 34 via 16. Compression / decompression processing unit 32, the luminance data Y (60a) supplied, color difference data C _r
(60b) and C _b (60c) may be supplied to the system bus 16 by passing through under the control of the system control unit 18, or a signal from the signal processing unit 32 may be supplied to the monitor 36 via the system bus 16. it can. When the compression / decompression processing unit 32 performs the decompression processing, on the contrary, the data read from the storage unit 34 is taken into the compression / decompression processing unit 32 via the system bus 16 and processed. After the processed data is stored in the frame memory, the compression / decompression processing unit 32 reads the data in the frame memory in a required order under the control of the system control unit 18 and supplies the data to the monitor 36 for display.

The storage section 34 includes a recording processing section for recording on a recording medium and a reproduction processing section for reading image data recorded from the recording medium (both not shown). Examples of the recording medium include a semiconductor memory such as a so-called smart media (registered trademark), a magnetic disk, and an optical disk. When a magnetic disk or an optical disk is used, there is a head for writing the image data together with a modulator for modulating the image data.

The monitor 36, the luminance data Y and color difference data C _r supplied via the system bus 16 in response to control of the system controller 18, as well as consideration of the size of the screen C _b or three primary RGB data , And a function of adjusting and displaying the timing. As shown in FIG. 1, the image data for the reduced display which is broadened and supplied from the broadband processing unit 30c is supplied to the monitor 36 via the system bus 16 to display the image data. Is also good.

Although not shown, the printer 38 includes a YMC conversion processing section, a drive signal generation section, a thermal head, and a photographic paper holding section. The YMC conversion unit converts the three primary colors RGB or luminance data Y and color difference data adjusted to the reduced display size.
YMC conversion processing is performed according to C _r and C _b . The drive signal generation unit includes:
Generates a drive signal that generates heat according to the processed YMC data. The thermal head includes a sensor that generates heat in the head according to the level of the supplied drive signal. The heat-sensitive head heats the photographic paper discharged from the photographic paper holding unit to the outside of the camera 10 while not in contact with the photographic paper, thereby developing a color.

The I / F unit 40 is, for example, a USB (Universal Serial
ial Bus), and has a processing function of inputting and outputting image data and the like with the outside. The input / output functions include data signal level conversion and control timing adjustment. I / F
The unit 40 can supply the generated image data and the reproduced image data to an external printer connected thereto.

As described above, the digital camera 10 is configured to perform signal processing on an image captured and record the image.
Further, the recorded image data is reproduced and displayed on the monitor 36.

Next, the operation of the digital camera 10 will be described. Reference is made to the drawings used in the above configuration as necessary. The digital camera 10 operates, for example, according to a main flowchart in photographing shown in FIG. The digital camera 10 according to the present embodiment has both an image size enlargement mode (interpolation processing mode) for photographing still images and an image size reduction mode (reduction processing mode). As described above, this operation procedure will be described focusing on the operation in the reduction mode in the processing in these modes in the still image shooting as described above.

After the power is turned on, the digital camera 10 selects which mode to operate from among a plurality of modes of the camera 10 (step S10). Here, in addition to selecting a shooting mode from various modes, still image shooting and image size setting are also performed. The image size setting includes a normal interpolation processing mode in which the number of pixels is increased to the number of actual pixels or more, and a reduction processing mode in which the number of pixels is decreased from the actual number of pixels. The image size setting selects one of these modes.

When the photographing mode is selected, the image of the scene taken through the optical lens system 12 is digitally processed and continuously displayed on the monitor 36 (step S12: preliminary image pickup). In particular, when the release shutter button (not shown) of the operation unit 14 is pressed by one step, the camera 10 not only displays the captured image of the object scene on the monitor 36 but also sets the exposure conditions before the main imaging. Processing and adjustment of the focal length with respect to the exposure target and the like are performed by the system control unit 18 to prepare for the main imaging. The parameter setting process related to the setting of the exposure condition may be performed by the signal processing unit 30.

At the desired timing, the user presses the release shutter button all the way down to two steps to perform the main image pickup (step S14). When the main imaging is performed, the system controller 18 is supplied with the operation signal 14a. At this timing, the system controller 18 sends a preset control signal 18b to the timing signal generator 20 and the driver 22 via the system bus 16 and the signal line 18c. The camera 10 operates according to a timing signal and a drive signal from the timing signal generator 20 and the driver 22. As shown in FIG. 2, the solid-state imaging device 26b converts the signal charge read from the pixel corresponding to the color filter segment RGB into the analog signal 27 via the color filter 26a of the honeycomb type G square lattice RB complete checkerboard pattern. The read analog signal 27 is supplied to the preprocessing unit 28.

The pre-processing unit 28 removes noise contained in the analog signal 27 supplied according to the timing signal 20b, performs A / D conversion processing on the signal from which the noise has been removed, and converts the signal into a digital signal 29 ( Step S16). The digital signal 29 is supplied to the signal processing unit 30 as captured image data. Although not shown in the flowchart, the signal processing unit 30
In the data correction unit 30a, for example, adjustment such as gamma correction and white balance is performed as a correction process. The data correction unit 30a may include a frame memory (not shown). In this case, the image data is temporarily stored when supplied, and thereafter read out for each pixel to perform the above-described correction. The corrected image data 42 may be stored again in the frame memory, or the correction result may be supplied to the interpolation processing unit 30b.

Next, it is determined whether or not the image size is set in the interpolation processing mode during still image shooting (step S1).
S18). When this setting is in the interpolation processing mode (YES), the process proceeds to the interpolation signal processing in which the normal correction processing unit 44 of the signal processing unit 30 expresses an image with a larger number of pixels than the number of existing pixels (subroutine SUB1). When this setting is in the reduction correction processing mode (NO), the reduction correction section 46 proceeds to a reduction correction processing for reducing the number of pixels from the actual number of pixels (subroutine SUB2).

In the interpolation signal processing (subroutine SUB1),
For example, high-frequency luminance data is generated at the position of a real pixel included in the CCD 26b, and interpolation generation of high-frequency luminance data is also performed on a virtual pixel as a new pixel so as to increase the number of pixels beyond the number of real pixels. For color, color interpolation development unit
In 44b, pixel data of two colors different from the color originally obtained for one real pixel is also generated by interpolation. Pixel data generated by the interpolation, i.e. the high-frequency luminance data Y _H and plain color data are respectively supplied to the wideband processing unit 30c.

The pixel data interpolated and generated is subjected to a band broadening process (subroutine SUB3). Broadband processing is
Using the pixel data supplied respectively generate wideband luminance data Y and color difference data C _r, the C _b as image data. The generated image data is output to the compression / decompression processing unit 32.

The processing shifts to a reduction correction processing performed when it is determined that the processing is not normal interpolation signal processing (subroutine SU
B2). The reduction correction processing is performed by the reduction correction unit 46, which will be described in detail later. The reduction correction unit 46 uses the pixel data of the actual pixel to generate pixel data of three primary colors RGB using the actual pixel as a target pixel. The reduction correction unit 46 generates pixel data of a color not included in the original pixel with respect to the pixel data having the color R or the color B as the target pixel. That is, color B and color G pixel data are generated for the color R pixel data. Also,
Pixel data of color R and color G is generated for the pixel data of color B. The generated pixel data is sent to the broadband processing unit 30c.
Supplied to

The widening processing section 30c performs widening processing on the target pixel in the reduction correction processing mode (subroutine SUB4). In this mode, a high-frequency luminance component G _h (= Y _h ) for the color G of the target pixel is extracted. Then, the broadband processing unit 30c, each generate luminance data Y and color difference data C _r, the C _b using three primary colors RGB of the pixel data of the target pixel. The generated luminance data Y has a high-frequency luminance component
Add G _h . Thereby, the luminance data Y is broadened. In addition, image processing and the like corresponding to a wider band are performed as post-processing. The image size obtained in this way is reduced.

Next, it is determined whether to print the image data generated in the reduction correction processing mode (step S20). When printing on the built-in photographic paper (YES), the process proceeds to image (photograph) printing processing. Also, when not printing an image (NO),
Proceed to compression processing.

The image printing process is performed by the printer 38 (step S22). The image data is adjusted to a reduced size that the printer 38 can print on photographic paper. The printer 38 performs YMC conversion on the image data to generate print image data. The printer 38 prints, for example, as a photograph using the generated print image data.

[0065] compression processing is performed by the compression / decompression processing unit 32 the luminance data Y and color difference data C _r respectively generated when no subroutines SUB3 and printing described above, with respect to C _b (step S24). In the still image mode, JPEG or the like is applied as compressed signal processing. The image data that has been subjected to the compression processing is recorded and stored on a recording medium mounted on the recording / reproducing device of the storage unit 34 (step S26).

Finally, it is determined whether or not the operation of the digital camera 10 is to be terminated (step S28). If the operation is still to be continued (NO), the flow returns to step S10 to continue a series of processing. When the operation is completed (YES), the power of the digital camera 10 is turned off. When the mode setting is used as it is, the operation may be continued from the processing in step S12.

Next, the interpolation signal processing will be briefly described (see FIG. 9). Generating a high-frequency luminance data Y _H to the pixel position of the checkered pattern of existing pixels represents (subroutine SU
B5). High-frequency luminance data Y generated at the position of a real pixel
_H is generated by performing weighted averaging using pixel data located in the direction of high correlation in consideration of the correlation between the target pixel data and the surrounding pixel data. The averaging by the resulting high frequency luminance data Y _H is the complementary color of the pixel data and color G color G, i.e. may be data indicating the color magenta Mg. The correlation detection is performed in the oblique, vertical and / or horizontal directions. In particular, the correlation detection in the vertical and horizontal directions is performed by changing surrounding pixels used for the target pixel while changing the area of the pixel actually used for the correlation detection in consideration of the color (same color or different color) of the target pixel. I have. This processing is performed by the high-frequency luminance data creation unit 440. In the present embodiment it has been generating the high-frequency luminance data Y _H at the position of the existing pixels using existing pixels, may generate a high-frequency luminance data Y _H of the virtual pixel from the existing pixels. In this case, the next interpolation processing is performed on the position of the actual pixel.

[0068] Next to interpolate the high-frequency luminance data Y _H (subroutine SUB6). This interpolation process produces a high-frequency luminance data Y _H at the position of the virtual pixel using the high-frequency luminance data Y _H of existing pixels. This generation is performed by the luminance data interpolation processing unit.
442. Frequency luminance data Y _H of this position may be generated in consideration of the correlation. The high frequency luminance data Y _H may be generated using the LPF function of the digital filter.

Next, color interpolation development processing is performed (subroutine).
SUB7). By using a single-plate color filter 26a, one color is assigned to an existing pixel via a color filter segment. The color interpolation development process interpolates and generates a color to which no actual pixel is assigned among the three primary colors RGB. With this processing, the color interpolation developing section 44b can generate pixel data of the three primary colors RGB. In each of the subroutines SUB5, SUB6, SUB7, processing is performed in units of one frame. When these processes are completed, the process returns to the main routine via return.

The procedure of the reduction correction processing in the reduction correction processing mode will be specifically described (see FIG. 10). This procedure will be described with reference to the relationship between the pixel positions schematically shown in FIGS. In the reduction correction processing (subroutine SUB2), the position of the target pixel which is a representative position when reduced is set (substep SS200). The target pixel is the color R or the color surrounded by four pixels of the color G arranged in a square lattice
The actual pixel (shaded line) of B is set (see FIG. 5). In this embodiment sets the position R ₂₂ colors R as the first target pixel.

[0071] Next to calculate the pixel data G ₂₂ color G at the pixel position R ₂₂ (substep SS202). Pixel data G ₂₂ is, for example, around the pixel data _{(G 11, G 13, G} 31,
G ₃₃ )

[0072]

G ₂₂ = (G ₁₁ + G ₁₃ + G ₃₁ + G ₃₃ ) / 4 Calculated by the averaging of (1).

Next, it is determined whether or not the color of the target pixel is R (substep SS204). When the target pixel corresponds to the color R (YES), the pixel data of the color R in the target pixel is used as it is. When the target pixel is not the color R (NO), the color of the target pixel is determined to be color B, and the pixel data of the color B is used as it is. If the pixel R ₂₂ for calculating now,
Determination is the pixel data of the color R for use as is reduction correction pixel data R ₂₂ because it is true.

Next, the color B that has not yet been calculated for the target pixel
Calculating the pixel data B ₂₂ (substep SS206).
The pixel data B ₂₂ is, as shown in FIG.
Pixels (B ₀₂ , B ₂₀ , B ₂₄ ,
B ₄₂ )

[0075]

[Number 2] _{B 22 = (B 02 + B} 20 + B 24 + B 42) / 4 is calculated by averaging the (2).

Next, it is determined whether or not the reduction correction process for one frame of pixel data has been completed (sub-step SS208). When processing is not yet completed (NO),
Proceed to the setting of the next target pixel (substep SS210). When the processing is completed (YES), the process proceeds to the return and ends this processing. At this point, since the processing has not been completed, the process proceeds to sub-step SS210.

The new target pixel advances the display indicating the column by +2 when the processing is advanced in the horizontal direction. When the processing is shifted in the vertical direction so as to start a new line, the display indicating the row is incremented by +2, and the display indicating the column is returned to the column display indicating the start of processing in the horizontal direction. By simply incrementing by +2, the new target pixel becomes the pixel data B ₂₄ of the color B. After this setting, sub-step SS
Returning to step 202, the above-described processing is repeated. Pixel data G ₂₄
, For example around the pixel data _{(G 13, G 15, G} 33, G 35)
Using

[0078]

G ₂₄ = (G ₁₃ + G ₁₅ + G ₃₃ + G ₃₅ ) / 4 Calculated by the averaging of (3).

It is determined whether or not the color of the target pixel is color R.
Not equal because it is determined (NO), calculates the pixel data R ₂₄ colors R not yet calculated in the target pixel (substep SS212). The pixel data R ₂₄ is, as shown in FIG.
Pixels surrounded by solid lines of the same color as the calculated color R (R ₀₄ ,
R ₂₂ , R ₂₆ , R ₄₄ )

[0080]

R ₂₄ = (R ₀₄ + R ₂₂ + R ₂₆ + R ₄₄ ) / 4 Calculated by the averaging of (4).

[0081] Also, the target pixel is calculated by a new line, for example, performs a line feed to the calculation of each color lacking change the setting as described above such time as the pixel position B _42. When showing a result, pixel data G ₄₂ is, for example, using the surrounding pixel data _{(G 31, G 33, G} 51, G 53)

[0082]

G ₄₂ = (G ₃₁ + G ₃₃ + G ₅₁ + G ₅₃ ) / 4 Calculated by the average of (5). Pixel data R _42, for example by using the surrounding pixel data _{(R 22, R 40, R} 44, R 62)

[0083]

R ₄₂ = (R ₂₂ + R ₄₀ + R ₄₄ + R ₆₂ ) / 4 Calculated by the averaging of (6).

As described above, in calculating each target pixel, the surrounding pixel data is used by overlapping a part of the pixel data. As a result of this calculation, RGB pixel data of the three primary colors at each position indicated by oblique lines in FIG. 5 is obtained. This is because the number of pixels is reduced, but the pixel data of the three primary colors RGB can be obtained at each target pixel. Therefore, the digital camera 10 is comparable to the case where the image data is reduced by the single-chip system but is determined by the three-chip CCD. Information is obtained.

[0085] real number of pixels N _r present around the area to be used to determine the pixel data of the target pixel, N _h (number of columns) in the horizontal direction, and N _v (the number of rows) in the vertical direction, the Expressing the number between the matrices using the Gaussian symbol [],

[0086]

[Equation 7] Nr = is calculated as _{([N h / 2] +1} ) × ([N v / 2] +1) + [N h / 2] × [N v / 2] ··· (7). When substituting the values, N _h = 7, N
_In the region 62 where _v = 7, the middle row is [N _h / 2] = 3, [N _v / 2] = 3
Therefore, the number Nr of actual pixels in the area 62 is Nr = 4 × 4 + 3 × 3
= 16 + 9 = 25 pieces Also, the number of virtual pixels in the area 62
Nv

[0087]

[Equation 8] _{Nv = [N h / 2]} × ([N v / 2] +1) + ([N h / 2] +1) × [N v / 2] ··· (8) , and the number of virtual pixels Nv = 3 × 4 +4 × 3 = 24 (= 25-1
). Therefore, pixel data is generated at 49 pixel positions, the total number N of the real and virtual pixels in this area 62 combined. In the reduction correction process, one pixel is represented from 25 regions 64 including the real and virtual pixels. In addition, as described above, the target pixel is generated in the area 66 in the reduction correction processing while overlapping the pixels.
In consideration of this, and by counting the number of overlapping pixels, in the region 62 in which the region 64 is expanded in the horizontal direction and the vertical direction (the total number N includes 49), the pixel count is 25+ (15 +10) + (6 + 9 + 4 + 6) + (9 + 6 + 6 + 4) = 100. Therefore, the number of pixels for reduction correction obtained from this area 62 is
100/25 = 4 It can be seen that the number of pixels can be significantly reduced in the reduction correction processing taking into account the overlap in the region 62 as an example, as compared with the case where the number of pixels is increased by performing the interpolation signal processing as 4/49 = 0.082.

When the G square RB complete checkerboard pattern is reduced and corrected as in the present embodiment, as a result, pixel data of the three primary colors RGB can be provided at the positions of the colors R and B.
Since the pixel data of the color G is arranged in a square, since the image structure of the image is unchanged as described later, the image quality of the image can be maintained even if the image size is reduced.

Next, a brief description will be given of a wideband processing corresponding to the interpolation signal processing for increasing the number of normal pixels (see FIG. 12). The configuration of the broadband processing corresponding to the interpolation signal processing is not shown. The broadband processing is performed by the high-frequency luminance data Y supplied from the luminance interpolation developing unit 44a.
_A high-pass filter (HPF) process is performed on _H (446) (substep SS300). And it outputs a signal of the high frequency components Y _h of the high-frequency luminance data Y _H obtained by this process to the adding unit. The pixel data 454, 456, and 458 of the plain form from the color interpolation developing unit 44b are supplied to one end of each adder. The other end of each of the adders high frequency component signal Y _h are supplied respectively. Each adder, pixel data 454 to be supplied, 456, 458 and the high-frequency component Y _h supplied signal and a color difference matrix unit by adding the (substep SS30
2). By this addition, each pixel data is converted into a signal extending to a wide band.

Next, in the color difference matrix section, color difference matrix processing is performed using the supplied plane pixel data RGB (substep SS304). A conventionally known coefficient is used as a coefficient used for the color difference matrix processing. Luminance data Y and color difference data C _r by performing color difference matrix processing to the supplied pixel data, and outputs the C _b.

[0091] Next luminance data Y and color difference data C _r is supplied, the anti-distortion processing on the C _b applying (sub-step
SS306). These obtained pixel data are processed as distortion prevention processing to prevent frequency duplication and LPF extended to a wide band.
Has been treated. In the duplication prevention processing, for example, if there is a frequency overlapping region in the vertical and horizontal directions, a band-pass filter (BP
F) Processing is performed. The presence or absence of overlap may be determined by interpolation processing or the like, for example, by referring to a flag previously attached to each pixel data.

In the LPF processing, a filter processing in which the characteristic of the band rejection frequency extends to a high frequency band is performed so that aliasing distortion does not occur. In addition, the sampling frequency applied to the analog imaging signal obtained from the imaging unit 26 is increased. Since the folding distortion is prevented from occurring, the optical LPF can be made thinner or, in the best case, provided with the optical LPF. The luminance data Y and the chrominance data C _r and C _b that have been subjected to the distortion prevention processing are output.

Aperture adjustment is performed on the luminance data Y as one of post-processing (sub-step SS308). The aperture adjustment is a process corresponding to the outline enhancement process. Then, gain adjustment is performed on the color difference data C _r and C _b that have been subjected to the distortion prevention processing so as to be at a predetermined level (sub-step SS310). With this processing, it is determined whether the processing for one frame is completed (substep SS312).
If the processing for one frame has not been completed (NO), the process returns to the beginning (sub-step SS300) and repeats a series of processing. If the processing for one frame has been completed (YE
S), the program proceeds to the return, ends this subroutine SUB3, and shifts to the main routine via the return.

Although the configuration for realizing the above-described procedure is different from that of the broadband processing unit 30c as shown in FIG. 7, the configuration may be the same as that of the broadband processing unit 30c. The configuration can be simplified by the common use.

Next, the procedure of the band broadening process for the pixel data obtained in the reduction correction processing mode will be described (see FIG. 13). Also in this broadband processing, first, a high-pass filter (HPF) process is performed on the high-frequency luminance data Y _H (460) supplied from the reduction correction unit 46 (substep SS400). And outputs the high frequency component Y _h (= G _h) of the signal 48a of the high-frequency luminance data Y _H obtained by this process to the adder 52.

Next, the pixel data 462, 464, and 466 subjected to the reduction processing are supplied from the reduction correction section 46 to the color difference matrix section 50. In the color difference matrix section 50, the primary colors G,
Luminance data Y using R, B pixel data 464, 462, 466
And color difference data C _r, to generate the C _b (substep SS40
2). As a coefficient used for the color difference matrix processing, a conventionally known coefficient is used as described above. The generated luminance data Y is supplied to one end 52a of the adder 52. Also,
The resulting color difference data C _r, and outputs the C _b to anti-distortion processing unit 54.

[0097] Adder At 52, the luminance data Y is supplied from one end 52a side, whereas, the high frequency component Y _h of the signal 48a is the other end 52
Supplied from b side. In this case, the adder 52, the luminance data Y by adding the signal 48a of the luminance data Y and the high-frequency component Y _h
To the signal extended to the high frequency range (substep SS4
04). The adder 52 outputs the luminance data Y (52a) to the distortion prevention processing unit 54.

[0098] In the anti-distortion processing unit 54, luminance data Y and color difference data C _r is supplied, the LPF process extending bandstop characteristic for C _b to a high frequency band subjected (substep SS
406). Further, when pixel data is generated in consideration of the correlation in the vertical and horizontal directions, processing according to the presence or absence of frequency overlap may be performed as in the above-described interpolation signal processing. By performing the LPF processing, the occurrence of aliasing distortion is prevented in the distortion prevention processing. This makes it possible to make the optical LPF thinner when the pixel pitch is reduced, and in the best case, it is possible to avoid disposing the optical LPF as in this embodiment.

By configuring the digital camera 10 in this manner, an image structure is provided in which pixels are supplied in a square lattice even when reduction correction is performed, and since each pixel has pixel data of the color G, Even if this correction is performed, it is possible to provide an image comparable to that of the three-panel system even though it is a single-panel, without deteriorating the image quality of the image obtained as compared with the conventional thinned display. Further, the camera 10 is excellent in that the configuration of the imaging unit can be simplified.

Next, another embodiment to which the present invention is applied will be described. This embodiment is different from the previous embodiment in the method (subroutine SUB8) of the reduction correction process performed by the reduction correction unit 46. The principle of the reduction correction processing of the present embodiment will be described below. In this reduction correction processing, actual pixels are arranged in a cross without overlapping areas to be reduced as in the previous method.
It is characterized in that new pixel data is generated using one pixel data. Here, as for the position of a pixel to be newly generated, an empty space is generated at a position of a virtual pixel by a honeycomb type pixel shift, and this position is set as a target pixel. As pixel data, pixel data of three primary colors RGB is generated for the position of the target pixel.

As shown in FIG. 14, pixel blocking is performed as follows.
There are two possibilities. That is, a block in which the colors R and R are arranged in the vertical direction and the color G is arranged in the horizontal direction (FIG. 14A), and a block in which the color G is arranged in the vertical direction and the colors R and B are arranged in the horizontal direction. (Fig. 14 (b)). Fig. 15 shows the relationship between blocks and the colors generated by each block when the latter blocking is adopted.
Shown in The area of each block is indicated by a dashed line. As for the color generated in each block, any one of the three primary colors RGB is shown in a virtual pixel arranged in the block. In this embodiment, the operation is performed according to this series of principles.

In this operation procedure, the pixel data obtained from the actual pixels supplied first are divided into blocks (sub-step SS800: see FIG. 15). Blocking is 4
This is performed so that one pixel data does not overlap each other. In the present embodiment, as shown in FIGS. 14 and 15, a virtual pixel located at the center of the block area is set as a target pixel to be generated. In addition,
A real pixel at the same position in a block may always be defined as a target pixel. It can be seen that, by performing the blocking in this way, each block is also blocked while maintaining the pixel shift arrangement.

Next, a color to be assigned to the block target pixel is specified (sub-step SS802). The color G is designated for the four blocks arranged in the square lattice of the blocks, and the color R or B is designated for the block located at the center of the four blocks. The specification of the color R and the color B is such that pixel data of the same color is generated in the block at the diagonal position. This is the same definition as the G square lattice RB perfect checkerboard pattern.

The pixel data of the designated color at this position is generated for the pixel data in the block which does not correspond to the designated color (sub-step SS804). Processing in this sub-step, for example, when the designated color block _{(G 11, B 20, R} 22, G 31) is the color G, since there is no pixel data of the color G is the pixel B ₂₀ and the pixel R ₂₂ , generated by two interpolation of pixel data G ₂₀ and the pixel data G ₂₂ it does not originally exist with the surrounding pixel data. Although not specifically described, the interpolation processing may use, for example, color interpolation development processing. When the color G is specified, two pixels of the remaining colors R and B are generated. When the color R is specified, three pixels of the remaining colors G and B are generated. When the color B is specified, three pixels of the remaining colors G and R are generated.

Next, pixel data of the designated color is generated (substep SS806). G ₁₁ , G ₂₀ , G
₂₂ , and G ₃₁ to the virtual pixel G ₂₁

[0106]

G ₂₁ = (G ₁₁ + G ₂₀ + G ₂₂ + G ₃₁ ) / 4 (9) is generated. In this way, the four pixel data is combined into one pixel data by the averaging process of the four pixel data.

The reduction correction processing of the supplied image data is 1
It is determined whether the process has been completed for the frame (substep SS808). When not completed (NO),
Proceed to the next block selection. When the processing has been completed for all the frames (YES), it is determined that the processing has been completed, and the process returns to the return. Return to the main routine.

Next block selection processing (substep SS)
In 810), an adjacent block which has been divided into blocks in advance is designated. Explaining the selection direction with the colors of the virtual pixels in FIG. 15, the colors G, G, G,... Are horizontally selected, and then the colors R, B,. R,
.., And a selection of shifting by half a block in the vertical direction is performed. After selecting this block, the process returns to the specification of the color to be generated (substep SS802).

By performing the reduction correction processing in this manner, an image in which the original G square lattice RB perfect checkerboard pattern is maintained even when the image size is reduced is obtained.

Note that this image processing method is not limited to the present embodiment, and the number of peripheral pixels used in a block is reduced and corrected by a method of adding 9 pixels, 16 pixels, and 2 to the Nth power of pixels. Needless to say, processing can be performed. Here, N is an integer.

The relationship in which this image is obtained will be briefly described (see FIG. 17A). When represented by the spatial frequency (h, v), the spatial frequency distribution becomes a square because the color G of the G square lattice RB perfect checkered pattern is arranged in the square lattice. Since the colors R and B are arranged at diagonal positions, a quadrilateral (square) having a vertex at a cutting point cut by the color G on the spatial frequency axes h and v is formed.

By the way, when the reduction correction processing is performed, the arrangement of the space newly set as the target pixel position for the reduction has exactly the same relationship as the case of the above-described pattern color G, as is apparent from FIG. Further, in the case of the other embodiments, since the G square RB complete checkerboard pattern is newly generated by performing the reduction correction processing, it can be seen that the spatial frequency structure is generated without being damaged (FIG. 17). (dashed line in (a)).
As a result, since this reduction correction processing is obtained in the same arrangement as the square grid obtained by the three-plate method, the image actually obtained is 3
It becomes comparable to the plate method.

Further, the pixel data generated by the reduction correction process does not need to be interpolated by performing the LPF process when creating the high-frequency luminance data _GH so as to correspond to the increase in the number of pixels. Then, in the reduction processing, it is sufficient to simply perform the averaging. This means that no false signal is generated in the reduction correction processing due to interpolation by LPF processing.

As a comparative example, the spatial frequency distribution in the case of the Bayer pattern will be examined. In this case, without shifting the pixels
It has the same relationship as the position where the G stripe RB perfect checkered pattern is rotated by 45 °. From this, the color G of the Bayer pattern
Is a quadrilateral rotated 45 ° compared to the square of the color G of the G square lattice RB perfect checkered pattern, and the colors R and B are squares formed by cutting each side of this quadrilateral. Therefore, the color R,
The spatial frequency distribution of B is inside the quadrilateral. When the Bayer pattern is subjected to the reduction correction processing performed in the present embodiment, when the spatial frequency distribution is calculated in consideration of the positional relationship of the generated color G, the spatial frequency distribution of the color R or the color B is obtained. It turns out that they are the same.

This is, as shown in this embodiment, a G square lattice
It is shown that by performing the reduction correction process using the pixel data of the RB complete checkerboard pattern, the obtained image can maintain higher spatial frequency characteristics than the Bayer pattern even when the image size is reduced.

FIG. 18 shows the response characteristics of the color G signal to the frequency distribution. The frequency-response characteristics in FIGS. 18 (a) and 18 (b) show the characteristics in the oblique direction and the vertical and horizontal directions, respectively. As is apparent from this characteristic, the response characteristics in the vertical and horizontal directions are superior to those in the oblique direction. As described above, the present invention is effective processing in image processing in which YC processing is performed using, for example, an image sensor having 10 million pixels or more.

In the above-described embodiment, the case where the supplied image data is supplied in the G square lattice RB complete checkerboard pattern has been described. However, the present invention is not limited to these embodiments, and the G square photon RB A checkered pattern or a complementary color system may be used. In the case of the G square photon RB checkerboard pattern, pixel data around the target pixel is obtained by mixing colors R and B. Pixel data of a color different from the color to be obtained can be handled by interpolating and generating pixel data of the same color as the color to be obtained at this position from pixel data around the pixel data.

With the above-described configuration, the reduction correction unit sets the target pixel as a real pixel or a virtual pixel to generate pixel data of a desired color, and uses the pixel data of the target pixel as a representative value of surrounding pixel data. Using, while reducing the number of pixels of each color used as a whole, the green pixels constituting the structural image were arranged in a square lattice to obtain reduced pixels, and further obtained in the reduction correction unit in the broadband unit. By subjecting the pixel data to a process of broadening the bandwidth, it is possible to further enhance the image quality and suppress false signals to the image as compared with a case where the image size is reduced by simply thinning out the pixels, and the optical
The configuration that eliminates the use of LPF can be simplified.

[0119]

As described above, according to the image processing apparatus of the present invention, the pixel data of a desired color is generated by setting the target pixel as a real pixel or a virtual pixel by the reduction arithmetic processing means, and the pixel data of the target pixel is surrounded. It is used as a representative value of the pixel data, and is determined in consideration of the arrangement of green pixels constituting a structural image while reducing the number of pixels of each color used as a whole. By performing the process of increasing the bandwidth on the pixel data obtained, it is possible to further improve the image quality and suppress the false signal to the image as compared with the case where the image size is reduced by simply thinning out the pixels.

Further, according to the image processing method of the present invention, a target pixel is set to a real pixel or a virtual pixel by using image data including pixel data in which pixels having a square lattice arrangement are assigned colors related to luminance data. A plurality of pixel data of a desired color is generated from surrounding pixel data surrounding the target pixel as a pixel, and the generated pixel data of the desired color is subjected to a frequency band broadening process. The data is used as the representative value of the surrounding pixel data, and by taking into account the green pixel arrangement when constructing the image while reducing the number of pixels of each color used as a whole, it is possible to increase the resolution of the structural image and Since the false signal is suppressed and the pixel data obtained by the band widening unit is subjected to the band widening process, the image quality can be further improved.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of a digital camera to which an image processing device according to the present invention is applied.

FIG. 2 is a schematic view of an imaging unit used in the digital camera of FIG. 1 as viewed from an incident light side.

FIG. 3 is a block diagram illustrating a schematic configuration of a normal correction unit of the digital camera in FIG. 1;

FIG. 4 is a block diagram illustrating a schematic configuration of a reduction correction unit of the digital camera in FIG. 1;

FIG. 5 is a diagram illustrating a color used in a reduction correction process performed by a reduction correction unit illustrated in FIG. 4;
FIG. 4 is a schematic diagram illustrating a positional relationship between a target pixel of G and a pixel used for processing.

6 is a diagram illustrating a color used in a reduction correction process performed by the reduction correction unit illustrated in FIG. 4;
FIG. 4 is a schematic diagram illustrating a positional relationship between target pixels of R and B and pixels used for processing.

FIG. 7 is a block diagram showing a schematic configuration of a broadband processing unit in the digital camera of FIG. 1 corresponding to a reduction correction processing mode.

FIG. 8 is a main flowchart for explaining the operation of the digital camera in FIG. 1;

FIG. 9 is a flowchart illustrating a procedure of a normal interpolation signal process (subroutine SUB1) shown in FIG. 8;

FIG. 10 is a flowchart illustrating a procedure (subroutine SUB2) of a reduction correction process in the reduction correction processing mode illustrated in FIG. 8;

11 is a schematic diagram illustrating a range of a processing area in the reduction correction processing illustrated in FIG.

12 is a flowchart illustrating a procedure (subroutine SUB3) of a broadband process performed on pixel data generated by the normal interpolation signal processing procedure illustrated in FIG. 9;

13 is a flowchart illustrating a procedure (subroutine SUB4) of a broadband process performed on pixel data generated by the procedure of the reduction correction process illustrated in FIG.

FIG. 14 is a schematic diagram illustrating a method of blocking in the image processing method of the present invention.

FIG. 15 is a schematic diagram illustrating a relationship between pixels and positions when a block is formed by using one of the blocks illustrated in FIG. 15;

FIG. 16 is a flowchart illustrating a procedure (subroutine SUB8) of a reduction correction process performed on pixel data obtained from the relationship in FIG.

17 is a graph showing a spatial frequency distribution when a reduction correction process is performed on the G square lattice RB perfect checkerboard pattern and the Bayer pattern shown in FIG. 5;

18 is a chart showing the frequency response of the low-pass filter in the diagonal, vertical, and horizontal directions in the G square lattice RB perfect checkerboard pattern of FIG. 17;

[Explanation of symbols]

 10 Digital camera 12 Optical lens system 14 Operation unit 16 System bus 18 System control unit 26 Imaging unit 30 Signal processing unit 30a Data correction unit 30b Interpolation processing unit 30c Broadband processing unit 32 Compression / decompression processing unit 38 Printer 40 I / F unit 44 Normal correction unit 46 Reduction correction unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 9/07 H04N 101: 00 5C079 9/64 1/40 D // H04N 101: 00 1/46 ZF Term (Reference) 5B057 BA02 BA12 CA01 CA08 CA12 CB01 CB08 CB12 CB16 CC02 CD06 CE03 CE06 CE16 CH09 5C065 AA03 BB10 BB12 CC01 CC02 CC03 CC08 CC09 DD02 DD17 EE06 GG02 GG15 GG21 HH02 5C066 AA01 GA02 EC02 KE05 KE07 KM05 5C076 AA22 AA26 BA01 BA06 BB07 BB40 5C077 LL19 MP08 PP20 PP32 PP34 RR19 RR21 5C079 HB01 HB04 JA12 LA02 LA10 LA14 LA26 LA28 LA37 MA11 MA17 NA02 NA04 PA03

Claims (10)

[Claims] 1. A pixel of each color obtained by displacing a plurality of image sensors that photoelectrically convert color-separated incident light in a row and / or column direction by a half pitch with respect to a geometric center of an adjacent image sensor. In an image processing apparatus that generates image data that provides a high-quality image by using data, the apparatus includes, in the image data, pixel data in which the color of the pixel data is obtained in a two-dimensional square lattice shape. The same color or different color as one color different from the color related to the generation of the brightness data, the pixel data at one diagonal position sandwiching the color related to the generation of the brightness data as one color different from the color related to the generation of the brightness data, and the brightness data Pixels that are supplied in the same or different colors as pixel data at the other diagonal position sandwiching the color related to the generation of the color as the color related to the generation of the luminance data and the other color different from the one color The supplied pixel data is used as the data from the existing real pixels, and the virtual pixel having the center of the area surrounded by the real pixels as the geometric center. A reduction operation processing unit that sets a target pixel used for reduced display to the real pixel or the virtual pixel and generates pixel data of a desired color from surrounding pixel data surrounding the target pixel; An image processing apparatus, comprising: a band-widening unit for broadening the frequency of the pixel data. 2. The apparatus according to claim 1, wherein the reduction operation processing means converts the target pixel to a real pixel located at the center of each area surrounded by a color related to the generation of the luminance data, and An image processing apparatus, comprising: a first calculation unit that calculates pixel data of a color different from a color inherent in a pixel and a color related to generation of the luminance data. 3. The apparatus according to claim 2, wherein the first calculating unit is configured to determine a position of the target pixel of the one color or the other color surrounded by real pixels of a color related to the generation of the luminance data. A first averaging unit that performs an averaging process using pixel data obtained from actual pixels of a color related to the generation of the luminance data; and the one of the one color and the other color is Second averaging in which pixel data surrounding the target pixel is used as it is as pixel data of the target pixel and pixel data surrounding the target pixel is converted into pixel data of the same color as pixel data of the remaining color, and averaging is performed. And an image processing apparatus. 4. The apparatus according to claim 1, wherein the reduction operation processing unit performs blocking including the virtual pixel at the center of four of the existing pixels in the supplied pixel data, and Is used as the virtual pixel of each block, and one of three colors to be color-separated into each block is designated, and pixel data of the designated color is calculated using pixel data included in each block. An image processing apparatus characterized by including means. 5. The apparatus according to claim 4, wherein the second calculating means performs color specification of the same color as the color of pixel data to which the specified color for each of the blocks is originally supplied; A designated color data generating means for generating pixel data of the pixel of the different color from pixel data of the same color as the designated color surrounding the pixel of a different color from the designated color among the pixel data in each block. An image processing apparatus characterized by the above-mentioned. 6. An image pickup device that separates incident light from an object field into three colors and photoelectrically converts the obtained three colors of incident light with respect to a geometric center of an adjacent image pickup device. And / or a plurality of pixels arranged at half-pitch shift in the column direction, the image sensor is used as an actual pixel, and each of image signals obtained from the image sensor using the image sensor is converted into digital pixel data. An image processing method for generating a luminance pixel and a color difference data corresponding to each pixel by calculating an enclosed space as a newly formed virtual pixel and calculating based on the supplied pixel data, wherein the color of the pixel data is The pixel data obtained two-dimensionally in a square lattice form is a color related to the generation of the luminance data included in the image data, and the pixel data at one diagonal position sandwiching the color related to the generation of the luminance data is the luminance data The same color or a different color as one color different from the color related to the generation, and the pixel data at the other diagonal position sandwiching the color related to the generation of the luminance data is the color different from the color related to the generation of the luminance data and the other color. Using the pixel data supplied in the same color or a different color as the color, the supplied pixel data is taken as the data from the existing real pixel, and the center of the area surrounded by the real pixel is set as the geometric center. A first pixel that generates a pixel data of a desired color from surrounding pixel data surrounding the target pixel by using the target pixel used for reduced display as the real pixel or the virtual pixel among the supplied pixel data as the virtual pixel. An image processing method, comprising: a step of: broadening a frequency band of pixel data obtained as the target pixel in frequency. 7. The method according to claim 6, wherein the first step is to convert the target pixel into a real pixel located at the center of each area surrounded by a color involved in the generation of the luminance data, An image processing method comprising calculating pixel data of a color different from a color inherent to the pixel and a color related to the generation of the luminance data. 8. The method according to claim 7, wherein the third step is the position of the target pixel of the one color or the other color surrounded by real pixels of a color involved in the generation of the luminance data. A fourth step of performing an averaging process using pixel data obtained from real pixels of a color related to the generation of the luminance data; and converting any one of the one color and the other color into the target pixel. A pixel data surrounding the target pixel among the three colors used for the reduction pixel, and performing an averaging process by converting the pixel data to the same color pixel data as the remaining color pixel data. An image processing method comprising: 9. The method according to claim 6, wherein the first step is to block the supplied pixel data including the virtual pixel at the center of four of the real pixels, and to set the target pixel to A sixth step of specifying one of three colors for each block as the virtual pixel of each block, and calculating pixel data of the specified color using pixel data included in each block. Characteristic image processing method. 10. The method according to claim 9, wherein
In the step (c), the designated color for each of the blocks is designated in the same pattern as the color of the pixel data originally supplied.
And an eighth step of generating pixel data in the pixel of the different color from the pixel data of the same color as the designated color around the pixel of a different color from the designated color among the pixel data in each block. An image processing method comprising: