JP2001177767A - Image data filtering device and method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To suppress a false signal of an image expressed in image data obtained from a honeycomb type CCD without deteriorating the resolution. SOLUTION: An image of an object is picked up by using the honeycomb type CCD where many photoelectric converting elements are arranged in row and column directions, photoelectric converting elements are arranged on odd number rows of odd number columns and arranged on even number rows of even number columns. A low pass filter circuit having a filter characteristic in an oblique direction applies filtering processing to image data obtained by the CCD. A false signal in the oblique direction can be reduced even from an image expressed by the image data obtained from the honeycomb type CCD where the photoelectric conversion elements are arranged in the oblique direction.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an arrangement in which a large number of pixels are arranged in a column direction and a row direction, photoelectric conversion elements are arranged in odd rows or even rows for odd columns, and photoelectric conversion elements are arranged in even rows or odd rows for even columns. The present invention relates to a device and a method for filtering image data obtained by imaging a subject using a solid-state electronic imaging device in which elements are arranged.

[0002]

BACKGROUND OF THE INVENTION In the natural world, energy is concentrated in the horizontal and vertical directions when viewed in frequency space.
In human vision, the spatial frequency is lowest in the direction at 45 degrees from the horizontal direction.

Considering these facts, if the number of pixels is the same, pixels are arranged in odd or even rows for odd columns and pixels are arranged for even columns, rather than arranging pixels in columns and rows. The resolution is higher when pixels are arranged in an even or odd row (so-called honeycomb arrangement).

A CCD having a honeycomb arrangement (honeycomb type C)
In order to increase the horizontal or vertical reproduction band of the image represented by the image data obtained from the CD), it is necessary to increase the cutoff frequency for cutting the low frequency components of the image data in the filtering process. There is. However, when the horizontal or vertical cutoff frequency is increased, the resolution in the horizontal and vertical directions is improved, but the resolution in the diagonal direction is also improved, and false signals and false colors generated in the diagonal direction are increased. I do.
If filtering processing for cutting low frequency components is performed to reduce false signals and false colors in the oblique direction, the resolution in the horizontal and vertical directions may be unnecessarily deteriorated.

[0005]

DISCLOSURE OF THE INVENTION The present invention relates to a so-called honeycomb type CC.
An object is to improve the resolution while suppressing a false signal of the image represented by the image data obtained from D.

The image data filtering devices according to the present invention are arranged in a large number in a column direction and a row direction.
A solid-state electronic imaging device in which photoelectric conversion elements are arranged in odd rows or even rows for odd columns and photoelectric conversion elements are arranged in even rows or odd rows for even columns is provided. Imaging means for outputting image data representing an image, and interpolating pixels between adjacent pixels in a row direction and a column direction in a subject image represented by the image data output from the imaging means.
Interpolating means for interpolating the image data, synchronizing means for synchronizing image data for a plurality of lines out of the image data interpolated by the interpolating means, and a filter having a filter characteristic in an oblique direction. The image processing apparatus further includes a filtering unit for filtering the image data synchronized by the synchronization unit.

[0007] The present invention also provides a method suitable for the above device. That is, in this method, a large number of cells are arranged in the column direction and the row direction, photoelectric conversion elements are arranged in odd or even rows for odd columns, and photoelectric conversion elements are arranged in even or odd rows for even columns. Including the solid-state electronic image pickup device arranged, image data representing a subject image is obtained by taking an image of the subject, and between the pixels adjacent in the row direction and the column direction in the subject image represented by the obtained image data. The image data is interpolated so as to interpolate the pixels of the image data, the image data of a plurality of rows of the interpolated image data is synchronized, and the synchronization processing is performed using a filter having a filter characteristic in an oblique direction. The filtering process is performed on the processed image data.

According to the present invention, a large number are arranged in the column direction and the row direction, photoelectric conversion elements are arranged in odd rows or even rows for odd columns, and photoelectric conversion elements are arranged in even rows or odd rows for even columns. A subject is imaged using a solid-state electronic imaging device in which the elements are arranged. That is, for example, in a solid-state electronic imaging device in which photoelectric conversion elements are arranged in odd rows for odd columns and in even rows, photoelectric conversion elements are arranged in even rows for odd columns. For the even columns, a solid-state electronic imaging device in which photoelectric conversion elements are arranged in odd rows is used. Interpolation processing is performed on image data obtained by imaging so as to interpolate pixels adjacent in the row and column directions. Of the interpolated image data, image data for a plurality of rows (for example, three rows) are synchronized. A filtering process is performed on the synchronized image data using a filter having a filter characteristic in an oblique direction.

According to the present invention, filter characteristics in an oblique direction (for example, characteristics for filtering pixels existing in an oblique direction with respect to a column direction or a row direction).
Filtering (processing to adjust the level of image data, etc.) is performed using a filter (digital filter) having a diagonal false signal and a false color in the oblique direction without deteriorating the resolution in the horizontal and vertical directions. Etc. can be reduced.

The first filtering means for filtering the image data synchronized by the synchronizing means using a filter having a filter characteristic in the diagonal left direction, and the filter having a filtering characteristic in the diagonal right direction. The image data can be filtered using the second filtering means for filtering the image data filtered by the first filtering means.

Since the filtering process is performed twice using the two filtering means, the accuracy of the filtering can be improved.

The filtering means includes a first filtering means for filtering the image data synchronized by the synchronizing means using a filter having a filter characteristic in a diagonal left direction, and a filter characteristic in a diagonal right direction. The image processing apparatus may further include a second filtering unit that performs a second filtering by using the filter to filter the image data synchronized by the synchronization unit.

In this case, it is determined whether or not the edge portion of the image in the subject image represented by the image data interpolated by the interpolation means is in a diagonally left direction, a diagonally right direction, or neither of these directions. . The image data filtered by the first filtering means is selected and output by judging that the image data is in the diagonal left direction, and the image data filtered by the second filtering means is selected by judging that the image data is in the diagonal right direction. It is preferable that the image data output from the synchronizing means be selected and output when it is determined that the image data is not in any of the directions.

It is determined whether the direction of the edge of the image in the subject image is diagonally left or diagonally right or any of these directions, and the filtering process is changed according to the result of the determination. Filtering processing with different filter characteristics can be performed in accordance with the edge direction of the image in the subject image so that a false signal and a false color do not occur. For example, if the edge is in the diagonally left direction, filtering is performed using a filter having characteristics in the diagonally left direction. If the edge is in the diagonal right direction, filtering having characteristics in the diagonal right direction is performed. If the edge is neither in the left diagonal direction nor in the right diagonal direction, no filtering processing is performed.

The filtering process is performed in an appropriate direction according to the edge direction of the image in the subject image.

[0016]

DESCRIPTION OF THE PREFERRED EMBODIMENTS FIG. 1 is a block diagram showing a part of the electrical configuration of a digital camera. FIG. 2 schematically shows a part of the light receiving surface of the CCD of the digital camera shown in FIG.

The CCD 10 is a so-called honeycomb type CCD.
It is. In this honeycomb type CCD, a large number of photoelectric conversion elements 1 are arranged in a row direction and a column direction. These photoelectric conversion elements 1 are arranged at odd-numbered rows for odd columns, and are arranged at even-numbered rows for even columns. Therefore, the photoelectric conversion elements 1 are not arranged at the positions of the even rows for the odd columns, and the photoelectric conversion elements 1 are not arranged at the positions of the odd rows for the even columns. Of course, for the odd columns, the photoelectric conversion element 1 is arranged at the position of the even row, and for the even columns, the photoelectric conversion element 1 is placed at the position of the odd row.
May be arranged.

A signal output from the photoelectric conversion element 1 is output from the CCD 10 as a video signal representing a subject image.
The video signal output from the CCD 10 is input to an analog / digital conversion circuit 11 and is converted into digital image data. An RGB color filter is arranged on the light receiving surface of the CCD 10 (on the light receiving surface of the photoelectric conversion element 1), and the analog / digital conversion circuit 11 outputs RGB image data serially. Not even.

The image data output from the analog / digital conversion circuit 11 is input to a gamma correction circuit 15 via a linear matrix circuit 12, an offset circuit 13, and a gain correction circuit 14. The image data gamma-corrected by the gamma correction circuit 15 is input to an RGB interpolation circuit 18 and a high luminance data generation circuit 16.

In the high luminance generation circuit 16, the input RG
High brightness data is generated from the B image data. The generated high luminance data is input to the interpolation circuit 17. Interpolator 17
In FIG. 3, as shown in FIG. 3, a pixel adjacent to the pixel 2 in the vertical and horizontal directions (the photoelectric conversion element 1 exists) using the high luminance data representing the pixel 2 corresponding to the photoelectric conversion element 1 forming the CCD 10. (Pixels corresponding to non-existing positions) are interpolated. Specifically, the pixels in the even-numbered rows in the odd-numbered columns and the pixels in the odd-numbered rows in the even-numbered columns in which the photoelectric conversion element 1 of the CCD 10 does not exist are generated using the high luminance data indicating the adjacent pixels 2 (see FIG. Indicates that a pixel generated by interpolation is represented by a black circle 3). The interpolation process is performed, for example, by arithmetically averaging high luminance data representing adjacent pixels. Of course, other interpolation methods can be used.

The high-luminance data interpolated by the interpolation processing circuit 17 is input to an RGB interpolation circuit 18.

The RGB interpolation circuit 18 uses the high luminance data generated by the interpolation circuit 17 and the RGB image data output from the gamma correction circuit 15 to convert the RGB image data representing the interpolation pixel 3 for the RGB image data. Generated. Thereby, the number of pixels of the image represented by the image data output from the RGB conversion circuit 18 is equal to the number of pixels of the CCD.
The number of pixels of the image represented by the video signal output from 10 is almost twice as large.

The RGB image data output from the RGB interpolation circuit 18 is input to a YC conversion circuit 19, where luminance data Y and color difference data C (RY and BY) are generated. The luminance data Y generated in the YC conversion circuit 19
Is input to the luminance filtering circuit 20. In the luminance filtering circuit 20, filtering processing is performed on the input luminance data. Details of this filtering process will be described later.

The luminance data output from the luminance filtering circuit 20 is input to the contour correction circuit 21. The contour correction circuit 21 performs a contour correction process (eg, edge enhancement) on the input luminance data. The contour-corrected luminance data is output.

The color difference data C generated in the YC conversion circuit 19 is input to a color difference filtering circuit 22. In the color difference filtering circuit 22, a filtering process is performed on the color difference data in the same manner as the process in the luminance filtering circuit 20. Color difference filtering circuit
The color difference data subjected to the filtering processing in 20 is input to the hue correction circuit 23. In the hue correction circuit 23, hue correction is performed on the color difference data C, and the color difference data C is output.

The luminance data output from the contour correction circuit 21 and the color difference data output from the hue correction circuit 23 are data-compressed and recorded on a memory card or the like.

FIG. 4 shows the configuration of the color difference filtering circuit 20.

The color difference data output from the YC conversion circuit 19 is input to a synchronization circuit 31. The synchronization circuit 31 has three line memories (not shown). With these three line memories, three lines of color difference data are synchronized. The synchronized luminance data is
Input to the filter 32.

The low pass filter 32 has a filter characteristic as shown in FIG. This filter characteristic is for performing band limiting processing on pixels in oblique directions. The color difference data output from the synchronization circuit 31 is low.
By inputting to the path filter 32, the low pass
The filter 32 can reduce false signals generated in pixels in oblique directions. The color difference data output from the low pass filter 32 is input to the hue correction circuit 23 as described above.

The luminance filtering circuit 20 can be similarly configured using a low-pass filter having a cutoff frequency higher than the cutoff frequency of the filter characteristic for luminance data. A high-resolution luminance image can be obtained.

FIG. 6 is a block diagram showing another circuit configuration of the color difference filtering circuit 22.

In FIG. 6, a first low-pass filter 34 and a second low-pass filter 36 are included. The first low pass filter 34 has a filter characteristic in the diagonally left direction as shown in FIG. 7A, and performs a process of reducing a false signal for pixels existing in the diagonally left direction. The second low pass filter 36 is shown in FIG.
As shown in (B), it has a filter characteristic in the diagonal right direction, and performs a false signal reduction process on pixels existing in the diagonal right direction.

The synchronizing circuits 33 and 35 are the same as the synchronizing circuit 31 shown in FIG. 4, and include three line memories.

The color difference data output from the YC conversion circuit 19 is input to the synchronization circuit 33. In the synchronization circuit 33,
The color difference data synchronization processing is performed. The synchronized three-line color difference data is input to the first low-pass filter 34, and a false signal reduction process is performed on pixels in the diagonally left direction. The color difference data output from the first low pass filter 1 is input to a second low pass filter 36 via a synchronization circuit 35. Second low pass filter
At 36, a false signal reduction process is performed on the pixels in the diagonally right direction.

The color difference data output from the second low pass filter 36 becomes output luminance data of the color difference filtering circuit 22 and is input to the hue correction circuit 23 as described above.

In the circuit shown in FIG. 6, the false signal reduction processing for the pixels in the diagonal left direction and the false signal reduction processing for the pixels in the diagonal right direction are performed using different low-pass filters. Therefore, a relatively accurate false signal reduction process can be realized.

FIG. 8 is a block diagram showing still another configuration of the color difference filtering circuit 20.

In this circuit, an image represented by input image data has an edge in a diagonal left direction, an edge in a diagonal right direction, and a diagonal left or diagonal direction. Detect if there is no edge. If there is an edge in the diagonal left direction, processing to reduce false signals of pixels in the diagonal left direction is performed. If there is an edge in the diagonal right direction, processing to reduce false signals in pixels in the diagonal right direction is performed. If no edge is found in any of the left and right diagonal directions, the process for reducing the false signal is not performed in either direction.

In the synchronizing circuit 41, of the color difference data output from the YC conversion circuit 19, three lines of color difference data are synchronized. The synchronized color difference data is supplied to a first low-pass filter 42, a second low-pass filter 43,
Input to the oblique direction detection circuit 44 and selector 45.

The first low-pass filter 42 has the configuration shown in FIG.
It has characteristics in the diagonally left direction shown in FIG. 2A, and performs processing for reducing false signals for pixels in the diagonally left direction and outputs the processed pixels. The luminance data output from the first low pass filter 42 is input to a selector 45. The second row pass
The filter 43 has the characteristics in the diagonal right direction shown in FIG. 7B, and performs processing for reducing false signals for pixels in the diagonal right direction and outputs the processed pixels. Second low pass filter
The color difference data output from 43 is input to the selector 45.
The oblique direction detection circuit 44 detects whether the edge direction included in the image represented by the input color difference data is the left oblique direction, the right oblique direction, or any of the left oblique direction and the right oblique direction. A signal indicating the detection result is input to the selector 45.

If the signal supplied from the oblique direction detection circuit 44 to the selector 45 is a signal indicating the left oblique direction, the color difference data output from the first low pass filter 42 is output from the selector 45, Output from the filtering circuit 22. If the signal supplied from the diagonal direction detection circuit 44 to the selector 45 is a signal indicating the right diagonal direction, the color difference data output from the second low pass filter 43 is output from the selector 45 and the color difference filtering circuit 22 Output. If the signal supplied from the oblique direction detection circuit 44 to the selector 45 is not a signal indicating either the left oblique direction or the right oblique direction, the output of the synchronization circuit 41 is output from the selector 45.

Since the direction of the edge in the image represented by the image data is detected and the filtering process is performed so as to reduce the generation of the false signal according to the detected direction, a relatively appropriate filtering process is performed. Can be applied.

FIGS. 9 to 12 are diagrams for explaining the procedure for detecting the edge direction in an image in the oblique direction detection circuit 44. FIG. FIG. 9 and FIG. 11 show pixels constituting a part of an image. FIG. 10 and FIG. 12 show the characteristics of the filter.

9 and 11, in the row direction (5 rows)
And the pixels P1 to P25 in the column direction (five columns) indicate the same part of the image.

First, filters F1, F2 and F3 having filter characteristics as shown in FIG. 10 are prepared. Although the filters F1, F2 and F3 have the same filter characteristics, different symbols are used in this embodiment. Using the filter F1, of the pixels P1 to P25, the upper left three rows x
Pixels P1, P2, P3, P6, P7, P8, P1 in three columns
1, filtering is performed on P12 and P13. The output value of this filtering processing is defined as f1. Similarly, pixels P7, P8, P9, P12, and P3 of 3 rows × 3 columns at the center portion
A filtering process is performed on 13, P14, P17, P18 and P19 using the filter F2. The output value of this filtering process is f2. Further, a filtering process is performed on the pixels P13, P14, P15, P18, P19, P20, P23, P24 and P25 in the lower right 3 rows × 3 columns using the filter F3. The output value of this filtering process is defined as f3.

Further, filters F4, F5 and F6 having filter characteristics as shown in FIG. 12 are prepared. Although the filters F4, F5 and F6 have the same filter characteristics, different symbols are used in this embodiment. Using the filter F4, pixels P3, P4, P5, P8, P9, P10, P in the upper right 3 rows × 3 columns among the pixels P1 to P25.
13, filtering is performed on P14 and P15. The output value of this filtering process is defined as f4. Similarly,
Pixels P7, P8, P9, P12 of 3 rows × 3 columns at the center part
A filtering process is performed on P13, P14, P17, P18 and P19 using the filter F5. The output value of this filtering process is defined as f5. Further, pixels P11, P12, P13, P16, P17, P18, P21, P22 in the lower left 3 rows × 3 columns
And P23 are subjected to a filtering process using the filter F5. The output value of this filtering process is defined as f5.

In these filtering processes, the obtained output values f1, f2, f3, f4, f5 and f6
If Equation 1 is satisfied by using the pixel P1 as shown in FIG.
To P25, there is an edge in the diagonal left direction. If Expression 2 is satisfied, as shown in FIG. 11, the image formed by the pixels P1 to P25 has an edge in the diagonally right direction. If neither expression 1 or expression 2 is satisfied, no edge is detected in any of the left and right diagonal directions.

If (not ((| f4> threshold) and (| f5 |> threshold) and (| f6 |> threshold)) and ((| f1 |> threshold) and (| f2 |> threshold) and (| f3 |> threshold) ... Equation 1

Equation 1 shows that the absolute values of f4, f5 and f6 are not all greater than a predetermined threshold, and that the absolute values of f1, f2 and f3 are all greater than a predetermined threshold.

If (((| f4> threshold) and (| f5 |> threshold) and (| f6 |> threshold)) and not ((| f1 |> threshold) and (| f2 |> threshold) and (| f3 |> threshold) ... Equation 2

Equation 2 indicates that the absolute values of f4, f5 and f6 are all larger than the threshold value, and that the absolute values of f1, f2 and f3 are not all larger than the predetermined threshold value.

In the above edge detection processing, when an edge in the diagonal left direction is detected, as described above, the filter having the filter characteristic in the diagonal left direction (see FIG. 7A) is used to filter the diagonal left direction. Output image data of a filter having characteristics is output from the selector 45. When the edge in the diagonal right direction is detected, the output image data of the filter having the filter characteristic in the diagonal right direction (see FIG. 7B) is output from the selector 45. If no edge in any direction is detected, the selector outputs the image data which is output from the synchronization circuit 41 and is not subjected to the filtering process.

In the embodiment described above, the low pass
Although the filtering process using the filter has been described, it goes without saying that the filtering process is not limited to the low pass filter. It can also be used for other processing using a filter, for example, contour correction processing, smoothing processing, and the like. Further, in the above-described embodiment, the filtering process is performed using the filter of 3 rows × 3 columns, but it goes without saying that other filters can be used.

[Brief description of the drawings]

FIG. 1 is a block diagram showing an electrical configuration of a digital camera.

FIG. 2 shows a part of a photoelectric conversion element constituting a honeycomb CCD.

FIG. 3 shows a state where pixels are interpolated.

FIG. 4 is a block diagram illustrating an electrical configuration of a luminance filtering circuit.

FIG. 5 shows an example of a filter characteristic.

FIG. 6 is a block diagram illustrating another electrical configuration of the luminance filtering circuit.

FIGS. 7A and 7B show another example of filter characteristics.

FIG. 8 is a block diagram showing another electrical configuration of the luminance filtering circuit.

FIG. 9 illustrates an example of a pixel forming an image.

FIG. 10 shows an example of a filter characteristic.

FIG. 11 shows an example of a pixel constituting an image.

FIG. 12 shows an example of a filter characteristic.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 Photoelectric conversion element 2,51 pixel 3 Interpolated pixel 10 CCD 17 Interpolation circuit 18 RGB interpolation circuit 20 Luminance filtering circuit 22 Color difference filtering circuit 31,33,35,41 Synchronization circuit 32,34,36,41 Low pass・ Filter 44 Oblique direction detection circuit 45 Selector

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) H04N 9/07 H04N 1/40 101D F-term (Reference) 4M118 AA05 AA10 AB01 BA10 CA02 FA06 GC08 GC14 5C024 AA01 CA06 CA11 DA01 DA07 FA01 FA12 GA11 HA02 HA08 HA14 5C051 AA01 BA03 DA06 DB01 DC02 DE13 DE19 EA01 FA01 FA02 FA04 5C065 BB13 CC01 CC09 DD02 GG05 GG13 GG18 5C077 LL02 MM03 MM22 MP07 MP08 PP01 PP32 PP33 PP08 PP47 PP59 P06

Claims (4)

[Claims] 1. A large number of photoelectric conversion elements are arranged in a column direction and a row direction, photoelectric conversion elements are arranged in odd rows or even rows for odd columns, and photoelectric conversion elements are arranged in even rows or odd rows for even columns. Means for outputting image data representing a subject image by taking an image of the subject, and a row direction and a column direction of the subject image represented by the image data output from the imaging means. Interpolating means for interpolating the image data so as to interpolate pixels between pixels adjacent to the image data; synchronizing means for synchronizing image data for a plurality of rows of the image data interpolated by the interpolating means; And a filter for filtering the image data synchronized by the synchronization means using a filter having a filter characteristic in an oblique direction. Image data filtering device comprising: 2. The image processing apparatus according to claim 1, wherein the filtering means uses a filter having a filter characteristic in a diagonal left direction to filter the image data synchronized by the synchronizing means, and is arranged in a diagonal right direction. 2. The image data processing apparatus according to claim 1, further comprising: a second filtering unit configured to perform a filtering process on the image data that has been subjected to the filtering process by the first filtering unit using a filter having a filtering characteristic with respect to the selected pixel. Filtering device. 3. The filtering means according to claim 1, wherein said filtering means uses a filter having a filter characteristic in a diagonal left direction to filter the image data synchronized by said synchronizing means, and a filter characteristic in a diagonal right direction. A second filtering means for filtering the image data synchronized by the synchronizing means using a filter having: Determining means for determining whether the edge portion of the image is in the left diagonal direction, the right diagonal direction, or any of these directions; and the first determining means determines that the image is in the left diagonal direction. Select and output the image data filtered by the filtering means, The image data filtered by the second filtering means is selected and output by judging the direction, and the image data output from the synchronizing means is selected by judging that it is not in any of those directions. 2. The image data filtering device according to claim 1, further comprising a selector for selecting and outputting the selected data. 4. A plurality of photoelectric conversion elements are arranged in a column direction and a row direction, photoelectric conversion elements are arranged in odd rows or even rows for odd columns, and photoelectric conversion elements are arranged in even rows or odd rows for even columns. Image data representing a subject image by capturing the subject, and obtaining the image data representing the subject image between the adjacent pixels in the row direction and the column direction in the subject image represented by the obtained image data. Interpolating the image data so as to interpolate the pixels, synchronizing the image data for a plurality of lines out of the interpolated image data, and performing the synchronizing process using a filter having a filter characteristic in an oblique direction. Image data filtering method for filtering filtered image data.