WO2005055590A1 - 画像補間装置及び画像補間方法 - Google Patents
画像補間装置及び画像補間方法 Download PDFInfo
- Publication number
- WO2005055590A1 WO2005055590A1 PCT/JP2004/018296 JP2004018296W WO2005055590A1 WO 2005055590 A1 WO2005055590 A1 WO 2005055590A1 JP 2004018296 W JP2004018296 W JP 2004018296W WO 2005055590 A1 WO2005055590 A1 WO 2005055590A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- data
- pixel
- image
- image data
- index value
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
- H04N23/84—Camera processing pipelines; Components thereof for processing colour signals
- H04N23/843—Demosaicing, e.g. interpolating colour pixel values
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/134—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on three different wavelength filter elements
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
- H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
- H04N25/11—Arrangement of colour filter arrays [CFA]; Filter mosaics
- H04N25/13—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements
- H04N25/135—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements
- H04N25/136—Arrangement of colour filter arrays [CFA]; Filter mosaics characterised by the spectral characteristics of the filter elements based on four or more different wavelength filter elements using complementary colours
Definitions
- the present invention relates to an image interpolation device and an image interpolation method for generating a high-quality frame image from a plurality of pieces of image data input continuously in time series.
- Conventional interpolation methods include, for example, when increasing the number of lines in the vertical direction, use the values of lines above or below the line to be interpolated as they are, or use the average value of the lines above and below. Has been adopted. However, such a method has a problem that the contour of the interpolated image may be rattled or blurred when the photographing target has an oblique line.
- USP 5,825,429 proposes the following method.
- the image of the second field is converted into a frame image.
- it is determined whether or not there is any motion in the image within the window set at each position corresponding to the pixel to be interpolated in the second field on the first and third field images, and according to the determination, Perform interpolation.
- whether or not there is motion is determined by adding the absolute value difference at the pixel level between the first field corresponding to the pixel position to be interpolated and the window of the third field, and the obtained addition value Judgment is made based on whether or not a predetermined threshold is exceeded. If it is determined that there is movement between the windows, it is determined that the correlation in the window of the second field image is stronger than the correlation between the windows of the first field and the third field, and the image is vertically adjacent. Interpolated pixel data is generated based on the pixels. If it is determined that there is no motion between the windows, the direction of the correlation within the window is examined.
- the position to interpolate The pixel data above and below the pixel adjacent in the horizontal direction are extracted, and the added values are calculated.The obtained added value increases or decreases as the sum of the pixels existing in the horizontal direction increases. Determine whether or not. If the increase or decrease does not occur, an interpolated value is generated using the pixel data above and below the position to be interpolated. In addition, when the pixel value increases or decreases in such a manner, the image of that portion is determined to be a step-like region, and pixel data having a high correlation in the three directions of up and down and diagonally up and down of the pixel to be interpolated. From this, interpolation data is generated.
- Fig. 14 shows an example of the case where the judgment is wrong.
- the value of the pixel marked with “?” In the center of the figure is estimated, and the numbers in the figure are pixel value data. It is assumed that data is obtained from pixels marked with No and Tching.
- the added value becomes the same value in all cases and increases in the horizontal direction. It means that neither calorie nor decrease has occurred.
- the peripheral portion does not have a stepwise distribution, and the arithmetic mean of the upper and lower pixels is obtained as an interpolation value.
- the correlation direction is erroneously determined.
- the present invention has been made in view of the above points, and has as its object to provide an image interpolation device and an image interpolation method capable of generating a highly accurate interpolated image.
- a plurality of image data input continuously in a time series is An image interpolator for generating one image data from the data
- An index value for the predetermined direction is calculated from at least one set of pixel data arranged in a predetermined direction near a pixel position in the plurality of image data corresponding to a pixel position of the one image data.
- Index value calculation unit
- a direction determining unit configured to detect a strong correlation direction at the pixel position based on the index value calculated by the index value calculating unit;
- a switching unit configured to select and switch operation data to be used from the plurality of input image data based on the direction detected by the direction determination unit, and an operation switched by the switching unit
- An interpolator configured to obtain pixel data at the pixel position using the data
- An image interpolating device comprising:
- one piece of image data is converted from a plurality of pieces of image data that are successively input in a time series from an image sensor having a color filter disposed on the front side.
- a generated image interpolation device
- An index value for the predetermined direction is calculated from at least one set of pixel data arranged in a predetermined direction near a pixel position in the plurality of image data corresponding to a pixel position of the one image data.
- Index value calculation unit
- a direction determining unit configured to detect a strong correlation direction at the pixel position based on the index value calculated by the index value calculating unit;
- a switching unit configured to select and switch, based on the direction detected by the direction determining unit, calculation data to be used from the input plurality of image data for each color of the color filter.
- An interpolation unit configured to obtain pixel data at the pixel position by using the operation data switched by the switching unit;
- An image interpolating device comprising:
- a plurality of pieces of image data continuously and sequentially input from an image sensor having a color filter disposed on the front side are used to determine a horizontal direction and a vertical direction.
- An image interpolating apparatus that generates one image data whose size is scaled in at least one of the directions,
- Discrete pixel data is extracted from each of the plurality of image data, and a predetermined pixel data is extracted from pixel data near a pixel position corresponding to the pixel position of the one image data among the extracted pixel data.
- An index value calculation unit configured to select at least one set of pixel data arranged in the direction and to calculate an index value in the predetermined direction;
- a direction determining unit configured to detect a strong correlation direction at the pixel position based on the index value calculated by the index value calculating unit;
- a switching unit configured to select and switch, based on the direction detected by the direction discriminating unit, operation data that also uses the neutral force of the input discrete pixel data for each color of the color filter. Department and
- An interpolation unit configured to obtain pixel data at the pixel position by using the operation data switched by the switching unit;
- An image interpolating device comprising:
- the size of at least one of the horizontal direction and the vertical direction is scaled from a plurality of pieces of color image data input continuously in time series.
- Each of the plurality of color image data is extracted with discrete pixel data, and among the extracted pixel data, pixel data in the vicinity of a pixel position corresponding to the pixel position of the one image data is extracted.
- An index value calculation unit configured to select at least one set of pixel data arranged in a predetermined direction and calculate an index value in the predetermined direction;
- a direction discriminator configured to detect based on the index value calculated by the calculator,
- operation data to be used is selected and switched from among the input discrete pixel data for each of a plurality of primary colors constituting the color image data.
- An interpolation unit configured to obtain pixel data at the pixel position by using the operation data switched by the switching unit;
- An image interpolating device comprising:
- an image interpolation method for generating one image data from a plurality of image data input continuously in time series
- one piece of image data is converted from a plurality of pieces of image data continuously input in time series from an image sensor having a color filter disposed on the front side.
- the operation data to be used from the input plurality of image data is selected for each color of the color filter, and switching is avoided.
- a plurality of pieces of image data continuously and sequentially input from an image sensor having a color filter disposed on the front thereof are used to determine a horizontal direction and a vertical direction.
- Discrete pixel data is extracted from each of the plurality of image data, and a predetermined pixel data is extracted from pixel data near a pixel position corresponding to the pixel position of the one image data among the extracted pixel data. Selecting at least one set of pixel data arranged in the direction and calculating an index value in the predetermined direction;
- the size of at least one of the horizontal direction and the vertical direction is scaled from a plurality of pieces of color image data input continuously in time series.
- Each of the plurality of color image data is extracted with discrete pixel data, and among the extracted pixel data, pixel data in the vicinity of a pixel position corresponding to the pixel position of the one image data is extracted. Selecting at least one set of pixel data arranged in a predetermined direction and calculating an index value in the predetermined direction;
- FIG. 1 is a diagram showing a configuration of an image interpolation device according to a first embodiment of the present invention.
- FIG. 2A is a diagram showing an example of data stored in a first memory when thinning sampling is performed.
- FIG. 2B is a diagram showing an example of data stored in a second memory when thinning sampling is performed.
- FIG. 2C is a diagram showing an example of data obtained from the first and second memories.
- FIG. 2D is a diagram showing an example of data stored in a first memory when continuous pixel data is added and averaged and sampled.
- FIG. 2E is a diagram showing an example of data stored in a second memory when continuous pixel data is added and averaged and sampled.
- FIG. 3 is a diagram illustrating a detailed configuration of an index value calculation unit in FIG. 1.
- FIG. 4 is a diagram illustrating an example of pixel data captured by a data capturing unit during interpolation.
- FIG. 5 is a diagram showing an example of a pixel set used for detecting a direction having a strong correlation.
- FIG. 6 is a diagram showing an example of used pixels according to a correlation direction.
- FIG. 7 is a diagram showing a configuration of an image interpolation device according to a second embodiment of the present invention.
- FIG. 8 is a diagram showing an example of sampling in the horizontal direction.
- FIG. 9A is a diagram showing an example of data stored in a first memory when an image size is reduced to 1Z2.
- FIG. 9B is a diagram showing an example of data stored in a second memory when the size of an image is reduced to 1Z2.
- FIG. 9C is a diagram showing an example of data obtained from the first and second memories.
- FIG. 10 is a diagram illustrating an example of pixel data captured by a data capturing unit during interpolation.
- FIG. 11 is a diagram showing an example of a pixel set used for detecting a direction having a strong correlation.
- FIG. 12 is a diagram showing an example of used pixels (only G) according to a correlation direction.
- FIG. 13A is a diagram for explaining a top field interpolation method in a case where correlation is strong in the vertical direction!
- Fig.13B shows how to interpolate the bottom field when the correlation is strong in the vertical direction! It is a figure for explaining.
- FIG. 14 is a diagram showing a pixel data array that causes erroneous determination of a correlation direction in the related art.
- video data obtained by shooting with the image sensor 10 is converted into digital image data by AZD conversion 11.
- the image data output from the AZD converter 11 is transferred to the first memory 13 or the second memory 14 connected to the data control unit 12 under the control of the data control unit 12, and stored therein. .
- An index value calculation unit 15 is connected to the first memory 13 and the second memory 14.
- a direction discriminating unit 16 is connected to the index value calculating unit 15, and the direction discriminating unit 16 receives data output from the index value calculating unit 15, and operates based on the data.
- the data control unit 12, the first memory 13, the second memory 14, and the direction determination unit 16 are connected to a switching unit 17.
- the switching unit 17 operates according to the data output from the direction determination unit 16 and the control of the data control unit 12. Further, an interpolation unit 18 is connected to the switching unit 17.
- the image data output from the interpolation unit 18 is transferred to an output unit 19 such as a monitor or a memory card, where the data is displayed or stored.
- the image sensor 10 outputs analog video data corresponding to a subject image that has passed through a lens system (not shown) and a low-pass filter.
- the analog video data is converted into digital image data by the A / D converter 11.
- the image sensor 10 is a black-and-white image sensor without a color filter.
- the image data output from the AZD variable is connected under the control of the data control unit 12, and is selectively stored in a first memory 13 and a second memory 14, which are frame memories. That is, the data control unit 12 switches the connection between the first memory 13 and the second memory 14 for each frame so that the continuous image data is stored in the two memories 13 and 14 and alternates them. Transfer data to For example, in the case of interlaced scanning, image data composed of pixel data of odd lines is stored in one memory, and image data composed of pixel data of even lines is stored in the other memory.
- the index value calculation unit 15 connected to these two memories 13 and 14 receives pixel data around the position to be interpolated from each of the memories 13 and 14 and obtains an index value in a predetermined direction.
- the index value related to the predetermined direction obtained by the index value calculation unit 15 is sent to the direction discrimination unit 16.
- the direction determination unit 16 performs interpolation based on the index value. Then, it is determined in which direction the correlation is strong.
- the switching unit 17 switches the operation in accordance with the control from the data control unit 12 and the strong correlation direction output from the direction discrimination unit 16, and converts the pixel data necessary for interpolation into the first data.
- the memory 13 or the second memory 14 is also sent to the interpolator 18.
- the interpolation unit 18 obtains an interpolation value using the pixel data received from the switching unit 17, and forms interpolated image data. Then, the image data formed by the interpolation unit 18 is displayed on a display medium such as a monitor or recorded on a recording medium such as a memory card in the output unit 19.
- FIGS. 2A to 2E show an example of the data format stored in the first memory 13 and the second memory 14 when the image size is reduced to 1Z2 in the present embodiment.
- white pixels represent pixels that are read out of the input data to the data control unit 12 and stored in the memory 13 or 14, and pixels marked with “No” or “Tching” are not read out. In other words, it represents a pixel that is not saved.
- Data is data every other pixel in the horizontal direction and every other line in the vertical direction. It is made to be data.
- the data control unit 12 shifts the line that is not read for each frame, and controls the read pixel position so that it does not overlap with the continuous image data.
- the data as shown in FIG. 2C is obtained by combining the data stored in the memories 13 and 14 by the switching unit 17 under the control of the data control unit 12.
- the data stored in the first memory 13 is stored in the second memory 14 as shown in FIG. 2D, for example.
- the data is, for example, as shown in FIG. 2E.
- the data control unit 12 averages the value obtained by adding the pixel data connected by the white circles in those figures, that is, the data of two pixels that are continuous in the horizontal direction, and adds Saved on 13, 14.
- the switching unit 17 under the control of the data control unit 12, the arrangement shown in FIG. The corresponding data can be obtained.
- the interpolation unit 18 obtains an interpolation value at a position indicated by a white circle in FIG. 2C, whereby an interpolated image reduced to 1Z2 can be generated.
- the present invention described in the case where the size of the image is reduced to 1Z2 is, of course, not limited to the reduction ratio of 1Z2.
- the data control unit 12 sequentially transfers data every two pixels in the horizontal direction and every two lines in the vertical direction to these three memories for each frame.
- the lines that are not read are shifted for each image, and the read pixel positions are controlled so that they do not overlap in the continuous image data.
- one frame of image data is generated from the data thinned out for three consecutive screens.
- the index value calculation unit 15 used in the present embodiment includes a data acquisition unit 21, an absolute value difference detection unit 22, and an addition unit 23.
- the first memory 13 and the second memory 14 are connected to the data capturing unit 21, the data capturing unit 21 is connected to the absolute value difference detecting unit 22, and the absolute value difference detecting unit 22 is connected to the data capturing unit 21.
- Adder 23 is connected to adder 23 Is connected to the direction determination unit 16.
- the data acquisition unit 21 uses the first memory 13 and the second memory 14 to center the pixel position to be interpolated.
- the surrounding pixel data in a block of a predetermined size is taken in.
- the absolute value difference detection unit 22 extracts a set of pixels adjacent in a predetermined direction from the pixel data in the block fetched by the data fetch unit 21 at a plurality of locations, and extracts the pixel value of each set. Calculate the absolute value difference of.
- the sum of the absolute value differences obtained by the absolute value difference detection unit 22 in the above-described predetermined direction is calculated by the addition unit 23, and index values in each direction are output.
- data captured by the data capturing unit 21 is, for example, as shown in FIG. It will be something like Here, the data on the odd lines in the figure represents the data transferred from the first memory 13, and the data on the even lines represents the data transferred from the second memory 13. That is, in the present embodiment, pixel data in a block of 4 ⁇ 4 pixels around the data is fetched together with the data stored in the two memories 13 and 14 around the position to be interpolated. Note that white pixels in the drawing represent read pixel data.
- a set of pixel data used when the absolute value difference detection unit 22 detects an absolute value difference in a predetermined direction from the captured pixel data is, for example, as shown in FIG. Become. In this example, detection is performed in four cases of the predetermined directions: horizontal direction, vertical direction, rightward rising 45 degrees direction, rightward falling 45 degrees direction.
- the adder 23 calculates the sum of the absolute value differences of the pixel sets in each direction, and calculates the index value.
- the index value in each direction is V for the index value for the horizontal direction, V for the index value for the vertical direction, V for the index value for the upward direction, and hor ver sla for the downward direction.
- the index value is a force obtained by the average value of the absolute difference between two pixels of the pixel.
- the present invention does not need to be limited to this.
- the accumulated value of the absolute value difference or the ratio of the pixel values of two points may be used instead.
- the set of pixels for which the absolute value difference is to be calculated is not limited to the case of the set shown in Equation 1 as long as it is a set of pixels arranged in each direction. Further, the number of sets of pixels for which the absolute value difference is obtained is not limited to the case shown in the above equation (1).
- the direction discriminating unit 16 receives the index value data on the predetermined direction output from the index value calculating unit 15, and determines the direction having the highest correlation at the position to be interpolated. To determine the correlation, the ratio of each index value to the other index values is used. That is, when the condition shown in the following equation (2) is satisfied, it is determined that the correlation is strong in each direction.
- the correlation is If the directions are comparable, it is determined that there is no correlation in the specific direction.
- the direction discriminating section 16 outputs an identification signal associated with each direction.
- the direction determination is not limited to the force obtained from the ratio of the index value.
- a method of judging the direction indicating the minimum value of the index value to be the direction and the strength of the correlation can also be used.
- the data control unit 12 connected to the switching unit 17 switches the connection to the two memories 13 and 14 on a frame-by-frame basis so that continuous image data is It is stored alternately on 13, 14. Then, a control signal indicating to which memory the latest input data is recorded is transmitted to the switching unit 17. For example, in the case of the interlaced scanning, the image data having the pixel data power of the even-numbered line and the image data having the pixel data power of the odd-numbered line are sequentially stored in the memories 13 and 14 for each frame.
- ⁇ 0 '' is transmitted to the switching unit 17 as the control signal, and ⁇ 1 '' is transmitted if it is pixel data of an odd-numbered line. .
- control signal need not be limited to this as long as it is possible to distinguish which input data is the latest input data by using a numeral for the value of the control signal. is there.
- the switching unit 17 stores the data in the connected first memory 13 and second memory 14 according to the identification signal output from the direction determination unit 16 and the control signal from the data control unit 12.
- the data used for interpolation is switched among the image data that has been set, and the selected pixel data is transferred to the interpolation unit 18.
- FIG. As the pixel data to be used, which is determined according to the identification signal input from the direction determination unit 16 and the control signal from the data control unit 12, for example, FIG. As shown.
- the directional force with strong correlation is shown along the horizontal axis in the horizontal direction, in the vertical direction, in the 45 ° upward direction, in the 45 ° downward direction, and in the case of none. Is expressed.
- the vertical axis represents the case where the latest input data is even line data or odd line data.
- the interpolating unit 18 uses the pixel data input from the switching unit 17 in this way, and calculates an average obtained by applying a weighting coefficient according to the distance between the position of each pixel data and the position for obtaining the interpolation value. To obtain an interpolation value. That is, in the present embodiment, an interpolation value is obtained by the following expression (3) or (4) according to the latest input data and the direction having a strong correlation.
- a weighted average is used when obtaining an interpolation value, but the present invention is not limited to this.
- other methods such as the median value of adjacent pixels ⁇ mode value, arithmetic mean, etc. can be used.
- the image data thus interpolated is transferred from the interpolating unit 18 to the output unit 19, and is stored in a recording medium such as a memory card or a hard disk, or displayed on a display medium such as a monitor.
- a direction having a strong correlation is detected with high accuracy.
- a highly accurate interpolated image can be generated.
- the direction determining unit 16 can determine the direction of the movement of the subject or the camera between the frames. Therefore, the switching unit 17 It is also possible to switch the pixel data used for interpolation in the interval 18 according to the subject and the state of the camera.
- a color filter 30 is disposed in front of the image sensor 10.
- a subject image that has entered the image sensor 10 via a lens system and a single-pass filter (not shown) and the color filter 30 is captured by the image sensor 10.
- the color video data obtained by the image sensor 10 is converted into digital color image data by the AZD converter 11.
- the color image data output from the AZD variable is transferred to the first memory 13 or the second memory 14 connected to the data control unit 12 under the control of the data control unit 12. , Will be saved.
- An index value calculation unit 15 is connected to the first memory 13 and the second memory 14.
- a direction discriminating section 16 is connected to the index value calculating section 15, and the direction discriminating section 16 receives data output from the index value calculating section 15 and operates.
- the direction discriminating unit 16 and the data control unit 12, the first memory 13, and the second memory 14 are connected to a switching unit 17, and the switching unit 17 stores data output by the direction discriminating unit 16. , And operates according to the control of the data control unit 12. Further, an interpolation unit 18 is connected to the switching unit 17.
- the color image data output from the interpolation unit 18 is transferred to an output unit 19 such as a monitor or a memory card, where the data is displayed or stored.
- the basic operation of the image interpolation apparatus according to the present embodiment is substantially the same as that of the first embodiment. Therefore, the following mainly describes parts different from the first embodiment.
- the color filter 30 is disposed in front of the image sensor 10, and analog color video data is obtained from the image sensor 10, and the analog color video data is converted by the A / D converter 11 into a digital color image. Converted to image data.
- the sensor 10 is assumed to be a CMOS (Complementary Metal-Oxide Semiconductor), and the color filter 30 uses a primary color bayer type that also has R (red), G (green), and B (blue). There.
- the Bayer-type filter has the appearance frequency of the G pixel twice that of the other R and B pixels.
- the color image data read from the image sensor 10 via the AZD converter 11 is connected under the control of the data control unit 12, and connected to the first memory 13 and the second memory 13 which are frame memories. Stored in 14. That is, the data control unit 12 switches the connection between the first memory 13 and the second memory 14 for each frame and transfers data alternately.
- the data stored in the memories 13 and 14 is as follows. That is, since the primary color Bayer type color filter 30 is mounted on the image sensor 10, the pixel data is repeated in the horizontal and vertical directions at a two-pixel cycle.
- the image sensor 10 is a CMOS
- pixel data of the same color adjacent in the horizontal direction can be averaged and read. This is shown in FIG.
- the upper pixel group in the figure represents the input data
- the lower pixel group in the figure represents the read data.
- the same data as that obtained by adding and averaging adjacent pixels of the same color and extracting two pixels every two pixels is obtained.
- pixel lines to be read and pixel lines not to be read are repeated at a cycle of two lines.
- the data formats stored in the two memories 13, 14 are as shown in FIGS. 9A and 9B.
- the line read from the image sensor 10 is shifted by changing the position for each image data, and the read line position is changed so as not to overlap with the continuous image data.
- the interpolation unit 18 obtains an interpolation value at a position indicated by a white circle in the figure, whereby an interpolated image reduced to 1Z2 can be generated.
- the rate of enlarging / reducing the force image size described above is not limited to 1Z2.
- shrink to 1Z3 The same is true for a small case.
- a first memory, a second memory, and a third memory are configured, and the data control unit 12 adds data of three pixels of the same color that are adjacent in the horizontal direction and averages the data in the vertical direction. Means that two lines of data are transferred sequentially to three memories in a six-line cycle.
- the thinning position is shifted for each image, and is controlled by the data control unit 12 so that the read pixel positions do not overlap in continuous image data.
- image data for one frame is generated from the thinned data for three consecutive screens stored in the memory.
- the configuration of the index value calculation unit 15 in the present embodiment is the configuration shown in FIG. 3, similar to the configuration shown in the first embodiment. That is, pixel data in a block surrounding a position to be interpolated is taken in by the data taking section 21 for interpolation of an arbitrary point.
- the absolute value difference detection unit 22 extracts a set of pixels arranged in a predetermined direction from the pixel data captured by the data capture unit 21 at a plurality of locations, and extracts a pixel value of each pixel set. Calculate the absolute value difference of. Then, the sum of the obtained absolute value differences in the above-described predetermined direction is calculated in the adding unit 23, and the index value in each direction is output.
- the data captured from the memories 13 and 14 is, for example, as shown in FIG.
- the interpolation value at the position indicated by the white circle in the figure is obtained.
- pixel power indicated by adding “no” and “tching” represents read-out pixel data.
- the line data of the sixth row (G, B, G, B) is transferred from the first memory 13.
- the third line (R, G, R, G) and the fourth line (G, B, G) Represents the data obtained.
- the third line (R, G, R, G) and the fourth line (G, B, G) Represents the data obtained.
- Data represents data transferred from the second memory 14.
- pixel data in a block of surrounding 8 ⁇ 8 pixels including a position where data from the two memories 13 and 14 are interpolated together is taken in.
- the captured pixel data force is used to detect an index value in a predetermined direction.
- the pixel data set is as shown in FIG. 11, for example.
- the direction is detected in four patterns of a vertical direction, a horizontal direction, a right-up 45-degree direction, and a right-down 45-degree direction, and pixel data of the same color located in each direction is used.
- the pixels to be used are switched according to the obtained direction and the direction.
- the adder 23 calculates the sum of the absolute value differences of the pixel values in each direction, and calculates the index value.
- the index values to be obtained are V,
- the direction discriminating unit 16 receives the index value data relating to the predetermined direction output from the index value calculating unit 15, and outputs the direction with the highest correlation at the position to be interpolated. In determining the correlation, the ratio of each index value to the other index values is used, as in the first embodiment. It is determined whether the correlation is strong or not in the vertical direction, the horizontal direction, the 45-degree upward direction to the right, and the 45-degree downward direction to the right, and the identification signal associated with each direction is determined. Is output.
- the index value is obtained by the sum of the absolute value differences of the pixel values.
- the index value need not be limited to this, and a method using a ratio of two pixel values may be used.
- the set of pixels arranged in a predetermined direction is not limited to the case of the above-described set. Further, in the determination of the correlation, the determination is made based on the ratio of the respective index values, but a method of obtaining the minimum value of the index values may be used.
- the data control unit 12 connected to the switching unit 17 switches the connection to the two memories 13 and 14 for each image, and stores successive image data alternately.
- the control signal is transmitted to the switching unit 17 as to which memory the data is recorded in.
- the image data is read at the first two lines, and a frame that repeats a cycle of not reading the next two lines (hereinafter, referred to as “ When the next two lines are read without reading the first two lines, a frame that repeats a cycle (hereinafter referred to as a bottom field) is formed.
- the data control unit 12 outputs a control signal of “0” if the image data of the top field is the latest input data, and outputs a control signal of “0” if the image data of the bottom field is the latest input data.
- a control signal of “1” is transmitted to the switching unit 17.
- this control signal does not need to be limited to such a number as long as it can distinguish which memory data is the latest input data. Signals such as alphabets and letters may be used.
- the switching unit 17 according to the identification signal output from the direction determination unit 16 and the control signal output from the data control unit 12, the first memory 13 and the second memory 14 are connected. Switches the selection of data to be used for interpolation among the stored image data.
- the pixel data used in each case in the case of G (green) is, for example, as shown in FIG. As shown in 12.
- the directional forces with strong correlation are divided into horizontal direction, vertical direction, upward 45 degree direction, downward 45 degree direction, and none. It describes the case of field data.
- the interpolation unit 18 uses the pixel data located in the direction having a strong correlation input from the switching unit 17 in this way, according to the distance between the position of each pixel data and the position at which the interpolation value is to be obtained. An interpolated value is obtained by averaging weighted coefficients.
- FIGS. 13A and 13B show how to obtain the G (green) interpolation value when the correlation is strong in the vertical direction.
- the interpolation value at the position indicated by the black circle is obtained from the pixel data of only two points arranged in the vertical direction having a strong correlation, and the value is compared with another value.
- the interpolation value at the position indicated by the white circle in FIG. 10 is obtained from the pixel data of the point.
- the interpolation value of G at the position of the white circle in FIG. 10 is obtained by the following Expression (6) or Expression 612 or Expression (7) depending on the difference between the input data and the correlation direction.
- the position of the pixel data used for interpolation and the interpolation is obtained by a weighted average according to the distance from the position to be obtained.
- an interpolation value is obtained using a color correlation with another color.
- a direction having a strong correlation can be detected with high accuracy, and a highly accurate interpolated image can be generated.
- the direction of movement of the subject or the camera is determined, and pixel data used for interpolation is switched according to the state of the subject or the camera. It is also possible.
- the weighted average is used when obtaining the interpolation value.
- the present invention is not limited to this.
- Other methods, such as arithmetic averaging, can also be used.
- the color filter 30 does not need to be limited to the primary color type.
- Complementary color filters with C (cyan), M (magenta), ⁇ (yellow), and G (green) chromaticity can also be used.
- image sensor 10 does not need to be limited to CMOS.
- CCD Charge-Coupled Device
- Coupled device can also be used.
- the data obtained by sampling has a form in which the horizontal direction and the vertical direction are switched.
- interpolation is not limited to interpolation of image data from the image sensor 10, and it is possible to similarly interpolate broadcast image data of a television or the like.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Spectroscopy & Molecular Physics (AREA)
- Color Television Image Signal Generators (AREA)
- Studio Devices (AREA)
- Image Processing (AREA)
- Television Systems (AREA)
Abstract
Description
Claims
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
EP04819970A EP1703722A1 (en) | 2003-12-08 | 2004-12-08 | Image interpolation apparatus and image interpolation method |
US11/447,315 US20060222269A1 (en) | 2003-12-08 | 2006-06-05 | Image interpolation apparatus and image interpolation method |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2003-408885 | 2003-12-08 | ||
JP2003408885A JP3960965B2 (ja) | 2003-12-08 | 2003-12-08 | 画像補間装置及び画像補間方法 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US11/447,315 Continuation US20060222269A1 (en) | 2003-12-08 | 2006-06-05 | Image interpolation apparatus and image interpolation method |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2005055590A1 true WO2005055590A1 (ja) | 2005-06-16 |
Family
ID=34650405
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/JP2004/018296 WO2005055590A1 (ja) | 2003-12-08 | 2004-12-08 | 画像補間装置及び画像補間方法 |
Country Status (4)
Country | Link |
---|---|
US (1) | US20060222269A1 (ja) |
EP (1) | EP1703722A1 (ja) |
JP (1) | JP3960965B2 (ja) |
WO (1) | WO2005055590A1 (ja) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2011078092A (ja) * | 2010-09-08 | 2011-04-14 | Toshiba Corp | 色差信号フォーマット変換装置及び方法 |
US8068173B2 (en) | 2009-09-30 | 2011-11-29 | Kabushiki Kaisha Toshiba | Color difference signal format conversion device and method |
CN105847996A (zh) * | 2016-05-25 | 2016-08-10 | 腾讯科技(深圳)有限公司 | 一种视频播放方法及装置 |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5151075B2 (ja) * | 2005-06-21 | 2013-02-27 | ソニー株式会社 | 画像処理装置及び画像処理方法、撮像装置、並びにコンピュータ・プログラム |
JP5522890B2 (ja) * | 2007-07-31 | 2014-06-18 | キヤノン株式会社 | 画像処理装置及び方法 |
US20090092338A1 (en) * | 2007-10-05 | 2009-04-09 | Jeffrey Matthew Achong | Method And Apparatus For Determining The Direction of Color Dependency Interpolating In Order To Generate Missing Colors In A Color Filter Array |
US8135237B2 (en) * | 2008-02-25 | 2012-03-13 | Aptina Imaging Corporation | Apparatuses and methods for noise reduction |
US8130299B2 (en) * | 2009-03-10 | 2012-03-06 | Pixart Imaging Inc. | Method of performing interlaced preview for a CMOS sensor |
JP5625298B2 (ja) * | 2009-09-28 | 2014-11-19 | ソニー株式会社 | 撮像装置 |
GB2484071B (en) * | 2010-09-23 | 2013-06-12 | Imagination Tech Ltd | De-interlacing of video data |
JP5474258B2 (ja) | 2011-03-11 | 2014-04-16 | 富士フイルム株式会社 | 撮像装置及び撮像プログラム |
JP5607266B2 (ja) | 2011-12-27 | 2014-10-15 | 富士フイルム株式会社 | 撮像装置、撮像装置の制御方法、及び制御プログラム |
EP2800375A4 (en) | 2011-12-27 | 2015-10-21 | Fujifilm Corp | IMAGING DEVICE, CONTROL METHOD FOR IMAGING DEVICE, AND CONTROL PROGRAM |
RU2547140C1 (ru) | 2011-12-27 | 2015-04-10 | Фуджифилм Корпорэйшн | Устройство формирования изображений и способ управления устройством формирования изображений |
WO2013100097A1 (ja) | 2011-12-27 | 2013-07-04 | 富士フイルム株式会社 | 撮像装置、撮像装置の制御方法、及び制御プログラム |
WO2013100093A1 (ja) | 2011-12-27 | 2013-07-04 | 富士フイルム株式会社 | 撮像装置、撮像装置の制御方法、及び制御プログラム |
CN104011575B (zh) | 2011-12-27 | 2016-08-31 | 富士胶片株式会社 | 摄像装置及摄像装置的控制方法 |
JP5738904B2 (ja) * | 2013-01-28 | 2015-06-24 | オリンパス株式会社 | 画像処理装置、撮像装置、画像処理方法及びプログラム |
Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11205808A (ja) * | 1998-01-09 | 1999-07-30 | Fuji Photo Film Co Ltd | 信号処理装置およびその処理方法 |
JP2000125169A (ja) * | 1998-10-14 | 2000-04-28 | Mitsubishi Electric Corp | 撮像装置 |
JP2000244932A (ja) * | 1999-02-19 | 2000-09-08 | Mitsubishi Electric Corp | 撮像装置及び撮像方法 |
JP2001285885A (ja) * | 2000-03-31 | 2001-10-12 | Fuji Photo Film Co Ltd | 固体撮像装置および信号処理方法 |
JP2004015772A (ja) * | 2002-06-12 | 2004-01-15 | Olympus Corp | 撮像装置 |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS60171878A (ja) * | 1984-02-15 | 1985-09-05 | Victor Co Of Japan Ltd | 映像信号処理装置 |
US4630307A (en) * | 1984-09-10 | 1986-12-16 | Eastman Kodak Company | Signal processing method and apparatus for sampled image signals |
GB2301726B (en) * | 1992-11-10 | 1997-05-14 | Sony Uk Ltd | Motion compensated video signal processing |
US5373322A (en) * | 1993-06-30 | 1994-12-13 | Eastman Kodak Company | Apparatus and method for adaptively interpolating a full color image utilizing chrominance gradients |
US5991463A (en) * | 1995-11-08 | 1999-11-23 | Genesis Microchip Inc. | Source data interpolation method and apparatus |
JPH11122626A (ja) * | 1997-10-17 | 1999-04-30 | Nikon Corp | 画像処理方法及び装置並びに画像処理プログラムを記録した記録媒体 |
JP4317619B2 (ja) * | 1999-07-22 | 2009-08-19 | メディア・テック・ユーエスエイ・インコーポレーテッド | 画像処理装置 |
US6414690B1 (en) * | 1999-12-08 | 2002-07-02 | Xerox Corporation | Gamut mapping using local area information |
US6931062B2 (en) * | 2001-04-11 | 2005-08-16 | Koninklijke Philips Electronics N.V. | Decoding system and method for proper interpolation for motion compensation |
JP4054184B2 (ja) * | 2001-10-03 | 2008-02-27 | オリンパス株式会社 | 欠陥画素補正装置 |
JP2007504742A (ja) * | 2003-09-04 | 2007-03-01 | コニンクリユケ フィリップス エレクトロニクス エヌ.ブイ. | ビデオ信号の頑強なデインタレース |
US7263218B2 (en) * | 2003-12-22 | 2007-08-28 | Canon Kabushiki Kaisha | Dynamic generation of color look-up tables |
US7242412B2 (en) * | 2005-02-09 | 2007-07-10 | Seiko Epson Corporation | Method and apparatus for expanding a source pixel in a digital image |
KR100750128B1 (ko) * | 2005-09-06 | 2007-08-21 | 삼성전자주식회사 | 영상의 인트라 예측 부호화, 복호화 방법 및 장치 |
US7697046B2 (en) * | 2005-12-07 | 2010-04-13 | Hoya Corporation | Image signal processing device and method of image signal processing including pixel array pattern determination |
-
2003
- 2003-12-08 JP JP2003408885A patent/JP3960965B2/ja not_active Expired - Fee Related
-
2004
- 2004-12-08 WO PCT/JP2004/018296 patent/WO2005055590A1/ja active Application Filing
- 2004-12-08 EP EP04819970A patent/EP1703722A1/en not_active Withdrawn
-
2006
- 2006-06-05 US US11/447,315 patent/US20060222269A1/en not_active Abandoned
Patent Citations (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH11205808A (ja) * | 1998-01-09 | 1999-07-30 | Fuji Photo Film Co Ltd | 信号処理装置およびその処理方法 |
JP2000125169A (ja) * | 1998-10-14 | 2000-04-28 | Mitsubishi Electric Corp | 撮像装置 |
JP2000244932A (ja) * | 1999-02-19 | 2000-09-08 | Mitsubishi Electric Corp | 撮像装置及び撮像方法 |
JP2001285885A (ja) * | 2000-03-31 | 2001-10-12 | Fuji Photo Film Co Ltd | 固体撮像装置および信号処理方法 |
JP2004015772A (ja) * | 2002-06-12 | 2004-01-15 | Olympus Corp | 撮像装置 |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US8068173B2 (en) | 2009-09-30 | 2011-11-29 | Kabushiki Kaisha Toshiba | Color difference signal format conversion device and method |
JP2011078092A (ja) * | 2010-09-08 | 2011-04-14 | Toshiba Corp | 色差信号フォーマット変換装置及び方法 |
CN105847996A (zh) * | 2016-05-25 | 2016-08-10 | 腾讯科技(深圳)有限公司 | 一种视频播放方法及装置 |
Also Published As
Publication number | Publication date |
---|---|
EP1703722A1 (en) | 2006-09-20 |
JP3960965B2 (ja) | 2007-08-15 |
JP2005175590A (ja) | 2005-06-30 |
US20060222269A1 (en) | 2006-10-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20060222269A1 (en) | Image interpolation apparatus and image interpolation method | |
KR100782812B1 (ko) | 에지 적응적 컬러 보간 방법 및 장치 | |
USRE47062E1 (en) | Image processing apparatus and method for generating a restoration image | |
JP2931520B2 (ja) | 単板式カラービデオカメラの色分離回路 | |
CN101924947B (zh) | 图像处理装置及图像处理方法、摄像装置 | |
US9288454B2 (en) | Color imaging apparatus having color imaging element | |
US7973831B2 (en) | Solid-state image sensor and imaging apparatus using the same | |
CN1812592B (zh) | 用于处理滤色器阵列的图像数据的方法和设备 | |
EP1977613B1 (en) | Interpolation of panchromatic and color pixels | |
JP3962876B2 (ja) | ビデオ信号用補間フィルタ | |
JP4946581B2 (ja) | 画像処理装置 | |
US20060284862A1 (en) | Method and apparatus for processing bayer-pattern digital color video signal | |
JP2816095B2 (ja) | ビデオカメラの信号処理回路 | |
KR100765751B1 (ko) | 비월 주사 방식의 cmyg 컬러 포맷의 컬러 보간 방법 및그 장치 | |
WO2009145201A1 (ja) | 画像処理装置、画像処理方法及び撮像装置 | |
EP2359604A2 (en) | Modifying color and panchromatic channel cfa image | |
JP2002084547A (ja) | 画像データサイズ変換処理装置、電子スチルカメラ、および画像データサイズ変換処理用記録媒体 | |
JP2004015772A (ja) | 撮像装置 | |
US7489822B2 (en) | Image processing apparatus and method for detecting a direction of an edge in the vicinity of a pixel of interest and generating all color signals for each pixel by interpolation using color signals of a pixel of interest and its neighbor pixels, and a recording medium having a program recorded thereon for causing the apparatus to perform the method | |
JP4934991B2 (ja) | 撮像信号処理装置及び方法並びに撮像装置 | |
JP4043197B2 (ja) | 単板式カラーカメラの色分離回路 | |
JP4635769B2 (ja) | 画像処理装置、画像処理方法および撮像装置 | |
US8803996B2 (en) | Image pickup apparatus for preventing unnatural motion of a motion image | |
JP4280462B2 (ja) | 信号処理装置及び方法 | |
JP4385748B2 (ja) | 画像信号処理装置及び画像信号処理方法 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AK | Designated states |
Kind code of ref document: A1 Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BW BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NA NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW |
|
AL | Designated countries for regional patents |
Kind code of ref document: A1 Designated state(s): BW GH GM KE LS MW MZ NA SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LT LU MC NL PL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application | ||
WWE | Wipo information: entry into national phase |
Ref document number: 11447315 Country of ref document: US |
|
WWE | Wipo information: entry into national phase |
Ref document number: 2004819970 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 2004819970 Country of ref document: EP |
|
WWP | Wipo information: published in national office |
Ref document number: 11447315 Country of ref document: US |