JP4239063B2 - Image processing apparatus and method, recording medium, and program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing apparatus and method, a recording medium, and a program, and in particular, when converting an interlaced scan image to a progressive scan image, accurately identifies a still image region and a moving image region, and correct interpolation pixels. The present invention relates to an image processing apparatus and method, a recording medium, and a program that make it possible to stand still.
[0002]
[Prior art]
A technique for converting an interlaced scan type image into a progressive scan type image (IP conversion) is becoming popular.
[0003]
The interlaced scanning method is a scanning method called interlaced scanning, and is a method of scanning every other scanning line of an actual display image (interlaced scanning is performed every other scanning line). One image scanned by the interlace scan method is called a field, and when displaying continuous images, adjacent fields have pixels that are shifted by one scanning line, and the field is They are displayed alternately in succession.
[0004]
The progressive scan method is a method of scanning all the scanning lines of the display image as compared to the interlace scan method.
[0005]
Therefore, the conversion process from an interlaced scan image to a progressive scan image is performed by interpolating the pixels using the pixels that originally exist on the line where no pixels exist in the interlaced scan image. And generating pixels on all the scanning lines.
[0006]
Conventionally, as an interpolation method, one scanning line is delayed and displayed so that each scanning line is scanned twice, which is called double writing, or linear interpolation is performed using vertical pixels. The method of doing was used.
[0007]
In order to increase the vertical resolution, a method of using information of a plurality of fields has been proposed. In this method, in the interlace scan method, the previous field of consecutive fields is shifted by one scanning line as described above, so that the current field is displayed on the previous field. A pixel exists at the position to be interpolated above. Therefore, this pixel is used in an interpolation process using a plurality of fields. That is, when the pixel to be interpolated is a still pixel, the pixel in the previous field is used as it is for interpolation, and in the case of a moving pixel, the pixel by linear interpolation using the upper and lower pixels of the current field. Is to be interpolated.
[0008]
Conventionally, there is a technique that can reduce errors that occur when converting an image scanning method (see, for example, Patent Document 1).
[0009]
Furthermore, there is a technique that prevents oblique lines from appearing unnatural when converting the scanning method (see, for example, Patent Document 2).
[0010]
[Patent Document 1]
JP 2001-352525 A
[Patent Document 2]
JP 2001-218169 A
[0011]
[Problems to be solved by the invention]
However, in the above method, for example, when writing twice is used, there is a problem that an image that becomes an oblique edge is displayed in a large step shape by IP conversion.
[0012]
In addition, when linear interpolation is used, the display image is improved as compared with writing twice, but it may be displayed with a slight blur. Has a problem of being displayed in a staircase pattern.
[0013]
In order to prevent diagonal edges from being displayed stepwise, a method has been proposed in which IP conversion is performed while detecting the edge direction. This is described in Japanese Patent Publication No. 3-42832, and the absolute value of the difference between the upper and lower, upper right lower left, and lower right upper left three sets of pixels is examined around the target pixel to be interpolated. In this method, interpolation is performed with a combination of pixels having the smallest value, and if all are larger than a predetermined value, the above pixel value is copied and interpolated. As a result, even when an oblique edge exists, the direction is smoothly interpolated.
[0014]
However, in the method of converting while detecting the direction of the edge, the display of the edge in the oblique direction is improved, but in the still image area, the vertical resolution is disadvantageous compared to the method using a plurality of fields. There was a problem. In addition, when the method of converting while detecting the direction of the edge is applied to an actual image, the correlation in the oblique direction may be slightly higher than that in the vertical direction depending on, for example, noise or lighting conditions. There is a problem that even if the pattern is continuous up and down, an error that it is not displayed correctly is likely to occur.
[0015]
In addition, the method using multiple fields improves the problem that diagonal edges are displayed in a staircase pattern, but the vertical resolution is somewhat sacrificed because multiple fields are used in the still image area. was there.
[0016]
Also, in general video signals, etc., only a few high-frequency components are included, and edges appear only faintly, so it is difficult to accurately detect the edge direction locally, and a specific direction Even if the edge of the image is to be emphasized, a plurality of edge direction candidates are detected and cannot be selected, and as a result, an image can be generated only with the same image quality as the conventional linear interpolation. there were.
[0017]
Furthermore, there is a problem in that it may not be possible to accurately identify whether the pixel position to be interpolated is a still image region or a moving image region, and pixels may not be correctly interpolated.
[0018]
The present invention has been made in view of such a situation, and at the time of conversion from an interlaced scan image to a progressive scan image, the vertical resolution of the still image portion is improved and the edges in the moving image area are gently interpolated. In addition to making it possible to suppress errors in conversion, it accurately identifies whether the pixel to be interpolated is a moving image region or a still image region, and generates a correct interpolation pixel.
[0019]
[Means for Solving the Problems]
The image processing apparatus according to the present invention includes an intra-field interpolation unit that interpolates a pixel of a progressive image from a pixel in a field of an interlace image to be converted, and a field of an interlace image to be converted. Variation measurement that measures variation between fields from the difference between the average pixel value of two pixels adjacent to the target pixel in the vertical direction or horizontal direction of the target pixel of the target pixel and a pixel in a field different from the target pixel And inter-field interpolation means for interpolating progressive image pixels from the interlaced image field pixels to be converted and the variation measured by the variation measuring means, and the variation measuring means. Field size different from the target pixel and the magnitude relationship with the predetermined threshold value of the target pixel variation, and the target pixel variation Determining means for determining whether a pixel of the progressive image to be interpolated is a moving pixel or a still pixel based on the positive / negative of the multiplication result with the variation of the pixel at the same position; If the pixel of the progressive image to be interpolated is a moving pixel based on the determination result, the pixel interpolated by the intra-field interpolation means is selected, and the pixel of the progressive image to be interpolated is a still pixel. A selection unit that selects pixels interpolated by the inter-field interpolation unit, a band limiting unit that limits the band of the interlaced image field, and a field of the interlaced image that is band-limited by the band limiting unit. Thinning means for thinning out the number of pixels by half in the horizontal direction, and the variation measuring means reduces the number of pixels by 1 in the horizontal direction by the thinning means. The average pixel value between adjacent pixels above and below the pixel of the field thinned to 2 and the field where the number of pixels is thinned by half in the horizontal direction immediately before or after the field The difference between the pixel values of the pixels located in the center between the vertically adjacent pixels is measured as the variation between the fields of the pixels located in the middle between the vertically adjacent pixels, and the inter-field interpolation means tries to convert The pixel to be If it is the same as the horizontal position of the pixel that was not thinned by the thinning means, Of the pixels above and below Average pixel value And the pixel value to be converted by interpolating the pixel value by the difference of 1/2 of the fluctuation of the pixel to be converted, When the same as the horizontal position of the pixels thinned out by the thinning means, Of the pixels above and below Average pixel value And the fluctuation of the left and right pixels of the pixel to be converted average The pixel value is interpolated based on the difference between and.
[0020]
Edge enhancement means for enhancing the edge of the interlaced image to be converted may be further provided.
[0021]
The edge enhancement means can perform horizontal and vertical one-dimensional filter processing for each pixel of an interlaced image to enhance the edge of the image.
[0022]
From the pixel values in the vicinity of each pixel, it is possible to further provide a center pixel energy calculation means for calculating the center pixel energy in the vertical direction and the horizontal direction in the vicinity of the pixel. It is possible to perform filter processing on pixels in which the vertical and horizontal central pixel energies calculated by the central pixel energy calculating unit exceed a predetermined value.
[0026]
In the field of the interlaced image to be converted, there is further provided a correlation detecting means for detecting a correlation between pixels on the horizontal lines above and below the pixel to be interpolated and in a position contrasting with the pixel. The intra-field interpolation means can interpolate pixels from pixels having the strongest correlation detected by the correlation detection means.
[0027]
The correlation detecting means may detect an absolute value of a difference between pixel values of pixels at contrasting positions as a correlation around the pixel to be interpolated.
[0028]
It is possible to further provide a neighborhood energy calculating means for calculating the neighborhood energy of the pixel to be interpolated from the pixel values in the vicinity of the pixel to be interpolated, and try to perform the interpolation calculated by the side energy calculating means. When the value of the neighborhood energy of the pixel is larger than a predetermined threshold value, the intra-field interpolation means can be made to perform interpolation based on the pixel having the strongest correlation detected by the correlation detection means.
[0029]
The neighborhood energy is the sum of absolute values of differences between pixel values of pixels facing each other perpendicularly to the line among a predetermined number of pixels on a line extending horizontally above and below the pixel to be interpolated. The sum of the absolute values of the pixel value differences of the pixels existing at point-symmetric positions with the pixel to be interpolated as the center can be obtained.
[0030]
The image processing method of the present invention includes an intra-field interpolation step for interpolating a pixel of a progressive image from a pixel in a field of an interlace image to be converted, and a field of an interlace image to be converted. Variation measurement that measures variation between fields from the difference between the average pixel value of two pixels adjacent to the target pixel in the vertical direction or horizontal direction of the target pixel of the target pixel and a pixel in a field different from the target pixel Step, inter-field interpolation step for interpolating progressive image pixels from interlaced image field pixels to be converted, and variation measurement step processing, and variation measurement step processing The magnitude relationship with the predetermined threshold value of the variation in the pixel of interest measured by the Based on the positive / negative of the multiplication result of the fluctuation of the pixel in the same position and in the same field as the target pixel, it is determined whether the pixel of the progressive image to be interpolated is a moving pixel or a still pixel If the pixel of the progressive image to be interpolated is a moving pixel based on the determination step and the determination result in the determination step processing, the pixel interpolated by the intra-field interpolation step processing is selected and interpolated. When a pixel of a progressive image is a still pixel, a selection step for selecting a pixel interpolated by the inter-field interpolation step processing, a bandwidth limitation step for bandwidth limiting a field of an interlaced image, and a bandwidth limitation step The number of pixels in the field of the interlaced image, band-limited by the above process, is halved in the horizontal direction. The variation measurement step processing includes an average pixel value between adjacent pixels above and below the field pixels in which the number of pixels is thinned by a half in the horizontal direction by the thinning step processing, A difference between pixel values of pixels located at the center between vertically adjacent pixels on a field in which the number of pixels is thinned by half in the horizontal direction by the thinning step immediately before or after the field, Measured as the variation between fields of pixels located in the center between adjacent pixels above and below, and the inter-field interpolation step process If it is the same as the horizontal position of the pixel that was not thinned out by the thinning step, Of the pixels above and below Average pixel value And the pixel value to be converted by interpolating the pixel value by the difference of 1/2 of the fluctuation of the pixel to be converted, If it is the same as the horizontal position of the pixel thinned out by the thinning step, Of the pixels above and below Average pixel value And the fluctuation of the left and right pixels of the pixel to be converted average The pixel value is interpolated based on the difference between and.
[0031]
The recording medium program of the present invention includes an intra-field interpolation step for interpolating a progressive image pixel from a pixel in an interlace image field to be converted, and an interlace image to be converted. A variation that measures the variation between fields based on the difference between the pixel of interest in the field and the average pixel value of two pixels adjacent to the pixel of interest in the vertical or horizontal direction among the pixels of the field different from that of the field An inter-field interpolation step for interpolating a progressive image pixel from a measurement step, a variation measured by the processing of the variation measurement step, and a field pixel of the interlaced image to be converted; and a variation measurement step Magnitude relationship with the predetermined threshold of the variation of the pixel of interest measured by the processing, and the pixel of interest Whether the pixel of the progressive image to be interpolated is a moving pixel or a static pixel based on the result of the multiplication of the variation and the variation of the pixel in the same position and in the same position as the pixel of interest When the pixel of the progressive image to be interpolated is a moving pixel based on the determination result in the determination step and the determination step processing, the pixel interpolated by the intra-field interpolation step processing is selected, When the pixel of the progressive image to be interpolated is a still pixel, a selection step for selecting a pixel interpolated by the inter-field interpolation step processing, a band limitation step for band-limiting the field of the interlaced image, The number of pixels in the field of the interlaced image that has been band-limited by the band-limiting step processing in the horizontal direction The variation measurement step processing includes an average pixel value between adjacent pixels above and below the pixels in the field in which the number of pixels is thinned by half in the horizontal direction by the thinning step processing. And the pixel value of the pixel located at the center between the adjacent pixels on the top and bottom of the field in which the number of pixels is thinned by half in the horizontal direction by the thinning step immediately before or after the field. The difference is measured as the variation between the fields of the pixels located in the center between the adjacent pixels above and below, and the process of the inter-field interpolation step determines that the pixel to be converted is If it is the same as the horizontal position of the pixel that was not thinned out by the thinning step, Of the pixels above and below Average pixel value And the pixel value to be converted by interpolating the pixel value by the difference of 1/2 of the fluctuation of the pixel to be converted, If it is the same as the horizontal position of the pixel thinned out by the thinning step, Of the pixels above and below Average pixel value And the fluctuation of the left and right pixels of the pixel to be converted average The pixel value is interpolated based on the difference between and.
[0032]
The program of the present invention includes an intra-field interpolation step for interpolating a pixel of a progressive image from a pixel in a field of an interlace image to be converted, and attention of the field of the interlace image to be converted. A variation measuring step for measuring a variation between fields based on a difference between a pixel and an average pixel value of two pixels adjacent to the target pixel in a vertical direction or a horizontal direction among pixels of a field different from the pixel; Measured by inter-field interpolation step to interpolate progressive image pixels from variation measured in variation measurement step processing and interlaced image field pixels to be converted, and variation measurement step processing The magnitude relationship with the predetermined threshold value of the variation of the target pixel and the variation of the target pixel, Based on the positive / negative of the result of multiplication with the variation of the pixel at the same position in a field different from the eye pixel, it is determined whether the pixel of the progressive image to be interpolated is a moving pixel or a still pixel If the pixel of the progressive image to be interpolated is a moving pixel based on the determination step and the determination result in the determination step processing, the pixel interpolated by the intra-field interpolation step processing is selected and interpolated. When a pixel of a progressive image is a still pixel, a selection step for selecting a pixel interpolated by the inter-field interpolation step processing, a bandwidth limitation step for bandwidth limiting a field of an interlaced image, and a bandwidth limitation step The number of pixels in the field of the interlaced image, which is band-limited by this processing, is halved in the horizontal direction. A process including a decimation step is performed, and the process of the variation measurement step is performed by calculating an average pixel between adjacent pixels above and below the pixels of the field in which the number of pixels is decimation by half in the horizontal direction by the decimation step process The pixel value of the pixel located at the center between the adjacent pixels on the top and bottom on the field and the field where the number of pixels is thinned by half in the horizontal direction by the thinning step processing immediately before or after the field Is measured as a variation between fields of pixels located in the center between vertically adjacent pixels, and the inter-field interpolation step processing is performed when the pixel to be converted is If it is the same as the horizontal position of the pixel that was not thinned out by the thinning step, Of the pixels above and below Average pixel value And the pixel value to be converted by interpolating the pixel value by the difference of 1/2 of the fluctuation of the pixel to be converted, If it is the same as the horizontal position of the pixel thinned out by the thinning step, Of the pixels above and below Average pixel value And the fluctuation of the left and right pixels of the pixel to be converted average The pixel value is interpolated based on the difference between and.
[0033]
In the image processing apparatus, method, and program of the present invention, the pixels of the progressive image are interpolated from the pixels in the field of the interlaced image to be converted, and the interlaced image to be converted is converted. From the difference between the pixel of interest in the field and the average pixel value of two pixels adjacent to the pixel of interest in the vertical direction or horizontal direction among the pixels of the field different from the pixel of interest, the variation between the fields is measured, The magnitude relationship between the measured variation and the predetermined threshold value of the variation of the pixel of interest measured by the variation measuring means by interpolating the pixels of the progressive method image from the pixels of the field of the interlaced image to be converted , And the multiplication result of the variation of the pixel of interest and the variation of the pixel at the same position in the same field as the pixel of interest. Based on the negative, it is determined whether the pixel of the progressive image to be interpolated is a moving pixel or a still pixel, and the pixel of the progressive image to be interpolated is a moving pixel based on the determination result If the interpolated progressive image pixel is a static pixel, the interpolated pixel is selected and the interlaced image field is band-limited. The number of pixels of the field of the interlaced image whose band is limited is thinned by half in the horizontal direction, and pixels adjacent to the upper and lower sides of the field pixels in which the number of pixels is thinned by 1/2 in the horizontal direction. The average pixel value between and adjacent to the top and bottom on the field where the number of pixels is thinned by half in the horizontal direction immediately before or after the field Difference in pixel values of pixels located in the center of the Motokan is measured as the change between fields of pixels located at the center between the pixels adjacent in the vertical direction, the pixel to be converted, If it is the same as the horizontal position of the pixel that was not thinned, Of the pixels above and below Average pixel value And the pixel value is interpolated by the difference between ½ times the fluctuation of the pixel to be converted and the pixel to be converted is If it is the same as the horizontal position of the thinned pixel, Of the pixels above and below Average pixel value And the fluctuation of the left and right pixels of the pixel to be converted average The pixel value is interpolated by the difference between.
[0034]
DETAILED DESCRIPTION OF THE INVENTION
FIG. 1 is a diagram showing a configuration of an embodiment of an image processing apparatus according to the present invention. The field memories 11, 14, and 21 temporarily store information for one field of an input interlaced scan image (hereinafter referred to as an interlaced image) and output the information to a subsequent apparatus. That is, the field memories 11, 14, and 21 store the image information for one field input at the timing of the predetermined time t, and store the field data of the time t stored at the timing of the next time t + 1. Are output to the image refresher 12, the intra-field interpolation unit 15, the inter-field interpolation unit 16, and the fluctuation measurement unit 22.
[0035]
The image refresher 12 performs processing of the one-dimensional vertical filter and the one-dimensional horizontal filter on the field data input from the field memory 11 while using the buffer 13 as appropriate, and performs vertical and horizontal edge processing. After highlighting the portion, the data is output to the field memory 14.
[0036]
The intra-field interpolating unit 15 interpolates and generates pixels in the moving image area of the progressive image from the field data stored in the field memory 14 subjected to the edge enhancement process, and outputs the progressive image to the interpolation memory 17-2 as a progressive image. Let Details of the intra-field interpolation processing will be described later.
[0037]
The inter-field interpolation unit 16 interpolates and generates pixels of the still image area of the progressive image from the field data stored in the field memory 14 and the variation data input from the variation measurement unit 22, and the generated progressive image is generated. The data is output and stored in the interpolation memory 17-1. Details of the inter-field interpolation processing will be described later.
[0038]
An LPF (Low Pass Filter) 19 preliminarily band-limits the field data stored in the field memory 14 and outputs it to the down sampler 20. The LPF 19 reduces the modulation of the signal due to noise included in the field data, and erroneously determines that the still image region is erroneously determined as a moving image region due to the influence of noise in the still / motion region determining process described later. Suppress. Further, the band limitation can suppress the aliasing phenomenon that is likely to occur in the downsampling process by the downstream downsampler 20.
[0039]
The downsampler 20 downsamples the number of pixels in the horizontal direction to ½ in order to reduce the capacity of the band-limited field data input from the LPF 19 (the field data is thinned out every other column) and fluctuates. The data is output to the measurement unit 22 and the field memory 21. In the following description, the down-sampler 20 thins out pixels having horizontal coordinates x of even lines as field effective pixels and pixels having horizontal coordinates x of odd numbers as invalid pixels. However, as a matter of course, the odd / even relationship of the horizontal coordinate x that is the pixel position to be thinned out may be reversed.
[0040]
The variation measuring unit 22 subtracts the pixel value of the target pixel from the average of the pixel values positioned above and below the target pixel in the next timing field of the field where the target pixel exists and the previous timing field. The value is obtained as variation data, and is output to the variation data memory 23 and the still / moving image region determination unit 24, and the variation information is output to the inter-field interpolation unit 16.
[0041]
The still / moving image region determination unit 24 executes still / moving region determination processing from the current variation data input from the variation measurement unit 22 and the variation data of one timing before input from the field difference information memory 23, It is determined and stored whether the pixel to be interpolated in the latest field data is a still image region pixel or a moving image region pixel. Note that the details of the process of determining whether the pixel is a still image region pixel or a moving image region pixel will be described later.
[0042]
When the pixel to be interpolated is a pixel in the moving image region based on the information stored in the still / moving region determining unit 24, the selector 18 selects the pixel interpolated by the inter-field interpolating unit 16 from the interpolation memory 17-1. If the pixel to be interpolated is a pixel in the still image area, the pixel interpolated by the intra-field interpolating unit 15 is read from the interpolation memory 17-2, and a final progressive image is generated and output. To do.
[0043]
Next, an IP conversion process of the image processing apparatus to which the present invention is applied will be described with reference to the flowchart of FIG.
[0044]
In step S1, the field memory 11 outputs the stored field data to the image refresher 12, and then stores the input field data anew.
[0045]
In step S2, the image refresher 12 performs a one-dimensional vertical edge enhancement process.
[0046]
Here, the one-dimensional vertical edge enhancement processing of the image refresher 12 will be described with reference to the flowchart of FIG.
[0047]
In step S21, the image refresher 12 determines whether or not there is an unprocessed pixel in the field data input from the field memory 11. If it is determined that there is an unprocessed pixel, The process proceeds to step S22.
[0048]
In step S22, the image refresher 12 searches for unprocessed pixels, and calculates the vertical center pixel energy of the searched unprocessed pixels. For example, field data as shown in FIG. 4 exists, and pixels a to e, pixels f to j, and pixels k to o are arranged in each of the vertical lines y + 1, y, and y−1. Assuming that the vertical center pixel energy EV-h of the A area (range surrounded by the solid line in the figure) near the pixel h is obtained by the following equation.
[0049]
EV-h = | (b + c + d) − (l + m + n) | (1)
[0050]
Here, b, c, d, l, m, and n are pixel values of the pixels b, c, d, l, m, and n. That is, the vertical center pixel energy EV of the equation (1) is an absolute value of the difference between the sums of the pixel values existing in the upper line and the lower line with the unprocessed pixel as the center. For this reason, when correlated pixels are above and below each other, there is no large difference in pixel value, so the center pixel energy in the vertical direction is also reduced. Conversely, uncorrelated pixels are above and below each other. In this case, a large difference often appears in the difference between the pixel values, and as a result, the vertical center pixel energy also increases.
[0051]
The image refresher 12 calculates the center pixel energy EV-h in the vertical direction of the unprocessed pixel by calculating the above equation (1).
[0052]
In step S23, the image refresher 12 determines whether or not the obtained vertical center pixel energy EV is equal to or greater than a predetermined value, and when it is determined to be equal to or greater than the predetermined value, that is, unprocessed If it is determined that there is no strong correlation between the pixel and the pixel in the vertical direction, the process proceeds to step S24.
[0053]
In step S24, the image refresher 12 compares the pixel values of the upper and lower three pixels including the unprocessed pixel to obtain a maximum value and a minimum value. That is, for example, as shown in FIG. 4, when the unprocessed pixel is the pixel h, each pixel of the upper and lower pixels c, h, m (B area surrounded by a solid line in FIG. 4) including that pixel h The values are read, and the maximum value (c, h, m) and the minimum value (c, h, m) are obtained as shown in FIG.
[0054]
In step S25, the image refresher 12 performs one-dimensional vertical filter processing as shown in FIG. 5 on the pixels c, h, and m in the area B shown in FIG. That is, as a one-dimensional vertical filter, (1 / 2−α / 2, α, 1 / 2−α / 2) (1 <α ≦ 2), and the pixel value hV-filter that has been subjected to the filter processing is obtained by the calculation shown in the following equation (2).
[0055]
hV-filter = c × (1 / 2−α / 2) + h × α + m × (1 / 2−α / 2) (2)
[0056]
Where α is 1 It is a constant that can be arbitrarily set within the range of <α ≦ 2, and the degree of edge enhancement can be adjusted.
[0057]
In step S26, as shown in FIG. 5, the image refresher 12 compares the filtered pixel value hV-filter with the maximum value (c, h, m), and the filtered pixel value hV-filter is the maximum value. It is determined whether or not it is equal to or greater than (c, h, m), and when it is determined that it is equal to or greater than the maximum value (c, h, m), in step S27, the image refresher 12 sets the pixel value hV-filter. Replace with the maximum value (c, h, m).
[0058]
In step S28, the image refresher 12 stores the pixel value replaced with the maximum value (c, h, m) in the buffer 13 as the pixel value of the pixel h, and the process returns to step S21, and all the pixels are stored. The same processing is repeated until it is determined that the one-dimensional vertical edge emphasis processing has been performed.
[0059]
If it is determined in step S23 that the vertical center pixel energy EV is not greater than or equal to the predetermined value, that is, if it is determined that a strong correlation is found between the unprocessed pixel and the upper and lower pixels, The process proceeds to step S28, and the image refresher 12 stores the pixel value of the pixel h in the buffer 13 as it is without performing the filter process. The process returns to step S21, and the subsequent processes are repeated. .
[0060]
When it is determined in step S26 that the filtered pixel value hV-filter is not equal to or greater than the maximum value (c, h, m), in step S29, the image refresher 12 determines the filtered pixel value hV-filter and the minimum value. The values (c, h, m) are compared to determine whether the filtered pixel value hV-filter is less than or equal to the minimum value (c, h, m), and the filtered pixel value hV-filter is the smallest. When it is determined that the value is equal to or less than the value (c, h, m), the process proceeds to step S30.
[0061]
In step S30, the image refresher 12 replaces the pixel value hV-filter with the minimum value (c, h, m), and in step S28, converts the pixel value replaced with the minimum value (c, h, m) to the pixel value. The pixel value of h is stored in the buffer 13.
[0062]
If it is determined in step S29 that the filtered pixel value hV-filter is not less than or equal to the minimum value (c, h, m), the process proceeds to step S28, and the image refresher 12 performs the filtered pixel value. The value hV-filter is stored in the buffer 13 as the pixel value of the pixel h, and the process returns to step S21.
[0063]
That is, in the process of step S23, when the value of the vertical center pixel energy is equal to or greater than a predetermined threshold (when a strong correlation is not recognized between the unprocessed pixel and the upper and lower pixels), as shown in FIG. The maximum value (c, h, m) and the minimum value (c, h, m) obtained in step S24 are the maximum value and the minimum value of the local range of the pixels c, h, m. When the filtered pixel value obtained in the process of step S25 is included in the range of the minimum value and the maximum value, the filtered pixel value is stored in the buffer 13 and is less than the range. Stores the pixel value in the buffer 13 with the pixel value set to the minimum value, and when the value exceeds the range, the pixel value is set to the maximum value. If the value of the vertical center pixel energy is not equal to or greater than the predetermined threshold in the process of step S23, that is, if the correlation between the upper and lower pixels is strong, the original pixel value is stored in the buffer 13 as it is.
[0064]
Here, the description returns to the flowchart of FIG.
[0065]
After the one-dimensional vertical edge enhancement process is executed in step S2, the image refresher 12 executes the one-dimensional horizontal edge enhancement process in step S3.
[0066]
Here, the one-dimensional horizontal edge enhancement processing will be described with reference to the flowchart of FIG.
[0067]
In step S41, the image refresher 12 determines whether or not there is a pixel that has not been subjected to the one-dimensional horizontal edge process for each pixel of the field data stored in the buffer 13 and subjected to the one-dimensional vertical edge enhancement process, If it is determined that there is an unprocessed pixel, the process proceeds to step S42.
[0068]
In step S42, the image refresher 12 searches for unprocessed pixels and calculates the horizontal center pixel energy of the searched unprocessed pixels. For example, field data as shown in FIG. 7 exists, and pixels a to e, pixels f to j, and pixels k to o are arranged in each line of y + 1, y, y−1 in the vertical direction. Assuming that the horizontal center pixel energy EH-h of the A area (range surrounded by the solid line in the figure) in the vicinity of the pixel h is obtained by the following equation.
[0069]
EH−h = | (d + i + n) − (b + g + l) | (3)
[0070]
Here, b, d, g, i, l, and n are pixel values of the pixels b, d, g, i, l, and n. That is, the horizontal center pixel energy EH in Expression (3) is the absolute value of the difference between the sums of the pixel values existing in the right line and the left line with the unprocessed pixel as the center. For this reason, when correlated pixels are on the left and right, there is no large difference in pixel value difference, so the horizontal center pixel energy is also reduced. Conversely, uncorrelated pixels are on the left and right. In this case, since a large difference often appears in the difference between the pixel values, the central pixel energy in the horizontal direction also increases.
[0071]
The image refresher 12 calculates the above equation (3) to obtain the horizontal center pixel energy EH of the unprocessed pixel.
[0072]
In step S43, the image refresher 12 determines whether or not the obtained horizontal central pixel energy EV is equal to or greater than a predetermined value. If it is determined that the image refresher 12 is equal to or greater than the predetermined value, that is, unprocessed If it is determined that the pixel does not have a strong correlation with the left and right pixels, the process proceeds to step S44.
[0073]
In step S44, the image refresher 12 compares the pixel values of the five left and right pixels including the unprocessed pixel to obtain the maximum value and the minimum value. That is, for example, as shown in FIG. 7, when the unprocessed pixel is the pixel h, the pixel values of the left and right pixels f to j (B area surrounded by the solid line in FIG. 7) including the unprocessed pixel are As shown in FIG. 8, the maximum value (f, g, h, i, j) and the minimum value (f, g, h, i, j) are obtained.
[0074]
In step S45, the image refresher 12 performs one-dimensional horizontal filter processing as shown in FIG. 8 on the pixels f to j in the B area shown in FIG. That is, as the one-dimensional vertical filter, (1 / 4−α / 2, 1/4, α, 1/4, 1 / 4−α / 2) (1 <α ≦ 2), and the filtered pixel value hH-filter is obtained by calculating the following equation (4).
[0075]
hH-filter = f × (1 / 4−α / 2) + g × 1/4 + h × α + i × 1/4 + j × (1 / 4−α / 2) (4)
[0076]
Where α is 1 It is a constant that can be arbitrarily set within the range of <α ≦ 2, and the degree of edge enhancement can be adjusted.
[0077]
In step S46, the image refresher 12 compares the filtered pixel value hH-filter with the maximum value (f, g, h, i, j), and the filtered pixel value hH-filter has the maximum value (f, g, h, i, j) or not, and if it is determined that the value is greater than or equal to the maximum value (f, g, h, i, j), in step S47, the image refresher 12 The value hH-filter is replaced with the maximum value (f, g, h, i, j).
[0078]
In step S48, the image refresher 12 stores the pixel value replaced with the maximum value (f, g, h, i, j) in the field memory 14 as the pixel value of the pixel h, and the processing is performed in step S41. Returning, the same processing is repeated until it is determined that the one-dimensional horizontal edge enhancement processing has been performed on all the pixels.
[0079]
If it is determined in step S43 that the horizontal center pixel energy EH is not greater than or equal to the predetermined value, that is, if it is determined that a strong correlation is found between the unprocessed pixel and the left and right pixels, The process proceeds to step S48, and the image refresher 12 stores the pixel value of the pixel h in the field memory 14 as it is, and the process returns to step S41.
[0080]
If it is determined in step S46 that the filtered pixel value hH-filter is not equal to or greater than the maximum value (f, g, h, i, j), the image refresher 12 in step S49 determines that the filtered pixel value hH -filter and minimum value (f, g, h, i, j) are compared, and it is determined whether or not the filtered pixel value hH-filter is less than or equal to the minimum value (f, g, h, i, j) If it is determined that the filtered pixel value hH-filter is equal to or less than the minimum value (f, g, h, i, j), the process proceeds to step S50.
[0081]
In step S50, the image refresher 12 replaces the pixel value hH-filter with the minimum value (f, g, h, i, j). In step S48, the image refresher 12 sets the minimum value (f, g, h, i, j). The replaced pixel value is stored in the field memory 14 as the pixel value of the pixel h.
[0082]
If it is determined in step S49 that the filtered pixel value hH-filter is not less than or equal to the minimum value (f, g, h, i, j), the process proceeds to step S48, and the image refresher 12 The processed pixel value hH-filter is stored in the field memory 14 as the pixel value of the pixel h, and the process returns to step S41.
[0083]
That is, in the process of step S43, when the value of the horizontal center pixel energy is greater than or equal to a predetermined threshold (when a strong correlation is not recognized between the unprocessed pixel and the left and right pixels), as shown in FIG. The maximum value (f, g, h, i, j) and the minimum value (f, g, h, i, j) obtained in step S44 are the pixels f, g, h, i, j. When the filtered pixel value obtained in the process of step S45 is included in the range of the minimum value and the maximum value, the filtered pixel value is regarded as the maximum value and the minimum value of the local range of Is stored in the field memory 14, and when it is below the range, the pixel value is set to the minimum value, and when it is above the range, the pixel value is set to the maximum value and stored in the field memory 14. If the horizontal center pixel energy value is not equal to or greater than the predetermined threshold value in step S43, the original pixel value is stored in the field memory 14 as it is.
[0084]
Here, the description returns to the flowchart of FIG.
[0085]
In step S3, after the one-dimensional horizontal edge enhancement process is executed, in step S4, the LPF 19, the down sampler 20, the field memory 21, the fluctuation measuring unit 22, the field difference information memory 23, and the still / moving image region determining unit 24 are displayed. Cooperate to execute still / moving image region determination processing.
[0086]
Here, with reference to the flowchart of FIG. 9, the still / moving image region determination processing will be described.
[0087]
In step 61, the LPF 19 limits the band by smoothing the pixel value with a predetermined filter with respect to the pixel in the horizontal direction centered on the target pixel, and outputs it to the downsampler 20. That is, for example, when the pixels g, h, and i are arranged in the horizontal direction centering on the target pixel (in this case, the target pixel is the pixel h), the pixels g, h, and i are (1/4, The pixel value of the pixel h ′ is g / 4 + h / 2 + i / 4, which is output to the downsampler 20 with the band being limited.
[0088]
In step 62, the downsampler 20 downsamples the number of pixels of the image signal band-limited by the LPF 19 by ½ in the horizontal direction, and inter-field interpolator 16, field difference information memory 23. , And output to the still / moving image area determination unit 24. That is, for example, as field data band-limited by the LPF 19, as shown in FIG. 11, pixels a to e, pixels f to j, and pixel k are arranged in each line of y + 1, y, y−1 in the vertical direction. If down to o are arranged, the downsampler 20 has pixels a (x = 0), c (x = 2) on the line (y−1), where the horizontal coordinate x is an even number. e (x = 4), pixel f on line y (x = 0), h (x = 2), j (x = 4), and pixel k (x = 0), m on line (y + 1) (X = 2) and o (x = 4) are left as effective pixels, pixels b (x = 1) and d (x = 3) on line (y−1), and pixel g (x = x) on line y 1), i (x = 3) and pixels l (x = 1), n (x = 3) on line (y + 1) are left as invalid pixels and thinned out. And a, and generates field data down-sampled to 1/2 in the horizontal direction as a whole, the field memory 21, and outputs the variation measuring unit 22.
[0089]
In step S63, the variation measuring unit 22 reads out information on the pixel position that is the target pixel, and in step S64, the latest field data (downsampled to 1/2 in the horizontal direction) input from the downsampler 20 is input. Fluctuation data is measured from the field data stored in the field memory 21 one timing before (downsampled to ½ in the horizontal direction), and the fluctuation data memory 23 and the still / moving area determination unit 24 are measured. Output to. That is, for example, in the case of time t-1 shown in FIG. 12, when the pixel C existing on the field is the target pixel, it is viewed from the position of the pixel C on the field at time t-2, which is the previous timing. Then, the fluctuation data diff (D, E, C) is measured by calculating the following equation (5) from the pixel values of the pixels D and E positioned vertically in the horizontal direction and the pixel value of the pixel C.
[0090]
diff (D, E, C) = (D + E) / 2−C (5)
[0091]
Here, diff (D, E, C) is variation data obtained from the pixels D, E, C, and D, E, C indicate pixel values of the pixels D, E, C. That is, the pixel values of the pixels D and E positioned vertically in the horizontal direction when viewed from the position of the pixel C are read from the field data of one timing before stored in the field memory 21, and the average is obtained from the latest field data. The pixel value of the pixel C is read out, and the value subtracted from the average value of the pixels D and E is used as variation data. Further, at this next timing (time t), diff (A, B, C) is obtained. In the following description, for the target pixel, the currently obtained variation data is referred to as diff_c, and the variation data obtained at the previous timing and stored in the variation data memory 23 is referred to as diff_p. .
[0092]
In step S <b> 65, the still / moving image region determination unit 24 uses the variation data diff_c of the target pixel input from the variation measurement unit 22 and the variation data diff_p input from the variation measurement unit 22 at the previous timing. It is determined whether the data diff_c is larger than a predetermined threshold th and the multiplication result of the fluctuation data diff_c and the fluctuation data diff_p is negative or zero, for example, the fluctuation data diff_c is larger than the predetermined threshold th, If it is determined that the multiplication result of the variation data diff_c and the variation data diff_p is negative or zero, the target pixel is stored as a pixel in the moving image area in step S66.
[0093]
If it is determined in step S65 that the variation data diff_c is smaller than the predetermined threshold th or the multiplication result of the variation data diff_c and the variation data diff_p is positive, in step S67, the target pixel is set as a still image. Store as region pixels.
[0094]
In step S68, it is determined whether or not there is an unprocessed pixel. If it is determined that there is an unprocessed pixel, the process returns to step S63, and the subsequent processes are repeated. The processes in steps S63 to S68 are repeated until there are no more pixels. That is, when it is determined whether all the pixels are moving pixels or still pixels, the process ends.
[0095]
That is, when the pixel C existing on the field data at time t-1 in FIG. 12 is the target pixel, the variation data diff_p obtained from the pixels D and E on the field data at time t-2, which is the previous timing. As described above, diff_p = diff (D, E, C) = (D + E) / 2−C, and is obtained from the pixels A and B and the target pixel C on the field data at time t, which is the next timing. The fluctuation data diff_c to be obtained is diff_c = diff (A, B, C) = (A + B) / 2−C. At this time, when the variation data diff_c = diff (A, B, C) is larger than the threshold th and the multiplication result of the variation data diff_c and the variation data diff_p is negative or zero, the target pixel C is It is determined that the pixel is a pixel, and is stored in the still / moving image region determination unit 24. Conversely, when the variation data diff_c = diff (A, B, C) is smaller than the threshold th or the multiplication result of the variation data diff_c and the variation data diff_p is 1 or more, the pixel of interest C is a still image region. And is stored in the still / moving image region determination unit 24.
[0096]
The following effects can be obtained by such still / moving image region determination processing.
[0097]
That is, for example, as shown in FIG. 13, in the case of a still image in which white and black pixels are arranged in the vertical direction for each line, the field data is stored at time t (for each field data) at time t. In the field data, white pixels are arranged at Pix (y−2, t), Pix (y, t), and Pix (y + 2, t). Pixels are arranged in Pix (y-1, t-1), Pix (y + 1, t-1), Pix (y + 3, t-1), and the field data at time t-2 includes white It is assumed that pixels are arranged at Pix (y−2, t−2), Pix (y, t−2), and Pix (y + 2, t−2).
[0098]
In such a case, for example, when the change in the vertical direction is monotonously decreasing or monotonically increasing, that is, the pixel change (Pix (y−2, t) −Pix (y−1, t−1)) × If (Pix (y−1, t−1) −Pix (y, t))> 0, it is assumed that the image is a still image. In the case of FIG. 13, (Pix (y−2, t)) −Pix (y−1, t−1)) × (Pix (y−1, t−1) −Pix (y, t))> 0 is not satisfied, and may be regarded as a moving image region. . At this time, for example, when interpolating the pixels of the moving image area, if the average of the pixels above and below the target pixel is interpolated with the field data at time t or t-2, the pixels are all The pixel is interpolated to black, although the pixel is interpolated to field data on time t−1. As a result of such processing, as the time changes, the progressive image becomes white, black, white, as shown in FIGS. 14A, 14B, 14C, 14D, as time progresses as shown in FIGS. Since it is continuously switched to black, so-called flicker occurs.
[0099]
On the other hand, according to the present invention, diff (Pix (y-2, t), Pix (y-1, t-1), Pix (y-1, t-1)) * diff (Pix (y-2, t) -2), Pix (y, t-2), Pix (y-1, t-1))> 0, and diff (Pix (y-2, t), Pix (y, t), Pix ( When y−1, t−1))> th, the target pixel is set as a still image region. Therefore, even in the case of a still image as shown in FIG. Can be recognized. As a result, even if the image as shown in FIG. 13 is converted, the same pixel is set for each vertical coordinate position of each pixel, so as shown in FIGS. 15A, 15B, 15C, and 15D. In addition, even if images that change in time series are displayed, it is possible to display the images shown in FIG. 13 for all the images. As a result, it is possible to accurately identify whether the pixel to be interpolated is a moving image region or a still image region.
[0100]
In the above, in the still / moving area determination process, the determination of the still / moving area is performed based on the determination result of one pixel. For example, N (number of pixels) × centering on the target pixel × If a moving image area is determined even for one pixel out of M (number of pixels), the target pixel may be determined to be a moving image area, and otherwise, it may be determined to be a still image area.
[0101]
Here, the description returns to the flowchart of FIG.
[0102]
After the still / moving pixel region determination process is executed in step S4, the inter-field interpolation unit 16 executes inter-field interpolation processing in step S6.
[0103]
Here, the inter-field interpolation processing will be described with reference to the flowchart of FIG.
[0104]
In step S81, the inter-field interpolation unit 16 reads the position of the target pixel (the target pixel in inter-field interpolation or intra-field interpolation processing is a pixel to be interpolated), and in step S82, the target pixel position. It is determined whether or not the horizontal pixel position is an odd number. For example, when it is determined that the horizontal coordinate position of the target pixel is an odd number, the field interpolation unit 16 determines from the field data stored in the field memory 14 pixels above and below the target pixel position in step S83. Between the average value between them and the average value of the fluctuation data of the pixels existing on the left and right of the target pixel input from the fluctuation measuring unit 22 is interpolated as the pixel value of the target pixel and stored in the interpolation memory 17-1. .
[0105]
If it is determined in step S82 that the pixel position in the horizontal direction of the target pixel is not an odd number, that is, the pixel position in the horizontal direction is an even number, the field interpolation unit 16 stores the field pixel in the field memory 14 in step S84. The average value between the pixels existing above and below the target pixel position from the stored field data and the band existing at the same position as the target pixel input from the variation measuring unit 22 are limited, and ½ in the horizontal direction. Is interpolated as a pixel value of the pixel of interest and stored in the interpolation memory 17-1.
[0106]
In step S85, the inter-field interpolation unit 16 determines whether or not there is an unprocessed pixel. If it is determined that there is an unprocessed pixel, the process returns to step S81, and the subsequent processes Are repeated until the unprocessed pixels are eliminated, and the processes of steps S81 to S85 are repeated. If it is determined in step S85 that there are no unprocessed pixels, the process ends.
[0107]
That is, for example, as shown in FIG. 17, the field data at the time t stored in the field memory 14 has pixels A, B, and C in the horizontal direction at the position where the vertical coordinate is y, and is interlaced. It is assumed that G, H, and I exist in the horizontal direction at a position where the vertical coordinate of one line is y + 2. Also, the pixels D, E, and F are arranged at positions where the vertical coordinate on the field data that has been band-limited by 1/2 in the horizontal direction and y + 1 is limited by the band at the time t-1 of one timing in the past. It shall be. In the figure, the pixel at time t is indicated by a black circle, the pixel at time t-1 is indicated by a white circle, and the thinned pixels are indicated by dotted lines. Also, A, B, C, D, E, F, G, H, I, J, K, and L in the figure indicate pixels and are also used as pixel values in the following description. Furthermore, in the display of the pixel value, “′” indicates that the pixel value is band-limited by the LPF 19.
[0108]
In FIG. 17 shown in this way, when the target pixel is an odd number, for example, the target pixel is the pixel K. That is, in the field data that is band-limited and thinned by half in the horizontal direction, pixels with an odd horizontal coordinate position are thinned, and in this case, the pixel E is thinned. Among the pixels to be interpolated by the corresponding pixel K, the horizontal coordinate position becomes an odd number.
[0109]
In this case, the target pixel K is obtained by the following equation (6).
[0110]
K = (B + H) / 2− (diff (C ′, I ′, F ′) + diff (A ′, G ′, D ′) / 2 (6)
[0111]
Further, when the target pixel is an even number, for example, the target pixel is the pixel J or L. That is, in the field data that is band-limited and thinned by half in the horizontal direction, pixels with an odd horizontal coordinate position are thinned, and in this case, pixels D and F are not thinned. Among the pixels to be interpolated by the corresponding pixels J and L, the horizontal coordinate position is an even number.
[0112]
In this case, for example, the target pixel L is obtained by the following equation (7).
[0113]
L = (C + I) / 2− (diff (C ′, I ′, F ′) / 2 (7)
[0114]
That is, the terms “(B + H) / 2” and “(C + I) / 2” in Equations (6) and (7) are averages in the vertical direction, and the values of pixels that do not originally exist are adjacent in the vertical direction. This is set by interpolating between pixels. Also, the terms “(diff (C ′, I ′, F ′) + diff (A ′, G ′, D ′) / 2)” and “(diff (C ′, I ′, F ′) / 2) This is variation data indicating the relationship between the pixel to be interpolated on different fields and the pixel whose vertical position is different, so that the pixel of interest (pixel to be interpolated) is interpolated between fields by the above method. When interpolation is performed, an interpolated pixel can be generated in consideration of the relationship between pixels that should exist at the same position between fields, and as a result, a correct interpolated pixel can be generated.
[0115]
Here, the description returns to the flowchart of FIG.
[0116]
After inter-field interpolation processing is executed in step S5, the intra-field interpolation unit 15 executes intra-field interpolation processing in step S6.
[0117]
Here, the intra-field interpolation processing of the intra-field interpolation unit 15 will be described with reference to FIG.
[0118]
In step S101, the intra-field interpolation unit 15 searches the field data stored in the field memory 14, determines whether there is a non-interpolated pixel, and determines that there is a non-interpolated pixel. The process proceeds to step S102.
[0119]
In step S102, the intra-field interpolation unit 15 reads the position of the pixel to be interpolated, and in step S103, obtains near energy Near near the pixel position to be interpolated. Here, the neighborhood energy Near near the pixel position to be interpolated is, for example, as shown in FIG. 19, pixels a to e and pixels g to k exist on lines y−1 and y + 1 in the vertical direction, respectively. If the pixel f is to be interpolated on the y line in the meantime, it can be obtained from the following equation (8).
[0120]
Near-f = | a−k | + | b−j | + | c−i | + | d−h | + | e−g | (8)
[0121]
Here, a to k are pixel values of the pixels a to k.
[0122]
That is, the neighborhood energy Near in the vicinity of the pixel position to be interpolated is a pixel value at a position that is symmetric with respect to the pixel position to be interpolated among the pixels existing on the upper and lower lines of the pixel position to be interpolated. The absolute value of the difference between them is added. For this reason, the larger the value of the near energy Near, the higher the possibility that the pixel to be interpolated forms an edge portion. Conversely, the smaller the value is, the more the pixel to be interpolated is the edge or texture. The possibility that it does not exist in the part increases.
[0123]
In step S104, the intra-field interpolation unit 15 determines whether or not the near energy Near is greater than a predetermined value, and determines that it is large (when it is determined that the possibility of forming an edge portion is high). The process proceeds to step S105.
[0124]
In step S105, the intra-field interpolation unit 15 obtains a combination of pixels in a direction having a high correlation from pixels in the vicinity of the pixel to be interpolated. That is, the correlation is an absolute value of a difference in pixel value between pixels existing at a point-symmetric position with respect to the pixel to be interpolated. For example, as shown in FIG. 19, when the pixel f is to be interpolated, the correlations to be obtained are | a−k |, | b−j |, | c−i |, | d−h |, | e−. g |. Further, after obtaining these, the combination having the smallest value is detected, and this is set as the combination of pixels having the highest correlation. In other words, a high correlation means that there is no significant change in the pixel value, so that the smaller the absolute value of the difference, the higher the correlation. For example, when | b−j | takes the smallest value, the linear direction connecting the pixels b and j is a combination of pixels in a direction with high correlation.
[0125]
In step S106, an interpolation pixel is generated from the combination of pixels in the detected direction. That is, for example, when the combination of the pixels b and j shown in FIG. 19 is the combination of pixels in the direction with the highest correlation, the pixel value of the pixel f to be interpolated can be obtained from the following equation (9).
[0126]
f = 1/2 × (b + j) (9)
[0127]
In step S107, the inter-field interpolation unit 14 determines whether or not the interpolated pixel value is correctly changed in the vertical direction. For example, when interpolating the pixel f shown in FIG. 19, the intra-field interpolation unit 15 calculates a determination formula J shown in the following formula (10), and determines this based on the sign of the calculation result.
[0128]
J = (f−c) × (if) (10)
[0129]
Here, c and f are pixel values of the pixels c and f existing above and below the interpolated pixel f.
[0130]
That is, in the normal vertical direction, the pixel value is likely to monotonously increase or monotonously decrease, and it is determined whether or not the change in the pixel value is correct. The judgment formula J has a positive value when monotonically increasing or monotonically decreasing, and conversely, when judging non-monotonically increasing or monotonically decreasing, it has a negative value.
[0131]
Therefore, when the determination formula J is a positive value, the intra-field interpolation unit 15 determines that the pixel value has changed correctly in the vertical direction, and the processing proceeds to step S108.
[0132]
In step S108, the intra-field interpolation unit 15 stores the pixel value generated from the combination of pixels in the direction of high correlation in the interpolation memory 17-2 as the pixel value of the interpolation pixel, and the process returns to step S101. The above processing is repeated until all the pixels are interpolated.
[0133]
If it is determined in step S104 that the near energy Near is not larger than the predetermined threshold value, a pixel value is generated by linear interpolation using upper and lower pixels in step S109, and the generated pixel value is stored in the interpolation memory 17-2. Remember me. That is, for example, when the pixel f shown in FIG. 19 is interpolated, a pixel value is obtained by linear interpolation from the pixel values of the pixels c and i existing above and below by the following equation (11), and this is obtained as the pixel value of the interpolated pixel. Is stored in the interpolation memory 17-2, and the process returns to step S101, and the process is repeated until all the pixels to be interpolated are interpolated.
[0134]
f = 1/2 (c + i) (11)
[0135]
If it is determined in step S107 that the pixel value has not changed correctly in the vertical direction, that is, if the determination formula J is a negative value, the processing proceeds to step S109.
[0136]
Here, the description returns to the flowchart of FIG.
[0137]
When the intra-field interpolation processing is executed in step S6, the selection processing is executed by the selector 18 in step S7.
[0138]
Here, the selection process will be described with reference to the flowchart of FIG.
[0139]
In step S121, the selector 18 determines whether or not there is a pixel of interest that is not interpolated. In step S122, the selector 18 reads the position of the target pixel. In step S123, the selector 18 inquires to the still / moving image region determination unit 24 whether or not the pixel corresponding to the read pixel position is a moving pixel, and determines that the pixel is a moving pixel. If so, the process proceeds to step S124.
[0140]
In step S124, the selector 18 reads out the pixel value interpolated as a pixel in the moving image area stored in the interpolation memory 17-2 corresponding to the read pixel position, and inserts it as a pixel to be output as a progressive screen. And the process returns to step S121.
[0141]
If it is determined in step S123 that the target pixel is not a moving pixel, that is, a still pixel, in step S125, the selector 18 is stored in the interpolation memory 17-1 corresponding to the read pixel position. The pixel value interpolated as a pixel of the still image area is read out, inserted and stored in the pixel to be output as a progressive screen, and the process returns to step S121. In step S121, all the pixels are interpolated. Repeat this process until
[0142]
That is, the selector 18 makes an inquiry to the still / moving image region determination unit 24 based on the read position of the target pixel, and is stored as a still image region pixel or a moving image region pixel for each pixel. If it is a pixel in the moving image area, the pixel value interpolated as a moving pixel by the intra-field interpolation unit 15 stored in the interpolation memory 17-2 is inserted, and the target pixel is a still image area. If there is, the pixel value interpolated as a still pixel by the inter-field interpolation unit 16 stored in the interpolation memory 17-1 is inserted.
[0143]
Here, the description returns to the flowchart of FIG.
[0144]
When the selection process in step S7 ends, in step S8, the selector 18 outputs the generated progressive image to a subsequent apparatus (not shown).
[0145]
In the above description, the image refresher 12 has been described using a 3-tap vertical filter and a 5-tap horizontal filter. However, a 3-tap vertical filter and a 3-tap horizontal filter may be used. In this case, for example, as shown in FIG. 21, the vertical filter is (1 / 4−α / 2, α + 1/2, 1 / 4−α / 2), and as shown in FIG. May be (1 / 4−α / 2, α + 1/2, 1 / 4−α / 2). Here, the horizontal filter is used for every five pixel values read out (for example, when the pixels f, g, h, i, j are sequentially read, the horizontal filters are applied to the pixels f, h, j). ) Apply filter processing. In addition, the case of a 3-tap vertical filter and a 5-tap horizontal filter, and the case of a 3-tap vertical filter and a 3-tap horizontal filter may both be used and switched by the user. Further, the horizontal filter and the vertical filter may have different tap numbers. Further, the field data or pixels used for interpolation do not have to be continuous, and may be used, for example, one by one or more than that.
[0146]
Furthermore, the calculation formula of the neighborhood energy in the vicinity of the unprocessed pixel may use the following formula (12) in addition to the formula (8), or may be switched by the user.
[0147]
Near-f = | a−g | + | b−h | + | c−i | + | d−j | + | e−k | (12)
[0148]
According to the above, in the process of converting from an interlaced image to a progressive image, it is possible to correctly emphasize edges even in an image with a low high-frequency component, improve the vertical resolution of the still image region, and Edges can be interpolated smoothly, and errors during conversion processing can be suppressed, and whether the interpolated pixel is a moving image area or a still image area is accurate. And correct interpolation pixels can be generated. .
[0149]
The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.
[0150]
FIG. 23 shows a configuration of an embodiment of a personal computer when the image processing apparatus is realized by software. The CPU 101 of the personal computer controls the entire operation of the personal computer. Further, when a command is input from the input unit 106 such as a keyboard or a mouse from the user via the bus 104 and the input / output interface 105, the CPU 101 stores the instruction in a ROM (Read Only Memory) 102 correspondingly. Run the program. Alternatively, the CPU 101 reads a program read from the magnetic disk 111, the optical disk 112, the magneto-optical disk 113, or the semiconductor memory 114 connected to the drive 110 and installed in the storage unit 108 into a RAM (Random Access Memory) 103. To load and execute. Thereby, the functions of the above-described image processing apparatus are realized by software. Further, the CPU 101 controls the communication unit 109 to communicate with the outside and exchange data.
[0151]
As shown in FIG. 23, the recording medium on which the program is recorded is distributed to provide the program to the user separately from the computer, and a magnetic disk 111 (including a flexible disk) on which the program is recorded, By a package medium comprising an optical disk 112 (including compact disk-read only memory (CD-ROM), DVD (digital versatile disk)), a magneto-optical disk 113 (including MD (mini-disc)), or a semiconductor memory 114 In addition to being configured, it is configured by a ROM 102 in which a program is recorded and a hard disk included in the storage unit 108 provided to the user in a state of being pre-installed in a computer.
[0152]
In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.
[0153]
【The invention's effect】
According to the image processing apparatus, method, and program of the present invention, it is possible to accurately identify whether a pixel to be interpolated is a moving image region or a still image region, and to generate a correct interpolation pixel. .
[Brief description of the drawings]
FIG. 1 is a block diagram of an image processing apparatus to which the present invention is applied.
FIG. 2 is a flowchart for explaining IP conversion processing of the image processing apparatus in FIG. 1;
FIG. 3 is a flowchart illustrating a one-dimensional vertical edge enhancement process.
FIG. 4 is a diagram illustrating a one-dimensional vertical filter.
FIG. 5 is a diagram illustrating a one-dimensional vertical filter.
FIG. 6 is a flowchart illustrating a one-dimensional horizontal edge enhancement process.
FIG. 7 is a diagram illustrating a one-dimensional horizontal filter.
FIG. 8 is a diagram illustrating a one-dimensional horizontal filter.
FIG. 9 is a flowchart illustrating still / moving image region determination processing.
FIG. 10 is a diagram for explaining still / moving image region determination processing;
FIG. 11 is a diagram for explaining still / moving image region determination processing;
FIG. 12 is a diagram for explaining still / moving image region determination processing;
FIG. 13 is a diagram for describing still / moving image region determination processing;
FIG. 14 is a diagram illustrating still / moving image region determination processing.
FIG. 15 is a diagram for explaining still / moving image region determination processing;
FIG. 16 is a flowchart illustrating inter-field interpolation processing.
FIG. 17 is a diagram illustrating inter-field interpolation processing.
FIG. 18 is a flowchart illustrating intra-field interpolation processing.
FIG. 19 is a diagram for explaining intra-field interpolation processing;
FIG. 20 is a flowchart illustrating select processing.
FIG. 21 is a diagram illustrating a one-dimensional vertical filter.
FIG. 22 is a diagram illustrating a one-dimensional horizontal filter.
FIG. 23 is a diagram illustrating a medium.
[Explanation of symbols]
11 field memory, 12 image refresher, 13 buffer, 14 field memory, 15 intra-field interpolation unit, 16 inter-field interpolation unit, 17-1, 17-2 interpolation memory, 18 selector, 19 LPF, 20 down sampler, 21 field Memory, 22 Fluctuation measurement unit, 23 Fluctuation data memory, 24 Still / Movie area determination unit

Claims (11)

In an image processing apparatus that converts an interlaced image into a progressive image,
Intra-field interpolation means for interpolating the pixels of the progressive image from the pixels in the field of the interlaced image to be converted;
Of the target pixel of the field of the interlaced image to be converted and the pixel of the field different from the target pixel, an average pixel of two pixels adjacent to the target pixel in the vertical direction or the horizontal direction A variation measuring means for measuring a variation between fields from a difference from a value;
Inter-field interpolation means for interpolating the progressive image pixels from the variation measured by the variation measurement means and the interlace image field pixels to be converted;
The magnitude relationship with the predetermined threshold value of the variation of the pixel of interest measured by the variation measurement means, and the result of multiplying the variation of the pixel of interest with the variation of the pixel in the same position and in a different field from the pixel of interest Determination means for determining whether a pixel of the progressive image to be interpolated is a moving pixel or a still pixel based on the sign of
When a pixel of the progressive image to be interpolated is a moving pixel based on a determination result of the determination unit, the pixel interpolated by the intra-field interpolation unit is selected, and the progressive image to be interpolated is selected. A selection unit that selects the pixel interpolated by the inter-field interpolation unit;
Bandwidth limiting means for bandwidth limiting the field of the interlaced image;
A decimation unit that decimates the number of pixels of the field of the interlaced image, which is band-limited by the band-limiting unit, to 1/2 in the horizontal direction;
The variation measuring means includes an average pixel value between pixels adjacent to the upper and lower sides of a pixel in a field in which the number of pixels is thinned by a half in the horizontal direction by the thinning means, and the immediately preceding or immediately following the field, The difference between the pixel values of the pixels located in the center between the vertically adjacent pixels on the field in which the number of pixels is thinned by 1/2 in the horizontal direction by the thinning means is calculated at the center between the vertically adjacent pixels. Measured as the variation between the fields of the located pixels,
The inter-field interpolation means includes
When the pixel to be converted is the same as the horizontal position of the pixel that has not been thinned out by the thinning means, the average pixel value of the pixels above and below it and 1 / of the fluctuation of the pixel to be converted The pixel value is interpolated by the difference from 2 times,
When the pixel to be converted is the same as the horizontal position of the pixel thinned by the thinning means , the average pixel value of the pixels existing above and below the pixel and the left and right pixels of the pixel to be converted An image processing apparatus characterized by interpolating a pixel value based on a difference from an average of fluctuations. The image processing apparatus according to claim 1, further comprising an edge enhancement unit that enhances an edge of the interlaced image to be converted. The edge enhancement means enhances the edge of the image by performing horizontal and vertical one-dimensional filter processing for each pixel of the interlaced image. Image processing device. Center pixel energy calculating means for calculating center pixel energy in the vertical direction and in the horizontal direction in the vicinity of the pixel from the pixel values in the vicinity of the pixels,
The edge enhancement means performs a filtering process on the pixels in which the vertical and horizontal central pixel energies calculated by the central pixel energy calculating means exceed a predetermined value. The image processing apparatus according to claim 2. In the field of the interlaced image to be converted, further comprising a correlation detection means for detecting a correlation between pixels at positions opposite to the pixels on the upper and lower horizontal lines of the pixel to be interpolated,
The image processing apparatus according to claim 1, wherein the intra-field interpolation unit interpolates the pixels from pixels having the strongest correlation detected by the correlation detection unit. The image processing apparatus according to claim 5, wherein the correlation detection unit detects an absolute value of a difference between pixel values of pixels at contrasting positions with the pixel to be interpolated as a center, as a correlation. . It further comprises a neighborhood energy calculation means for calculating the neighborhood energy of the pixel to be interpolated from pixel values in the vicinity of the pixel to be interpolated,
If the value of the neighborhood energy of the pixel to be interpolated calculated by the neighborhood energy calculation means is greater than a predetermined threshold, the intra-field interpolation means determines the pixel with the strongest correlation detected by the correlation detection means. The image processing apparatus according to claim 5, wherein interpolation is performed based on the interpolation. The neighborhood energy is the sum of absolute values of differences between pixel values of pixels facing each other vertically among a predetermined number of pixels on a line extending horizontally above and below the pixel to be interpolated. Or a sum of absolute values of pixel value differences of pixels existing at point-symmetric positions with a pixel to be interpolated as a center. 8. The image processing apparatus according to 7. In an image processing method of an image processing apparatus for converting an interlaced image into a progressive image,
An intra-field interpolation step of interpolating pixels of the progressive image from pixels in the field of the interlaced image to be converted;
Of the target pixel of the field of the interlaced image to be converted and the pixel of the field different from the target pixel, an average pixel of two pixels adjacent to the target pixel in the vertical direction or the horizontal direction A variation measurement step for measuring the variation between fields from the difference from the value;
An inter-field interpolation step of interpolating the progressive image pixels from the interlaced image field pixels to be converted, and the variation measured in the variation measuring step;
The magnitude relationship with the predetermined threshold value of the variation of the target pixel measured by the processing of the variation measurement step, and the variation of the target pixel and the variation of the pixel in the same position and in a different field from the target pixel A determination step of determining whether a pixel of the progressive image to be interpolated is a moving pixel or a still pixel based on the result of multiplication;
Based on the determination result in the determination step processing, when the pixel of the progressive image to be interpolated is a moving pixel, the pixel interpolated by the intra-field interpolation step processing is selected and the interpolation is performed. A selection step of selecting the pixel interpolated by the processing of the inter-field interpolation step, when the pixel of the progressive image is a still pixel;
A bandwidth limiting step for bandwidth limiting the field of the interlaced image;
A decimation step of decimation of the number of pixels of the field of the interlaced image, which is band-limited by the processing of the band-limiting step, to 1/2 in the horizontal direction,
The process of the variation measuring step includes an average pixel value between adjacent pixels above and below a pixel of a field in which the number of pixels is thinned by a half in the horizontal direction by the process of the thinning step, and immediately before the field, or Immediately after that, the difference between the pixel values of the pixels located in the center between the vertically adjacent pixels on the field where the number of pixels is thinned by 1/2 in the horizontal direction by the thinning step processing Measured as the variation between the field of pixels located in the center between
The inter-field interpolation step processing is as follows:
When the pixel to be converted is the same as the horizontal position of the pixel that has not been thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below it and the variation of the pixel to be converted Interpolate the pixel value by the difference from 1/2,
When the pixel to be converted is the same as the horizontal position of the pixel thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below the pixel and the left and right of the pixel to be converted An image processing method characterized by interpolating pixel values based on a difference from an average of pixel fluctuations. A program for controlling an image processing apparatus that converts an interlaced image into a progressive image,
An intra-field interpolation step of interpolating pixels of the progressive image from pixels in the field of the interlaced image to be converted;
Of the target pixel of the field of the interlaced image to be converted and the pixel of the field different from the target pixel, an average pixel of two pixels adjacent to the target pixel in the vertical direction or the horizontal direction A variation measurement step for measuring the variation between fields from the difference from the value;
An inter-field interpolation step of interpolating the progressive image pixels from the interlaced image field pixels to be converted, and the variation measured in the variation measuring step;
The magnitude relationship with the predetermined threshold value of the variation of the target pixel measured by the processing of the variation measurement step, and the variation of the target pixel and the variation of the pixel in the same position and in a different field from the target pixel A determination step of determining whether a pixel of the progressive image to be interpolated is a moving pixel or a still pixel based on the result of multiplication;
Based on the determination result in the determination step processing, when the pixel of the progressive image to be interpolated is a moving pixel, the pixel interpolated by the intra-field interpolation step processing is selected and the interpolation is performed. A selection step of selecting the pixel interpolated by the processing of the inter-field interpolation step, when the pixel of the progressive image is a still pixel;
A bandwidth limiting step for bandwidth limiting the field of the interlaced image;
A decimation step of decimation of the number of pixels of the field of the interlaced image, which is band-limited by the processing of the band-limiting step, to 1/2 in the horizontal direction,
The process of the variation measuring step includes an average pixel value between adjacent pixels above and below a pixel of a field in which the number of pixels is thinned by a half in the horizontal direction by the process of the thinning step, and immediately before the field, or Immediately after that, the difference between the pixel values of the pixels located in the center between the vertically adjacent pixels on the field where the number of pixels is thinned by 1/2 in the horizontal direction by the thinning step processing Measured as the variation between the field of pixels located in the center between
The inter-field interpolation step processing is as follows:
When the pixel to be converted is the same as the horizontal position of the pixel that has not been thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below it and the variation of the pixel to be converted Interpolate the pixel value by the difference from 1/2,
When the pixel to be converted is the same as the horizontal position of the pixel thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below the pixel and the left and right of the pixel to be converted A recording medium on which a computer-readable program is recorded, wherein a pixel value is interpolated based on a difference from an average of pixel fluctuations. A computer that controls an image processing apparatus that converts an interlaced image into a progressive image.
An intra-field interpolation step of interpolating pixels of the progressive image from pixels in the field of the interlaced image to be converted;
Of the target pixel of the field of the interlaced image to be converted and the pixel of the field different from the target pixel, an average pixel of two pixels adjacent to the target pixel in the vertical direction or the horizontal direction A variation measurement step for measuring the variation between fields from the difference from the value;
An inter-field interpolation step of interpolating the progressive image pixels from the interlaced image field pixels to be converted, and the variation measured in the variation measuring step;
The magnitude relationship with the predetermined threshold value of the variation of the target pixel measured by the processing of the variation measurement step, and the variation of the target pixel and the variation of the pixel in the same position and in a different field from the target pixel A determination step of determining whether a pixel of the progressive image to be interpolated is a moving pixel or a still pixel based on the result of multiplication;
Based on the determination result in the determination step processing, when the pixel of the progressive image to be interpolated is a moving pixel, the pixel interpolated by the intra-field interpolation step processing is selected and the interpolation is performed. A selection step of selecting the pixel interpolated by the processing of the inter-field interpolation step, when the pixel of the progressive image is a still pixel;
A bandwidth limiting step for bandwidth limiting the field of the interlaced image;
A process including a decimation step of decimation of the number of pixels of the field of the interlaced image field, which is band-limited by the process of the band limitation step, to 1/2 in the horizontal direction,
The process of the variation measuring step includes an average pixel value between adjacent pixels above and below a pixel of a field in which the number of pixels is thinned by a half in the horizontal direction by the process of the thinning step, and immediately before the field, or Immediately after that, the difference between the pixel values of the pixels located in the center between the vertically adjacent pixels on the field where the number of pixels is thinned by 1/2 in the horizontal direction by the thinning step processing Measured as the variation between the field of pixels located in the center between
The inter-field interpolation step processing is as follows:
When the pixel to be converted is the same as the horizontal position of the pixel that has not been thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below it and the variation of the pixel to be converted Interpolate the pixel value by the difference from 1/2,
When the pixel to be converted is the same as the horizontal position of the pixel thinned out by the processing of the thinning step, the average pixel value of the pixels existing above and below the pixel and the left and right of the pixel to be converted A program that interpolates pixel values based on the difference from the average of pixel fluctuations.

Images