JP2008028621A - Video processor and video processing method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique for making conversion processing fast while preventing the distortion of a picture caused during sequential scan conversion of the picture. <P>SOLUTION: A video processor 1 comprises a data storage means (1) of storing a video signal of interlaced scanning for every frame comprising two fields; a distortion detecting means (2) of detecting whether each pixel of an interpolation field of the video signal in the data storage means has distortion relative to a reference field; interpolating function selecting means (3, 4) of selecting a pair of interpolating functions related to whether there is distortion for every pixel of the interpolation field from a plurality of interpolating functions corresponding to a plurality of pixel patterns defining an array of respective pixels of the interpolation field and reference pixels to be applied to the respective pixels; a selecting means (5) of selecting an interpolating function to be applied to each pixel of the interpolation field between the pair of interpolating functions selected for the pixel according to whether the pixel has distortion; and an interpolating means (6) of finding a value of each pixel of the interpolation field by the interpolating function selected for the pixel. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a video processing technique, and more particularly, to a process of converting an interlace video signal into a progressive video signal.

  2. Description of the Related Art Conventionally, video scanning formats include interlace scanning that considers suppression of the communication band of video signals and sequential scanning that considers improvement in image quality. In addition, various techniques have been proposed for so-called progressive scanning conversion in which interlaced scanning video signals are converted into progressive scanning video signals as the image quality is improved. As an example, there is a technique described in Patent Document 1 described later.

  In Patent Document 1, an interlaced scanned image obtained by performing a decoding process on each code string for each field or frame is applied to the interlaced scanned image composed of a plurality of fields using motion compensation. A progressive scan conversion method for converting into a progressive scan image is described. In this method, a filter coefficient of a VT filter applied to a moving image is obtained using data for two frames including a field subjected to progressive scan conversion.

Also, paragraphs [0121] to [0125] and FIGS. 20 and 21 of the same document describe a method of generating pixels by intra-field interpolation at the time of sequential scan conversion. Here, attention is paid to the interpolation pixel and two pixels linearly combined in the vertical and oblique directions around the interpolation pixel from the upper and lower lines of the generated interpolation pixel (FIG. 20). Then, the direction in which the difference absolute value between the two pixels is minimized is selected from the five directions indicated by the pixel combination of interest, and the average value of the two pixels in the selected direction is obtained as the value of the interpolation pixel.
JP 2002-223419 A

  According to the above-described method described in Patent Document 1, since interpolation pixels are obtained using two pixels that are linearly connected around the interpolation pixel, effective interpolation can be expected in a video portion including a straight line. However, the interlaced scanning image is not limited to a straight line, but includes an image portion including, for example, a curve. Even if two pixels having a linear positional relationship are used in the interpolation processing for such a portion, the two pixels do not correspond to the curved shape of the video. Therefore, in the video portion including the curve, it is difficult to perform the interpolation properly, and as a result, there is a possibility that the image quality is deteriorated.

  Further, in applications that require high-speed data processing such as moving images, it is required to perform sequential scan conversion more quickly. For this purpose, for example, it is beneficial to reduce the data amount of pixels handled in the interpolation process. However, in the above method using the VT filter method, it is necessary to read data for two frames and calculate a difference between fields included therein every time an interpolation filter coefficient is obtained. Therefore, the above method has a disadvantage that it is difficult to realize a rapid sequential conversion process.

  The present invention has been made in view of the above problems, and an object of the present invention is to provide a technique for speeding up conversion processing while eliminating image distortion that may occur at the time of progressive scan conversion of an image.

  The video processing apparatus according to the present invention includes a data storage means for storing a video signal for interlaced scanning for each frame of two fields, and the other of the pixels in the interpolation field to be interpolated in the video signal of the data storage means. Corresponding to a plurality of pixel patterns that define the arrangement of distortion detection means for detecting the presence or absence of distortion with respect to a reference field that is a field, and each pixel of the interpolation field and reference pixels of the reference field to be applied to interpolation of each pixel An interpolation function selection means for selecting, for each pixel in the interpolation field, a pair of interpolation functions associated with the presence or absence of the distortion from a plurality of interpolation functions, and an interpolation function to be applied to each pixel in the interpolation field is selected for the pixel Selection means for selecting according to the presence or absence of distortion of the pixel from a pair of interpolation functions, and each pixel of the interpolation field Values and an interpolation means for obtaining the interpolation function is selected for the pixel.

  According to the present invention, when a video signal is subjected to progressive scanning conversion, the interpolation function of the pixel is selected according to the presence or absence of the distortion of the pixel to be interpolated, so that the video distortion is removed after the conversion. The obtained video signal can be obtained. Further, since the interpolation process is performed within one frame, the data handled in the interpolation process can be simplified, and as a result, the process can be speeded up.

  FIG. 1 shows the configuration of the first embodiment of the present invention. Referring to FIG. 1, the first embodiment of the present invention includes a data storage unit 1, a distortion detection unit 2, a first interpolation function selection unit 3, a second interpolation function selection unit 4, Selection means 5 and interpolation means 6 are included.

  The data storage unit 1 includes a storage device and a function for controlling reading and writing of data with respect to the storage device. The data storage means 1 stores a top field video signal composed of the first line to the 240th line and a bottom field video signal composed of the 241st line to the 480th line, which are interlaced and scanned by the NTSC system.

  FIG. 3 schematically shows a storage form of the data storage unit 1. As shown in FIG. 3, the storage areas of odd lines such as “storage area of line 1”, “storage area of line 3”, “storage area of line 5”,..., “Storage area of line 479” The top field video signal is stored. In addition, in the storage area of even lines such as “storage area of line 2”, “storage area of line 4”, “storage area of line 6”,..., “Storage area of line 480”, the bottom field described above is used. Are stored. That is, a video signal for one frame is formed by the video signals stored in the storage areas from “the storage area of line 1” to “the storage area of line 480”.

  In the case of the PAL system, the top-field video signal composed of the first line to the 288th line is sequentially stored in the storage area of the odd lines. Further, the bottom-field video signal composed of the 289th line to the 576th line is sequentially stored in the storage area of the even line.

  In the above description, the effective area of the NTSC system is 480 lines, and the effective area of the PAL system is 576 lines. The line number of the data storage means 1 starts from an odd number line starting with “1”.

  Hereinafter, the configuration of the present embodiment will be described with reference to the flowchart shown in FIG. 22 based on the operation procedure of the present embodiment.

  The distortion detection means 2 reads the current pixel to be interpolated and its neighboring pixels from the data storage means 1 (step S1), and observes their continuity, thereby forming a comb shape as shown in FIG. Is detected (step S2).

  Here, an example in which an interpolation field to be interpolated is a bottom field and distortion of the bottom field with respect to the top field is detected will be described with reference to FIGS. 5, 6, and 8. From FIG. 5, it is assumed that a pixel in the bottom field that is a distortion detection target is “x”. When the distortion of the current pixel x is detected, the current pixel x, the top field pixels a, b, c, g, and h above and below it, and the pixels d, e, f, i, and j, respectively, The degree of connection is detected. That is, the presence / absence of distortion of the current pixel x is detected by verifying the continuity of the video regarding the current pixel x.

  Nine types of combinations of “pattern 1” to “pattern 9” shown in FIG. 6 are used for the combination of the reference pixel and the current pixel used for the above-described distortion detection. The nine types of patterns shown in the drawing are patterns in which a total of three pixels, one pixel at the top and one at the center of the current pixel x, are combined, focusing on five pixels in the top and bottom top fields sandwiching the current pixel x. These nine types of patterns include a pattern of three pixels adjacent in the vertical or oblique direction as “pattern 1” to “pattern 3” and three pixels not adjacent such as “pattern 4” and “pattern 5”. And a pattern of three pixels adjacent to each other by changing the direction in the vertical and oblique directions, such as “Pattern 6” to “Pattern 9”.

  In the evaluation of video continuity, it is determined that there is video continuity as the difference between the current pixel x and the reference pixel is small. Conversely, it can be considered that as the difference between the current pixel x and the reference pixel is larger, there is some distortion (step) between the pixels. For example, when the image is flat with little change, distortion (steps) hardly occurs. In this case, the difference between the current pixel x and all the reference pixels (a to j) is smaller than a predetermined distortion determination threshold value.

  On the other hand, when there is a pattern in the video, for example, when there is a pattern like a line passing from the reference pixel a through the current pixel x to the reference pixel f as shown in “Pattern 1” in FIG. 6, the current pixel x and the reference pixel a There is little difference in pixel values between the reference pixel f and the reference pixel f. In this case, it is determined that the continuity between pixels in “pattern 1” is high.

  As described above, when the image has a pattern, the current pixel x and the reference pixel (a to j) are not affected in any of the nine patterns shown in FIG. 6 centering on the current pixel x. The difference in pixel value becomes smaller. When the video signal is distorted, the current pixel x has a large difference from one or both of the upper and lower reference pixels. Distortion can be detected by utilizing such a property of video.

  FIG. 8 shows equations used for distortion detection. The “pattern 1” in FIG. 6 is evaluated using the combination of the formulas 1 and 2 shown in the figure. “Pattern 2” is evaluated by a combination of Equation 3 and Equation 4, “Pattern 3” is evaluated by Equation 5 and Equation 6, “Pattern 4” is evaluated by Equation 7 and Equation 8, and “Pattern 5” is evaluated by a combination of Equation 9 and Equation 10. “Pattern 6” is based on Formula 1 and Formula 4, “Pattern 7” is based on Formula 5 and Formula 4, “Pattern 8” is based on Formula 3 and Formula 6, and “Pattern 9” is based on Formula 2 and Formula 3, respectively. evaluate.

  When evaluating the pattern, if the difference between the current pixel x and the reference pixel (a to j) is greater than or equal to a preset distortion determination threshold in all patterns (1 to 9), the continuity between the current pixel and neighboring pixels is determined. Is low and is evaluated as being distorted.

  The first interpolation function selection means 3 reads the current pixel to be interpolated and its neighboring pixels from the data storage means 1 and selects an interpolation function that is optimal among the first interpolation functions ( FIG. 22: Step S3). In the selection, the first interpolation function selection unit 3 firstly selects between the two reference pixels centered on the current pixel x from the nine patterns in FIG. 6 based on Expressions 11 to 19 in FIG. A combination having the smallest difference is obtained, and this is set as the first optimum interpolation function. This means that a combination having the closest values of the current pixel and the reference pixel is obtained. By performing interpolation using the closest combination, the continuity of the video is maintained.

  The pattern group of FIG. 6 used by the first interpolation function selection unit 3 corresponds to the second pixel pattern group in the present invention, that is, the pattern group associated with the fact that the interpolation target pixel is not distorted.

  However, if the difference between the interpolation function having the smallest difference absolute value and the interpolation function having the second smallest absolute difference value is very small, the value having the smallest difference absolute value is the second. The obtained interpolation function may be considered optimal. Such a case will be described with reference to FIG.

  FIG. 18A shows the arrangement of pixels x to be interpolated and upper and lower line reference pixels (a to j) to be an interpolation function. FIG. 18B shows an example of pixel values of the top field serving as an interpolation function. Furthermore, FIG. 18C shows the state of the connection of the video predicted from the pixel value of the top field. As can be seen from the arrangement of pixel values, pixels with similar values are located from the upper left to the lower right of the figure. FIG. 18 (f) shows the absolute difference value for each interpolation pattern obtained by Expression 11 to Expression 19 in FIG. 9 under such conditions.

  As shown in FIG. 18F, “s5” indicates the smallest value among the absolute difference values, and “s1” is the second smallest value. In this case, the interpolation function corresponding to the first minimum value “s5”, which is the smallest value, is set as the first optimum interpolation function. The interpolation function corresponding to “s5” is a function for obtaining the value of the current pixel by rounding off the average value of the pixel h and the pixel i, which are reference pixels of “pattern 5” (FIG. 6).

  FIG. 18D shows the value of the current pixel x obtained from the first optimal interpolation function of “Pattern 5”. The current pixel x shown in FIG. 18D is interpolated with a value of “119” in the pixel value band around “200” from the upper left to the lower right as shown in FIG. 18C. Yes. In this way, when the interpolated pixel value (“119”) is a value that is significantly separated from the surrounding pixel values (around “200”), the interpolated pixel is conspicuous, resulting in distortion on the video. Since it appears easily, it is not preferable.

  Therefore, referring to FIG. 18F, the value of the second minimum value “s1” is referred to as “11”, and this value and the first minimum value “1” obtained in “s5” are The difference is “10”. When this difference is regarded as a small difference, the process proceeds from the expression 25 to the expression 33 shown in FIG. 19 in order to verify the appropriateness of the interpolation by the “pattern 1” in FIG. 6 corresponding to the second minimum value “s1”.

  Whether or not the difference between the first and second minimum values is regarded as a small difference is evaluated based on a preset threshold value. For example, when the threshold is set to “20”, it is determined that the difference “10” due to the above case corresponds to a narrow difference. At this time, if the difference between the first and second minimum values is “30”, it is determined that the difference is not a slight difference, and the process proceeds with the first minimum value.

  The processing using the equation shown in FIG. 19 regarding the second minimum value is obtained by calculating each value of the pixel b and the pixel e located at the center of the top field shown in FIG. 18A and the value of the pixel used as the reference pixel. In this process, the interpolation function indicated by the reference pixel when the difference is minimized is obtained as the second optimum interpolation function. FIG. 18G shows the values “s11” to “s19” obtained by using the equations 25 to 33 in FIG.

  From FIG. 18G, among “s11” to “s19”, the value “68” obtained by “s11” is the smallest. In this case, “pattern 1” of “s11” that is minimized is selected as the second optimum interpolation function. Note that “s15” according to Expression 29 (FIG. 19) corresponding to “Pattern 5” selected as the first optimal interpolation function in the previous processing is the pixel value (b, e) located at the center of the top field. The difference is “152”, which is the maximum value. In this case, it is determined that the value of the current pixel x is far away from the value of each reference pixel (b, e) in the center of the top field in the interpolation by “pattern 5”.

  FIG. 18E shows the value of the current pixel x when interpolation is performed using the second optimal interpolation function. As shown in FIG. 18E, the pixel value interpolated using the second optimal interpolation function is the average value “229” (“234” and “223”) of the reference pixels a and f of “pattern 1”. Rounded off). According to this interpolation value, continuity with neighboring pixels is maintained, so that it is understood that it is appropriate to select the second optimum interpolation function in this example.

  Here, with reference to the flowchart shown in FIG. 23, the above-described series of processing by the first interpolation function selection means 3 will be summarized. First, the first interpolation function selection unit 3 uses the nine patterns in FIG. 6 to obtain an absolute difference value from Equations 11 to 19 in FIG. 9 (Step S11). Then, the smallest first minimum value and the second smallest second minimum value are obtained (step S12).

  Subsequently, it is determined whether or not the obtained first and second minimum values are close to each other. If they are not close (step S13: No), the interpolation function based on the pattern that has obtained the first minimum value is set as the optimum interpolation function. Confirm (step S14). If it is determined that the first and second minimum values are very small (step S13: Yes), the reference pixel and each pixel (b, e) at the center of the top field are calculated according to the equations 25 to 33 in FIG. The difference is obtained (step S15). Then, the interpolation function based on the pattern that minimizes the difference is determined as the optimum interpolation function (step S16).

  The second interpolation function selection unit 4 in FIG. 1 reads the interlaced scanning video signal data from the data storage unit 1 and displays the three types of patterns “pattern 1”, “pattern 2” and “pattern 3” shown in FIG. The optimum interpolation function is selected by using equations 11, 12, and 13 in FIG. 9 and equations 25, 26, and 27 in FIG. 19 (FIG. 22: step S3). The three types of patterns shown in FIG. 7 are a part of the nine types of patterns shown in FIG. The processing procedure by the second interpolation function selection means 4 is in accordance with the procedure by the first interpolation function means 3 described above, and the description is omitted.

  The pattern group of FIG. 7 used by the second interpolation function selection unit 4 corresponds to the first pixel pattern group in the present invention, that is, the pattern group associated with the fact that the interpolation target pixel is distorted. The second interpolation function selection unit 4 uses only a pattern indicating that the reference pixel is adjacent to the interpolation target pixel (x), such as “Pattern 1” to “Pattern 3” in FIG.

  The first selection unit 5 includes a first optimum interpolation function that is an output of the first interpolation function selection unit 3, a second optimum interpolation function that is an output of the second interpolation function selection unit 4, and distortion in the current pixel. If the current pixel is not distorted, the first optimum interpolation function from the first interpolation function selecting means 3 is output. If the current pixel is distorted, a signal indicating the second optimum interpolation function from the second interpolation function selection means 4 is output.

  As described above, when there is no distortion in the current pixel (FIG. 22: Step S4 · No), the optimum interpolation function obtained using the nine types of patterns (FIG. 6) is selected (Step S5). If there is distortion (FIG. 22: Step S4, Yes), an optimal interpolation function using three patterns (FIG. 7) included in the same nine patterns is selected (Step S6).

  The interpolation means 6 reads from the data storage means 1 a pixel in the vicinity of the pixel to be interpolated, that is, the reference pixel of the pattern corresponding to the optimum interpolation function obtained by the first selection means 5, and the value of the current pixel Is interpolated by calculating (FIG. 22: step S7). As a result, a video signal that has been subjected to progressive scanning conversion and from which distortion has been removed is output (step S8).

  Here, when selecting an optimal interpolation function, the use of a pattern (FIG. 6 / FIG. 7) depending on the presence or absence of distortion of the current pixel will be described using a specific example. In the following description, it is assumed that the value shown in FIG. 20B is obtained as the pixel value of the top field in the pixel array shown in FIG. Further, in order to compare the interpolation results based on the difference in the number of patterns to be used, interpolation with 9 patterns (FIG. 6) and interpolation with 3 patterns (FIG. 7) are performed on the current pixel x with distortion.

  FIG. 20C shows the calculation result of the absolute difference value based on the values in FIG. These calculations are performed using the equations shown in FIGS. 9 and 19 described above. From FIG. 20C, the optimum interpolation function obtained by using nine patterns “pattern 1” to “pattern 9” is “pattern 5” in which the absolute difference value is the minimum value “4”. Further, the optimum interpolation function based on the three patterns “Pattern 1” to “Pattern 3” is “Pattern 3” obtained the minimum value “60”.

  FIG. 21 shows the result of interpolation of the current pixel x using the 9 patterns and 3 patterns of optimal interpolation functions. FIG. 21A shows a pixel value array as a result of interpolation by nine patterns, and the entire gradation is shown in FIG. Also, FIGS. 21C and 21D show the results of interpolation using three patterns.

  Originally, when all nine patterns are used, a pixel (g, j) that is not adjacent to the current pixel x is also a reference pixel, as in “pattern 4” of FIG. However, when this method is applied to a current pixel x having a distortion, as shown in FIG. 21B, the connection between the current pixel x and a neighboring pixel becomes weak, and as a result, the distortion of the current pixel x is removed. It becomes difficult to be done. On the other hand, when only three patterns of three adjacent pixels are applied to the interpolation of the distorted pixel x, as shown in FIGS. 21 (c) and (d), the connection with the neighboring pixels becomes strong, Distortion is eliminated more naturally.

  Thus, distortion can be appropriately removed by applying only the three patterns in FIG. 7 to the current pixel x having distortion. Further, when there is no distortion in the current pixel x, it is possible to flexibly deal with a curve of an image by using all nine patterns including a refracted pattern such as “pattern 6” to “pattern 9” (FIG. 6). can do.

  According to the first embodiment of the present invention described above, the interpolating function is selected according to whether or not the interlaced video signal is distorted during the progressive scan conversion of the video signal. A video signal from which distortion has been removed can be obtained. Further, since the interpolation process is performed within one frame, the data handled in the interpolation process can be simplified, and as a result, the process can be speeded up.

  In the above embodiment, the pattern as shown in FIG. 6 is used as the pixel pattern used for the interpolation process. However, in implementing the present invention, the number of patterns and the pattern used depending on the presence / absence of distortion of the current pixel are used. Various changes can be made to the pixel arrangement. For example, as shown in FIG. 5, the pattern in which the reference pixels are “a” and “d”, the pattern in which the reference pixels are “g” and “i”, and the like are image portions that are refracted at an acute angle at the current pixel x. This is suitable for interpolation.

  In addition, as a pattern to be applied when the current pixel is distorted, a pattern in which three pixels are linearly connected as shown in FIG. 7 is used in this embodiment, but the reference pixel is a pixel (x) to be interpolated. As long as the patterns indicate that they are adjacent to each other, a pattern in which three pixels are refracted and connected, for example, as “pattern 6” in FIG. 6 may be used.

  Next, a second embodiment of the present invention will be described. FIG. 2 shows the configuration of the second embodiment. In the present embodiment, the data storage means 1, the distortion detection means 2, the first interpolation function selection means 3, the second interpolation function selection means 4, and the first function that function in the same manner as the first embodiment (FIG. 1) described above. 1 selection means 5 is provided.

  In this embodiment, in addition to these configurations, the frame delay means 7, the frame difference means 8, the fineness measurement means 9, the vertical line detection means 10, the rigid body approximation means 11, the significant pixel counting means 12, the control means 13, And the 2nd selection means 14 is provided. Among them, the frame delay means 7, the frame difference means 8, the fineness measurement means 9, the rigid body approximation means 11 and the significant pixel counting means 12 surrounded by a chain line in FIG. 2 correspond to the motion detection means in the present invention.

  The interpolation means 6 of this embodiment reads pixels in the vicinity of the pixel to be interpolated from the data storage means 1, and vertical line information 106 described later given from the vertical line detection means 10 indicates that the current pixel belongs to the vertical line. When it is shown that, the interpolation is performed using the vertical interpolation function ("pattern 2" in FIG. 6). When the current pixel does not belong to the vertical line, as described in the above-described embodiment, the interpolation function is selected according to the signal indicating the optimum interpolation function given from the first selection unit 5, and the distortion is removed. Output video signal.

  The frame delay means 7 delays the video signal read from the data storage means 1 by one frame time. The frame difference means 8 obtains a frame difference between the video signal from the data storage means 1 and the video signal from the frame delay means 7.

  The fineness measuring means 9 measures the fineness of the video signal read from the data storage means 1 and generates information indicating the degree of fineness. The fineness measuring means 9 will be described with reference to FIGS.

  In the determination of fineness, the degree of change of the image in the five pixels by the current pixel x and the two left and right pixels (a, b, c, d) around the current pixel x is measured. Specifically, as shown in Expression 20 in FIG. 14, a change in video between the five pixels is grasped by obtaining a difference between the maximum value and the minimum value among the five pixels. For example, the smaller the difference between the maximum value and the minimum value, the smaller the image change between the five pixels, and the larger the difference between the maximum value and the minimum value between the five pixels, the larger the image changes. I can see that. Then, the difference value between the obtained maximum value and minimum value is accumulated for one frame to obtain control information.

  Further, instead of the above method, the reference pixels used for measuring the fineness may be one pixel on both sides of the current pixel, and measurement may be performed with three pixels including the current pixel. As yet another method, the threshold value is determined based on a predetermined threshold value for the difference between the maximum value and the minimum value between five or three pixels. The number of pixels larger than the threshold can be accumulated for one frame, and this value can be used as one piece of control information.

  The vertical line detection means 10 will be described with reference to FIGS. 15 and 16. The detection of the vertical line is performed by detecting the pixels b and c above and below the current pixel x shown in “Combination 1” in FIG. 15, the top field pixel d located three pixels below the current pixel x, and the “combination” in FIG. Whether or not the current pixel x belongs to the vertical line is determined based on the pixels b and c above and below the current pixel x and the top field pixel a located on three pixels of the current pixel x.

  Determination by “combination 1” in FIG. 15 uses Expression 22 and Expression 23 in FIG. Also, the determination by “combination 2” uses Expression 21 and Expression 22 in FIG. In any one or both of “combination 1” and “combination 2”, if all of the absolute differences between the pixels are less than the threshold value, it is determined that the current pixel belongs to the vertical line.

  The result of the determination by the vertical line detection means 10 is used for selection of an interpolation function by the interpolation means 6 as vertical line information 106 (FIG. 2) for each pixel. In addition, the number of pixels determined to belong to the vertical line is counted for one frame, and the value is used as control information.

  The rigid body approximating unit 11 performs rigid body approximation of the motion region based on the frame difference information from the frame difference unit 8, and generates information indicating the motion region. Details of the rigid body approximating means 11 will be described with reference to FIGS. 10, 11 and 12. FIG.

  As shown in FIG. 10, the rigid body approximating unit 11 includes a binarizing unit 1101, a horizontal isolated data removing unit 1102, a vertical isolated data removing unit 1103, a frame delay unit 1104, and a second vertical isolated unit. Data removal means 1105.

  The binarizing unit 1101 binarizes the frame difference value given as an input of the rigid body approximating unit 11 by threshold determination. The binarization means 1101 outputs “1” when the frame difference signal is less than the binarization threshold, and outputs “0” when the frame difference signal is equal to or greater than the binarization threshold.

  As shown in FIG. 11, the horizontal direction isolated data removal unit 1102 removes the isolated data by correcting the value of the target data based on the values on both sides of the pixel from which the isolated data is to be removed. Specifically, when the target data is “0” and both sides are “1”, the target data is corrected to “1”. If the target data is “1” and both neighbors are “0”, the target data is corrected to “0”. In other cases, that is, when all the three pixels have the same value, or when the target data and either of the adjacent data have the same value, the value of the target data is maintained as it is. By this process, the isolated state of the data isolated in the horizontal pixel array is eliminated.

  The vertical direction isolated data removal unit 1103 performs vertical direction isolated data removal on the binarized data from which the horizontal direction isolated data has been removed using the isolated data removal patterns “a” to “h” of FIG. . To remove the isolated data in the vertical direction, the pattern “a” is used first, and then the pattern “b” is used. Then, the process sequentially proceeds from the pattern “c” to the pattern “h”.

  In each pattern, the isolated data removal process is executed when the target central data is “1”, and the isolated data removal process is not performed when the target data is “0”. For example, in the case of the illustrated pattern “a”, the target data is “1” and the upper and lower reference data is “0”. In this case, the target data is corrected from “1” to “0”. In the process of the pattern “a”, if the target data is “1” and one or both of the upper and lower reference data are “1”, the process of the pattern “b” is performed without correcting the value of the target data. Migrate to

  In the processing by the pattern “b”, the target data is corrected with reference to the data values of the two samples and the data of the two samples below the central target data. In the illustrated example, the target data is “1”, the reference data on the two samples and the reference data on the two samples are both “0”. In this case, the target data is corrected from “1” to “0”. To do. If either one or both of the reference data two samples above and below the target data “1” is “1”, the process proceeds to the pattern “c”.

  Thereafter, similarly, when the target data is “1”, the process proceeds to the patterns “d”, “e”, “f”, “g”, “h”, and the value of the reference data is “0” both above and below Only, the target data is corrected from “1” to “0”.

  The frame delay means 1104 in FIG. 10 delays binary data from which isolated data in the horizontal and vertical directions are removed by one frame time.

  The second vertical direction isolated data removing unit 1105 includes binary data from which the horizontal and vertical isolated data have been removed, and binary data from which the isolated data has been removed by the frame delay unit 1104 by one frame time. The isolated data in the vertical direction is removed based on the logical OR.

  The second vertical direction isolated data removal unit 1105 uses the four patterns “a”, “b”, “d”, and “e” in FIG. 12, and is similar to the method of the vertical direction isolated data removal unit 1103 described above. The isolated data is removed. The binarized data from which the isolated data has been removed by the second vertical isolated data removing unit 1105 is a signal indicating a motion region as an output of the rigid body approximating unit 11. Note that a pixel whose output of the rigid body approximating means 11 is “1” is a pixel in a still part of the image, and a pixel whose output of the rigid body approximating means 11 is “0” is a moving pixel.

  The significant pixel counting unit 12 performs threshold determination on the frame difference information obtained by the frame difference unit 8 based on a predetermined frame difference threshold, and counts the number of pixels having a frame difference value larger than the frame difference threshold. The value counted for each frame is output as the number of significant pixels.

  The control unit 13 includes a frame difference value obtained by the frame difference unit 8, information indicating the degree of fineness of the video generated by the fineness measurement unit 9, and vertical line information 106 detected by the vertical line detection unit 10. And information indicating the motion region generated by the rigid body approximating unit 11 and the number of significant pixels in the output frame of the significant pixel counting unit 12 are obtained. Then, a control signal indicating whether to select the output of the interpolation unit 6 or to select the direct output from the data storage unit 1 is generated by condition determination described later based on the information.

  The second selection unit 14 selects and outputs either the output of the data storage unit 1 or the output of the interpolation unit 6 according to the control signal generated by the control unit 13.

  The control means 13 controls the second selection means 14 for each pixel according to the flowchart shown in FIG. First, a first determination (step S21) is performed to determine whether the number of significant pixels in the screen is equal to or less than a first threshold value. If this condition is satisfied, a signal for selecting the output of the data storage means 1 is output (step S25).

  If the determination condition is not satisfied in the first determination, the process proceeds to the second determination. In the second determination (step S22), the number of pixels belonging to the vertical line in the screen is greater than or equal to the second threshold, the degree of fineness is less than the fourth threshold, and the upper and lower pixels When the condition that the frame difference value is less than the sixth threshold value and the output of the rigid body approximation means 11 is “1” (still pixel) is satisfied, a signal for selecting the output of the data storage means 1 is output. (Step S25).

  If the second determination condition is not satisfied, the process proceeds to the third determination. In the third determination (step S23), the number of pixels belonging to the vertical line in the screen is less than the third threshold, the degree of fineness is greater than or equal to the fifth threshold, and the rigid body approximation means 11 When the condition for the output of “1” is satisfied, a signal for selecting the output of the data storage means 1 is output (step S25).

  If the third determination condition is not satisfied, a signal for selecting the output of the interpolation means 6 is output (step S24). Note that the first threshold value, the second threshold value, the third threshold value, the fourth threshold value, the fifth threshold value, and the sixth threshold value used in the control unit 13 are obtained in advance by statistical data collection. Use the optimal value.

  According to the second embodiment described above, in the progressive scan conversion of the interlaced video signal, the resolution that occurs due to the inter-field motion is eliminated by performing the interpolation while removing the distortion caused by the inter-field motion. It is to be able to obtain a video signal that is kept high. This is because motion information between frames is used to interpolate the motion part to eliminate distortion caused by image shift, and the still image part without distortion outputs the original video signal as it is. It is.

  The present invention is suitable for various video processes that perform progressive scan conversion. For example, the present invention can be applied to a progressive scan conversion process as a preprocess of a progressive scan video encoding device. The present invention can also be applied to applications such as progressive scan conversion for displaying interlaced video signals on a progressive scan display.

It is a block diagram which shows the structure of the 1st Embodiment of this invention. It is a block diagram which shows the structure of the 2nd Embodiment of this invention. It is explanatory drawing regarding the memory | storage form of the data storage means of embodiment. It is explanatory drawing of the comb-shaped distortion detected by the distortion detection means of embodiment. It is explanatory drawing of the basic pixel arrangement | sequence in embodiment. It is explanatory drawing of the pixel arrangement | sequence of nine types of patterns in embodiment. It is explanatory drawing of the pixel arrangement | sequence of 3 types of patterns in embodiment. It is explanatory drawing of the calculation formula which the distortion detection means of embodiment uses. It is explanatory drawing of the type | formula which calculates | requires the difference absolute value used for selection of the 1st optimal interpolation function of embodiment. It is a block diagram which shows the structure of the rigid body approximation means of embodiment. It is explanatory drawing regarding the process of the horizontal direction isolated data removal means of embodiment. It is explanatory drawing regarding the process of the vertical direction isolated data removal means and vertical direction isolated data removal means of embodiment. It is explanatory drawing regarding the process of the fineness measurement means of embodiment. It is explanatory drawing of the calculation formula which the fineness measurement means of embodiment uses. It is explanatory drawing regarding the process of the vertical line detection means of embodiment. It is explanatory drawing of the calculation formula which the vertical line detection means of embodiment uses. It is a flowchart which shows the process sequence of the control means of embodiment. It is explanatory drawing regarding the selection process of the interpolation function selection means of embodiment. It is explanatory drawing regarding the type | formula for calculating | requiring the 2nd optimal interpolation function in embodiment. It is explanatory drawing regarding selection of the optimal interpolation function in embodiment. It is explanatory drawing regarding the difference in the optimal interpolation function by 9 patterns and 3 patterns in embodiment. It is a flowchart which shows the process sequence of 1st Embodiment. It is a flowchart which shows the process sequence of the interpolation function selection means of embodiment.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Data storage device 2 Distortion detection means 3 1st interpolation function selection means 4 2nd interpolation function selection means 5 1st selection means 6 Interpolation means 7 Frame delay means 8 Frame difference means 9 Fineness measurement means 10 Vertical line detection means 11 Rigid body approximation means 12 Significant pixel counting means 13 Control means 14 Second selection means 1101 Binarization means 1102 Horizontal direction isolated data removal means 1103 Vertical direction isolated data removal means 1104 Frame delay means 1105 Second vertical direction isolated data removal means

Claims (38)

Data storage means for storing a video signal of interlaced scanning for each frame of two fields;
Distortion detecting means for detecting the presence or absence of distortion with respect to the reference field as the other field for each pixel of the interpolation field to be interpolated in the video signal of the data storage means;
A pair of interpolation functions associated with the presence / absence of the distortion from a plurality of interpolation functions corresponding to a plurality of pixel patterns defining an array of each pixel of the interpolation field and a reference pixel of the reference field to be applied to the interpolation of each pixel. Interpolation function selection means for selecting for each pixel of the interpolation field;
Selecting means for selecting an interpolation function to be applied to each pixel of the interpolation field from a pair of interpolation functions selected for the pixel according to the presence or absence of distortion of the pixel;
An image processing apparatus comprising: interpolation means for obtaining a value of each pixel in the interpolation field by an interpolation function selected for the pixel. Motion detection means for detecting information about the motion of the video represented by the difference between the interpolation field and the reference field;
Vertical line detection means for determining whether each pixel of the interpolation field is included in the vertical line of the video;
Second selection means for outputting either the video signal of the data storage means and the video signal interpolated by the interpolation means;
2. The video processing apparatus according to claim 1, further comprising a control unit that determines a video signal to be output from the second selection unit based on detection results of the motion detection unit and the vertical line detection unit.   The motion detection means includes a frame delay means for delaying a video signal read from the data storage means by one frame time, a video signal read from the data storage means, and a video signal delayed by the frame delay means. Frame difference means for obtaining frame difference data, fineness measuring means for measuring the degree of change in pixel value for a pixel group in a video signal of the data storage means, and a rigid body for obtaining a motion region of the frame based on the frame difference data 3. The video processing apparatus according to claim 2, further comprising approximation means and significant pixel counting means for counting significant pixels of the frame based on the frame difference data.   The distortion detection unit performs a difference calculation between the current pixel to be processed in the interpolation field and two pixels in the two scanning lines of the reference field sandwiching the current pixel for each pixel pattern of the plurality of pixel patterns, 3. The video processing apparatus according to claim 1, wherein when the result of the difference calculation exceeds a reference value in all pixel patterns, it is determined that the current pixel has the distortion.   The interpolation function selection means includes a first pixel pattern group associated with the distortion among the plurality of pixel patterns and a second associated with no distortion for each pixel of the interpolation field. A difference value between two reference pixels in each pixel pattern is obtained for each of the pixel pattern groups, and the first minimum value that is the smallest difference value in each pixel pattern group and the next of the first minimum value A second minimum value that is a very small difference value, and if the difference between the obtained first and second minimum values exceeds a reference value, the pixel pattern that has obtained the first minimum value is the current pixel. The video processing apparatus according to claim 1, wherein the video processing apparatus is applied to an interpolation function of   When the difference between the first and second minimum values is lower than the reference value, the interpolation function selection unit sandwiches the current pixel of the interpolation field between the reference pixel and the reference field for each pixel pattern in the pixel pattern group. 6. The video processing apparatus according to claim 5, wherein a difference value with respect to one pixel is obtained, and a pixel pattern having the smallest obtained difference value is applied to the interpolation function of the current pixel.   The fineness determination means obtains a difference value between the current pixel of the interpolation field and a plurality of pixels in the vicinity of the current pixel in the scan line of the current pixel, and determines the difference between the maximum value and the minimum value of the obtained difference value. 4. The video processing apparatus according to claim 3, wherein a degree of change of the pixel value is obtained based on the video value.   In the reference field, the vertical line detection means has a difference between two pixels sandwiching the current pixel of the interpolation field in the reference direction in a vertical direction lower than a reference value, and two pixels in the vertical direction from each of the two pixels and the two pixels. 4. The method according to claim 3, wherein if at least one of the differences between the reference field and the other two pixels located below the reference value is below a reference value, it is determined that the current pixel is included in a vertical line of the image. Video processing device. The rigid body approximating means includes: binarizing means for binarizing the frame difference data; lateral isolated data removing means for removing data isolated in the scanning line direction from the binarized frame difference data; Vertically isolated data removing means for removing data isolated in the vertical direction from the converted frame difference data, and frame delay means for delaying the frame difference data from which data isolated in the scanning line direction and the vertical direction are removed by one frame time And calculating the logical sum of the frame difference data from which the data isolated in the scanning line direction and the vertical direction is removed and the frame difference data delayed by one frame time, and obtaining the data isolated in the vertical direction in the frame difference data composed of the logical sum. Second vertical isolated data removing means for removing,
4. The video processing apparatus according to claim 3, wherein the frame difference data that has undergone the processing of the second vertical isolated data removing unit is output as information indicating a motion region of the frame.   The horizontal direction isolated data removing means has a value that two pixels adjacent to a reference pixel serving as a reference for removal of data isolated in the scanning line direction have the same value, and a value indicated by the two pixels adjacent to the both sides. 10. The video processing apparatus according to claim 9, wherein when the value of the reference pixel is different from the value of the reference pixel, the value of the reference pixel is corrected to a value indicated by the two adjacent pixels.   The vertical direction isolated data removing means has two pixels positioned at equal intervals in the vertical direction with respect to a reference pixel serving as a reference for removal of data isolated in the vertical direction and showing the same value. In addition, when a value indicated by the two pixels in the vertical relationship is different from a value of the reference pixel, the value of the reference pixel is corrected to a value indicated by the two pixels in the vertical relationship. Item 10. The video processing device according to Item 9.   The second vertical isolated data removing means has two pixels located at equal intervals in the vertical direction with respect to a reference pixel serving as a reference for removal of data isolated in the vertical direction, having the same logical sum value. When the logical sum value indicated by the two pixels positioned above and below is different from the value of the reference pixel, the value of the reference pixel is corrected to the logical sum value indicated by the two pixels positioned above and below. The video processing apparatus according to claim 9.   The control means determines whether or not the counting result of the significant pixel counting means is below a reference value, and when the counting result is below the reference value, the second selection means outputs the video signal of the data storage means. The video processing apparatus according to claim 9, wherein the video processing apparatus is output.   When the determination result is negative, the control means determines whether the detection result of the vertical line detection means exceeds a reference value, whether the measurement result of the fineness measurement means is lower than a reference value, and the rigid body When the second determination for determining whether or not the motion area obtained by the approximating means indicates a stationary area is executed, and when all the conditions for the second determination are satisfied, the data selecting means is selected by the second selecting means. 14. The video processing apparatus according to claim 13, wherein the video signal is output.   When the results of the second determination are all negative, the control means determines whether the detection result of the vertical line detection means is below a reference value different from the reference value, and the measurement result by the fineness measurement means. A third determination is performed to determine whether or not the reference value exceeds a reference value different from the reference value, and whether or not the motion region obtained by the rigid body approximation means indicates a stationary region, 15. The video processing apparatus according to claim 14, wherein when all of the conditions are satisfied, the video signal of the data storage unit is output by the second selection unit.   16. The video processing apparatus according to claim 15, wherein the control means causes the second selection means to output the video signal interpolated by the interpolation means when all the results of the third determination are negative. .   The interpolation means obtains the value of the current pixel by an interpolation function of a pixel pattern composed of two reference pixels sandwiching the current pixel in the vertical direction in the reference field when the current pixel of the interpolation field belongs to a vertical line. The video processing apparatus according to claim 2, wherein:   6. The video processing apparatus according to claim 5, wherein the interpolation function selection unit uses only a pixel pattern indicating that a reference pixel is adjacent to a pixel to be interpolated as the first pixel pattern group.   The first pixel pattern group includes a pixel pattern having two reference pixels adjacent to the current pixel in a vertical direction or an oblique direction with the current pixel of the interpolation pattern sandwiched in the reference pattern, and the second pixel pattern The group includes each pixel pattern of the first pixel pattern group, and a pixel pattern that includes two reference pixels that sandwich the current pixel and are adjacent to the current pixel in a vertical direction and an oblique direction. The video processing apparatus according to claim 18. An image processing apparatus comprising data storage means for storing an interlaced scanning image signal for each frame composed of two fields,
A distortion detection step of detecting the presence or absence of distortion with respect to a reference field as the other field for each pixel of the interpolation field to be interpolated in the video signal of the data storage means;
A pair of interpolation functions associated with the presence / absence of the distortion from a plurality of interpolation functions corresponding to a plurality of pixel patterns defining an array of each pixel of the interpolation field and a reference pixel of the reference field to be applied to the interpolation of each pixel. An interpolation function selection step for selecting for each pixel of the interpolation field;
A selection step of selecting an interpolation function to be applied to each pixel of the interpolation field according to the presence or absence of distortion of the pixel from a pair of interpolation functions selected for the pixel;
An image processing method comprising: performing an interpolation step for obtaining a value of each pixel in an interpolation field by an interpolation function selected for the pixel. The video processing device further includes:
A motion detection step of detecting information about the motion of the video represented by the difference between the interpolation field and the reference field;
A vertical line detection step for determining whether each pixel of the interpolation field is included in the vertical line of the video;
A second selection step of outputting either the video signal of the data storage means and the video signal interpolated by the interpolation step;
21. The video processing method according to claim 20, further comprising a control step of determining a video signal to be output in the second selection step based on detection results in the motion detection step and the vertical line detection step. In the motion detection step, the video processing device,
A frame delay step for delaying the video signal read from the data storage means by one frame time, and a frame for obtaining frame difference data between the video signal read from the data storage means and the video signal delayed by the frame delay step A difference step; a fineness measuring step for measuring a degree of change of a pixel value for a pixel group in a video signal of the data storage means; a rigid body approximation step for obtaining a motion region of the frame based on the frame difference data; The video processing method according to claim 21, wherein a significant pixel counting step of counting significant pixels of the frame based on the difference data is executed. In the distortion detection step, the video processing device,
The difference calculation between the current pixel to be processed in the interpolation field and the two pixels in the two scanning lines of the reference field sandwiching the current pixel is executed for each pixel pattern of the plurality of pixel patterns, and all the results of the difference calculation are obtained. The video processing method according to claim 20 or 21, wherein if the pixel pattern exceeds a reference value, it is determined that the current pixel has the distortion. In the video processing device, in the interpolation function selection step,
For each pixel in the interpolation field, each of the first pixel pattern group associated with the distortion and the second pixel pattern group associated with no distortion among the plurality of pixel patterns. A difference value between two reference pixels in each pixel pattern is obtained, and a first minimum value that is the smallest difference value in each pixel pattern group and a second difference value that is the next smallest difference value after the first minimum value. A minimum value is obtained, and if the difference between the obtained first and second minimum values exceeds a reference value, the pixel pattern obtained from the first minimum value is applied to the interpolation function of the current pixel. The video processing method according to claim 20 or 21. In the video processing device, in the interpolation function selection step,
When the difference between the first and second minimum values is lower than the reference value, for each pixel pattern in the pixel pattern group, the difference value between the reference pixel and two pixels sandwiching the current pixel of the interpolation field in the reference field is calculated. 25. The video processing method according to claim 24, wherein the obtained pixel pattern having the smallest difference value is applied to the interpolation function of the current pixel. In the video processing device, in the fineness determination step,
The difference value between the current pixel in the interpolation field and a plurality of pixels near the current pixel in the scan line of the current pixel is obtained, and the degree of change in the pixel value based on the difference between the maximum value and the minimum value of the obtained difference value 23. The video processing method according to claim 22, wherein the video processing method is obtained. In the video line detection step, the vertical line detection step,
In the reference field, the difference between the two pixels sandwiching the current pixel of the interpolation field in the vertical direction is less than the reference value, and the reference field of the reference field is located through the two pixels and two pixels in the vertical direction from each of the two pixels. 23. The video processing method according to claim 22, wherein when at least one of the differences from the other two pixels is less than a reference value, it is determined that the current pixel is included in a vertical line of the video. In the rigid body approximating step, the video processing device,
A binarization step for binarizing the frame difference data, a horizontal direction isolated data removal step for removing data isolated in the scanning line direction from the binarized frame difference data, and binarized frame difference data A vertical isolated data removal step for removing data isolated in the vertical direction, a frame delay step for delaying the frame difference data from which data isolated in the scan line direction and the vertical direction is removed by one frame time, a scan line direction and The second vertical length is obtained by calculating the logical sum of the frame difference data from which the data isolated in the vertical direction is removed and the frame difference data delayed by one frame time, and removing the data isolated in the vertical direction in the frame difference data composed of the logical sum. Performing a directional orphaned data removal step,
23. The video processing method according to claim 22, wherein the frame difference data having undergone the second vertical direction isolated data removal step is output as information indicating a motion area of the frame. In the video processing device, in the horizontal direction isolated data removal step,
Two pixels adjacent to the reference pixel serving as a reference for removal of data isolated in the scan line direction show the same value, and the value indicated by the two adjacent pixels is different from the value of the reference pixel. 29. The video processing method according to claim 28, wherein the value of the reference pixel is corrected to a value indicated by the two adjacent pixels. In the video processing apparatus, in the vertical direction isolated data removal step,
Two pixels that are positioned at equal intervals in the vertical direction with respect to a reference pixel that serves as a reference for the removal of data isolated in the vertical direction and have an upper and lower relationship exhibit the same value, and the two pixels that have the upper and lower relationship 29. The video processing method according to claim 28, wherein when a value indicated by a pixel is different from a value of the reference pixel, the value of the reference pixel is corrected to a value indicated by the two pixels in the vertical relationship. In the second vertical direction isolated data removal step, the video processing device,
Two pixels positioned at equal intervals in the vertical direction with respect to a reference pixel serving as a reference for removing data isolated in the vertical direction exhibit the same logical sum value, and the two pixels positioned above and below 29. The video processing according to claim 28, wherein when the logical sum value indicated by is different from the value of the reference pixel, the value of the reference pixel is corrected to a logical sum value indicated by the two pixels located above and below. Method. In the control step, the video processing device,
It is determined whether or not the counting result in the significant pixel counting step is lower than a reference value. If the counting result is lower than the reference value, the video signal of the data storage means is output in the second selection step. The video processing method according to claim 28. In the control step, the video processing device,
If the determination result is NO, whether the detection result in the vertical line detection step exceeds a reference value, whether the measurement result in the fineness measurement step is less than a reference value, and the rigid body approximation step When the second determination for determining whether or not the obtained motion region indicates a still region is executed, and all of the conditions for the second determination are satisfied, the image of the data storage means in the second selection step The video processing method according to claim 32, wherein a signal is output. In the control step, the video processing device,
If all of the results of the second determination are NO, whether the detection result in the vertical line detection step is below a reference value different from the reference value, and the measurement result in the fineness measurement step are the reference A third determination is performed to determine whether or not a reference value separate from the value is exceeded, and whether or not the motion region obtained by the rigid body approximation step indicates a stationary region, and all the conditions for the third determination are 34. The video processing method according to claim 33, wherein if satisfied, the video signal of the data storage means is output in the second selection step. In the control step, the video processing device,
35. The video processing method according to claim 34, wherein when all the results of the third determination are negative, the video signal interpolated in the interpolation step is output in the second selection step. The video processing device, in the interpolation step,
The value of the current pixel is obtained by an interpolation function of a pixel pattern composed of two reference pixels sandwiching the current pixel in the vertical direction in the reference field when the current pixel of the interpolation field belongs to a vertical line. The video processing method according to claim 21. In the video processing device, in the interpolation function selection step,
25. The video processing method according to claim 24, wherein only a pixel pattern indicating that a reference pixel is adjacent to a pixel to be interpolated is used as the first pixel pattern group.   The first pixel pattern group includes a pixel pattern having two reference pixels adjacent to the current pixel in a vertical direction or an oblique direction with the current pixel of the interpolation pattern sandwiched in the reference pattern, and the second pixel pattern The group includes each pixel pattern of the first pixel pattern group, and a pixel pattern that includes two reference pixels that sandwich the current pixel and are adjacent to the current pixel in a vertical direction and an oblique direction. The video processing method according to claim 37.