WO2012053978A1 - Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels - Google Patents

Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels Download PDF

Info

Publication number
WO2012053978A1
WO2012053978A1 PCT/SG2011/000364 SG2011000364W WO2012053978A1 WO 2012053978 A1 WO2012053978 A1 WO 2012053978A1 SG 2011000364 W SG2011000364 W SG 2011000364W WO 2012053978 A1 WO2012053978 A1 WO 2012053978A1
Authority
WO
WIPO (PCT)
Prior art keywords
pixel
pair
image
deinterlacing
pixels
Prior art date
Application number
PCT/SG2011/000364
Other languages
English (en)
Inventor
Zaw Min Oo
Kok Seng Aw
Kwong Huang Goh
Jo Yew Tham
Wei Siong Lee
Original Assignee
Agency For Science, Technology And Research
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Agency For Science, Technology And Research filed Critical Agency For Science, Technology And Research
Priority to SG2013019054A priority Critical patent/SG188546A1/en
Priority to US13/879,686 priority patent/US9171370B2/en
Publication of WO2012053978A1 publication Critical patent/WO2012053978A1/fr

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/144—Movement detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
    • H04N7/012—Conversion between an interlaced and a progressive signal

Definitions

  • Various embodiments relate to a method for deinterlacing, an apparatus for deinterlacing and a computer program product for deinterlacing.
  • the method, apparatus and computer program product relate to deinterlacing an image having a plurality of pixels.
  • Interlace video cameras capture scenes as fields instead of frames.
  • a field is half the vertical width (i.e. height) of a frame.
  • a frame is composed of two fields.
  • a frame of an interlaced video sequence is formed by combining alternate lines of a pair of fields.
  • Interlacing was originally designed for cathode ray tube (CRT) displays in which phosphor glows to illuminate the screen.
  • CRT cathode ray tube
  • One reason why interlacing works well on CRT displays is that human beings notice details flickering less than mass flickering.
  • CRT displays are being replaced by displays having newer technologies, such as, for example, liquid crystal displays (LCD) and plasma displays. Both LCD and plasma displays use progressive scanning or frames as inputs, rather than fields.
  • LCD liquid crystal displays
  • plasma displays use progressive scanning or frames as inputs, rather than fields.
  • FIG. 1(a) illustrates some signage, wherein interlacing artifacts can be clearly seen at the edges between the pale lettering of 'FOST' and the darker background behind.
  • Figure 1(b) illustrates a person pointing, wherein interlacing artifacts can be clearly seen at the edges between the pale post to the right of the person and the darker background behind.
  • the interlacing artifacts can reduce the coding efficiency of video encoders.
  • De-interlacing is a mechanism to remove or reduce as many interlacing artifacts as possible when two fields are combined into a frame. As interlacing artifacts are removed, visual quality and coding efficiency of the frame may be improved.
  • Various embodiments provide, a method for deinterlacing an image having a plurality of pixels, the method comprising: calculating a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and deinterlacing the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • an apparatus for deinterlacing an image having a plurality of pixels comprising: at least one processor; and at least one memory including computer program code; the at least one memory and the computer program code being configured, with the at least one processor, to cause the apparatus to perform at least the following: calculate a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and deinterlace the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • a computer program product for deinterlacing an image having a plurality of pixels
  • the computer program product comprising at least one computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising: program code for calculating a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and program code for deinterlacing the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • Figure 1 illustrates two examples of interlacing artifacts
  • Figure 2 illustrates an image according to an embodiment
  • Figure 3 illustrates two images, (a) one with interlacing artifacts and (b) the other without interlacing artifacts;
  • Figure 4 illustrates a method for deinterlacing according to an embodiment
  • Figure 5 illustrates two frames of a video sequence according to an embodiment
  • Figure 6 illustrates one exemplary sequence for considering pixels of an image
  • Figure 7 illustrates a method for deinterlacing according to another embodiment
  • Figures 8 to 14 illustrate simulation results relating to a first set of experiments
  • Figure 15 illustrates simulation results relating to a second set of experiments.
  • Figure 16 illustrates a computing device according to an embodiment. Detailed Description
  • Various embodiments provide, a method for deinterlacing an image having a plurality of pixels, the method comprising: calculating a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and deinterlacing the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • the first pixel is deinterlaced only if both differences corresponding to the pixel pair exceed the predefined threshold.
  • the first pixel is deinterlaced only if both differences corresponding to the pixel pair exceed the predefined threshold and have matching signs.
  • the first pixel is deinterlaced only if both differences corresponding to another pixel pair exceed the predefined threshold and have matching signs.
  • deinterlacing comprises setting the first pixel to equal an average value, the average value being calculated using both differences which exceed the predefined threshold and have matching signs.
  • the at least one pixel pair comprises a plurality of pixel pairs, the plurality of pixel pairs having a predefined order, and the method being performed in respect of each pixel pair sequentially and in accordance with the predefined order.
  • the method is performed in respect of a pixel pair which is vertically aligned with the first pixel before the method is performed in respect of a pixel pair which is diagonally aligned with the first pixel.
  • the image is a frame of a video and, if deinterlacing is not performed, the method further comprises: detecting motion in respect of the first pixel by comparing the first pixel with a corresponding pixel of at least one other frame of the video; and deinterlacing the first pixel only if motion is detected in respect of the first pixel.
  • motion is detected if a difference between the first pixel and the corresponding pixel of the at least one other frame exceeds a second predefined threshold.
  • at least one pixel of a pixel pair is positioned adjacent to the first pixel.
  • the first pixel and each pixel of a pixel pair are aligned.
  • the first pixel and each pixel of the pixel pair are vertically aligned.
  • an apparatus for deinterlacing an image having a plurality of pixels comprising: at least one processor; and at least one memory including computer program code; the at least one memory and the computer program code being configured, with the at least one processor, to cause the apparatus to perform at least the following: calculate a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and deinterlace the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • a computer program product for deinterlacing an image having a plurality of pixels
  • the computer program product comprising at least one computer-readable storage medium having computer-executable program code instructions stored therein, the computer-executable program code instructions comprising: program code for calculating a difference between a first pixel of the image and each pixel of at least one pixel pair, each pixel pair comprising one pixel being positioned above the first pixel and another pixel being positioned below the first pixel; and program code for deinterlacing the first pixel only if at least one difference corresponding to a pixel pair exceeds a predefined threshold.
  • Figure 2 illustrates an embodiment of an image 2 including a plurality of pixels arranged in a grid.
  • the image 2 includes a 5x5 grid of pixels, wherein each pixel is referenced using a coordinate system.
  • the pixel in the centre of the grid is referenced as pixel (x, y); the pixel in the top right corner of the grid is referenced as pixel (x+2, y-2); the pixel in the bottom left comer of the grid is referenced as pixel (x-2, y+2); and so on.
  • the image 2 includes an interlaced image made up of a pair of interlacing fields. Accordingly, the following rows may form part of an even interlace field: y-2, y and y+2; whereas the following rows may form part of an odd interlace field: y-1 and y+1. It is to be understood that the image 2 will be used in the forthcoming description to illustrate the operation of various embodiments.
  • Figure 3 illustrates the presence and absence of interlacing artifacts.
  • Figure 3(a) illustrates the presence of interlacing artifacts.
  • the even interlace field may have been captured first, whereas the odd interlace field may have been captured second.
  • an object e.g. a person
  • the pixels of the even field are clear.
  • the pixels of the odd field are shaded indicating the presence of the object. Therefore, when the even and odd fields are combined into an image, the resultant image includes a striped appearance.
  • Figure 3(b) illustrates the absence of interlacing artifacts. Specifically, in the case of Figure 3(b), the above-mentioned object does not enter the camera's field of view and, therefore, both the even and odd fields do not capture the object. Accordingly, when the even and odd fields are combined to form an image, no interlacing artifacts are present.
  • interlacing artifacts may occur if the subject of the image is moving or if the camera capturing the image is moving. Accordingly, the presence of interlacing artifacts is related to the presence of motion.
  • Figure 4 illustrates a method of interlacing according to an embodiment.
  • the method of Figure 4 will be described in conjunction with the image 2 of Figure 2.
  • the flow diagram of Figure 4 starts at 98.
  • a first pixel of the image 2 is obtained.
  • the first pixel is pixel (x,y).
  • a pixel pair is obtained.
  • the pixel pair includes a pair of pixels of the image 2.
  • each pixel of the pixel pair is adjacent the first pixel. Accordingly, each pixel of the pixel pair shares an edge and/or a corner with the first pixel.
  • pixels are considered adjacent to the first pixel: (x-1, y-1), (x, y-1), (x+1, y-1), (x-1, y), (x+1, y), (x-1, y+1), (x, y+1) and (x+1 , y+1).
  • a first pixel of the pixel pair is positioned above the first pixel, whereas a second pixel of the pixel pair is positioned below the first pixel.
  • the first pixel may be any one of the following pixels: (x-1, y-1), (x, y-1) and (x+1, y-1), whereas the second pixel may be any one of the following pixels: (x-1, y+1), (x, y+1) and (x+1, y+1).
  • the pixel pair obtained at 102 consists of pixel (x, y-1) and pixel (x, y+1).
  • the difference between the value of the first pixel and the value of each pixel of the pixel pair is calculated.
  • the value of a pixel indicates a brightness and/or a color of a pixel.
  • the image 2 may be a grayscale image and each pixel may have a value between 0 and 255, wherein 0 indicates black, 255 indicates white, and the numbers in-between indicate different shades of grey.
  • the image 2 may be a color image and each pixel may include a vector of values, wherein each of the values represents an intensity of a different color component of the pixel.
  • the vector may comprise three values which represent the colors red, green and blue.
  • the vector may include four values which represent the colors cyan, magenta, yellow and black.
  • a difference vector comprising a plurality of difference values may be calculated.
  • a single difference value may be calculated by combining the vector values. It is to be understood that in the following description, reference will be made to a difference between a pixel under consideration and a pixel of a pixel pair; however, this difference may be a single value or a vector of values.
  • the differences calculated in 104 exceed a threshold and have matching signs. Specifically, the difference relating to the first pixel of the pixel pair is compared to a predefined threshold. Either in series or in parallel, the difference relating to the second pixel of the pixel pair is compared to the predefined threshold. If the two differences exceed the predefined threshold, the sign of each difference is compared. For example, two pixels of a pixel pair may be associated with the same difference value. Either the pixels of the pixel pair could both be different from the first pixel in the same way or they could be different from the first pixel in different ways.
  • the pixels of the pixel pair may be the same color or one may be lighter than the first pixel by the same amount that the other is darker than the first pixel.
  • interlacing artifacts are detected based on whether or not both pixels of the pixel pair have the same sign, i.e. are different from the first pixel in the same or a similar way. For example, if the pixels of the pixel pair are associated with a difference exceeding the threshold but different signs, this may indicate a gradient of shading of an object rather than an interlacing artifact. Conversely if the pixels of the pixel pair are associated with a difference exceeding the threshold and the same sign, this may indicate an interlacing artifact.
  • the first pixel is deinterlaced.
  • the first pixel is deinterlaced using the pixels of the pixel pair.
  • the first pixel is deinterlaced by taking an average of the pixels of the pixel pair, for example, by summing the pixels of the pixel pair and dividing the result by two.
  • the first pixel may be deinterlaced by taking an average of the pixels of the pixel pair and the first pixel.
  • the first pixel may be deinterlaced by summing each pixel of the pixel pair with twice the value of the first pixel and then dividing the result by four.
  • a weighted average may be applied.
  • one or more of the above-mentioned techniques may be applied and an average taken of the result. After deinterlacing processing flows to 112 which will be described below. It is to be understood that the multiplications and/or divisions used in determining average values may be left and right shifted to make implementation more efficient computationally.
  • the first pixel is analyzed to identify motion in respect of the first pixel. Although interlacing artifacts were not detected in 106, they may still be present.
  • the pixel pair obtained in 102 may only detect a certain type of artifact. For example, the pixel pair including (x, y-1) and pixel (x, y+1) may only detect artifacts on vertical edges since the pixel pair is vertically aligned with the first pixel. In another example, the pixel pair including (x-1, y-1) and pixel (x+1, y+1) may only detect artifacts on top-left to bottom-right diagonal edges.
  • the image 2 forms a frame, or part of a frame, of a video sequence.
  • motion is detected by comparing the first pixel to one or more other pixels taken from other frames of the video sequence.
  • the first pixel may be compared to a corresponding pixel from the previous frame.
  • Figure 5 illustrates what is meant by corresponding pixel within the context of one embodiment.
  • the image 2 is shown in combination with a preceding frame 2'.
  • the structure of the image 2' is analogous to the above- described structure of image 2.
  • the first pixel (x,y) of image 2 is indicated by a bold border.
  • pixel of image 2' which corresponds to the first pixel (x,y) of image 2.
  • the corresponding pixel is pixel (x',y') of image 2'.
  • a pixel corresponding to the first pixel is a pixel of another image which has the same position within the grid of the other image as the first pixel has within the grid of the image 2. Stated differently, if the first pixel is (x,y), the corresponding pixel of another image is the pixel having the position (x,y) within that other image.
  • motion is detected by calculating the difference between the value of the first pixel (x,y) of the image 2 and the value of the corresponding pixel (x',y') of the image 2'. If the difference is above a predefined threshold then it is considered that motion is detected in respect of the first pixel. It is to be understood that the predefined threshold relating to motion detection may be the same or different from the predefined threshold relating to artifact detection mentioned above. If motion is detected, processing moved to 108, which was discussed above. Therefore, if motion is detected, the first pixel is deinterlaced as described above. In an embodiment, deinterlacing is performed when motion is detected because motion can cause the formation of interlacing artifacts.
  • motion detection is performed by comparing the first pixel to a corresponding pixel of a single previous frame, such as, for example, the adjacent previous frame.
  • the first pixel may be compared to one or more other frames of the video sequence. Additionally, these other frames may be before or after the image 2 in the video sequence. Also, these other frames may be adjacent to the image 2 frame or spaced from the image 2 by one or more other frames. Furthermore, the first pixel may be compared to more than one pixel from another image.
  • a test is performed to identify whether or not all pixels of the image 2 have been processed. If there are no remaining pixels of the image 2 to process, processing flows to 114, wherein the method finishes. Alternatively, if there are remaining pixels of image 2 to process, processing flows to 116. In the present case, only the first pixel of the image 2 has been processed. As can be seen from Figure 2, there are more pixels of the image 2 to process. Therefore, in the present case, processing flows to 116, wherein the next pixel to process is obtained. Once the next pixel has been obtained, processing flows back to 102.
  • the method is performed for each pixel in the image 2. In some embodiments, the method is only performed for one or more but not all of the image's pixels.
  • the order in which pixels of the image 2 are processed follows a spiral formation. For example, the pixels are considered in the following order: (x, y), (x+1, y), (x+1, y+1), (x, y+1), (x-1, y+1), (x-1, y), (x-1, y-1), (x, y-1), (x+1 , y-1), (x+2, y-1), and so on.
  • the pixels of the image 2 are processed line by line, as seen more particularly on Figure 6.
  • the pixels may be processed in any order.
  • the method is only performed for pixel rows in-between the top and bottom rows of an image, i.e. the method is not performed in respect of pixels in the top or bottom rows.
  • the method may be performed in respect of all pixels of the image. For example, a default value may be used in place of the value of the upper or lower pixel of the pixel pair if one is not available.
  • one or more pixels of the image 2 are processed to detect the presence of interlacing artifacts.
  • the pixel being processed is deinterlaced.
  • one or more pixels of the image 2 are processed to detect the presence of motion.
  • the pixel being processed is deinterlaced.
  • the method is adaptive because it adapts the performance of deinterlacing in dependence on the presence of interlacing artifacts or motion. Therefore, deinterlacing is performed in respect of pixels which do, or are likely to, contain interlacing artifacts.
  • the method detects the pixels of an image in which there are interlacing artifacts and only deinterlaces those pixels. Accordingly, the perceptual visual quality of the image is improved. In various embodiments, the perceptual quality of video having motion can be improved. Also, coding efficiency of the image is improved since interlacing artifacts are reduced.
  • An advantage of the above-described method may be that deinterlacing can be performed using a simple averaging technique. Stated differently, the embodiment is computationally efficient. Therefore, the method can be implemented on computing devices having relatively low processing capabilities. [0056] In the above-described embodiment, one pixel pair was tested before motion detection was performed. However, in some other embodiments, one or more additional pixel pairs may be tested before motion detection is performed.
  • Figure 7 illustrates a method of interlacing according to an embodiment.
  • the method of Figure 7 will be described in conjunction with the image 2 of Figure 2.
  • the flow diagram of Figure 7 starts at 200.
  • a first pixel is obtained.
  • the first pixel is as described above with reference to the flow diagram of Figure 4.
  • the first pixel is pixel (x, y).
  • a first pixel pair is obtained.
  • the general form of each pixel pair is as described above with reference to the flow diagram of Figure 4.
  • a pixel pair and the first pixel may be aligned.
  • the alignment may be vertical, as in the case of pixel pair (x, y-1) and (x, y+1).
  • the alignment may be diagonal, as in the case of pixel pair (x-1, y-1) and (x+1, y+1) or pixel pair (x+1, y-1) and (x-1, y+1).
  • a plurality of pixel pairs may be defined and arranged in a predefined order. Accordingly, each pixel pair may be processed in order (i.e. in a prioritized fashion), as will be described below. In some other embodiments, the pixel pairs may not be in any particular order and each pixel pair may be processed in any order.
  • the following three pixel pairs are defined: a first pixel pair of (x, y-1) and (x, y+1), a second pixel pair of (x-1, y-1) and (x+1, y+1), and a third pixel pair of (x+1, y-1) and (x-1, y+1).
  • the first pixel pair is ordered first, followed by the second pixel pair, followed by the third pixel pair. Therefore, at 204, the first pixel pair of (x, y-1) and (x, y+1 ) is obtained.
  • differences between the value of the first pixel and each pixel of the pixel pair are calculated. This process is analogous to 104 of Figure 4.
  • a test is performed to determine whether or not both differences calculated in 206 are above a predefined threshold.
  • This predefined threshold may be the same or different from the predefined threshold of 106 of Figure 4. If both calculated differences are not above the threshold, processing flows to 210, which will be described later. If both calculated differences are above the threshold, processing flows to 212.
  • the next pixel pair is obtained.
  • the next pixel pair to be processed is the second pixel pair.
  • the order may be different or there may be no order.
  • the differences relating to the second pixel pair are calculated. This process is analogous to that of 206 and 104 of Figure 4.
  • a test is performed to determine whether or not both differences relating to the second pixel pair are above a predefined threshold and have matching signs. This predefined threshold may be the same as, or different to, the threshold relating to 208 or 106 of Figure 4. This operation is analogous to that of 106 of Figure 4.
  • a test is performed to determine whether or not all pixel pairs have been processed. If all pixel pairs have been processed, processing flows to 228, otherwise processing flows back to 216. In the present embodiment, three pixel pairs are defined. Therefore, one pixel pair (the third pixel pair) has not yet been processed. Accordingly, processing flows back to 216 wherein the processing loop defined by 216, 220, 222 and 224 or 226 is performed again, but this time in respect of the third pixel pair.
  • a test is performed to determine whether or not one of the calculated differences is above the threshold. If only one of the calculated differences is above the threshold, processing flows to 216. However, if neither of the calculated differences is above the threshold, processing flows to 228. Processing from 216 is as described above.
  • Processing may flow to 228 from 214, 224, 226 or 210 mentioned above.
  • a pixel being processed has either been deinterlaced or not deinterlaced in dependence on whether or not an interlacing artifact has been detected in respect of the pixel.
  • a test is performed to determine whether or not all pixels in the image have been processed, in an analogous way to 112 of Figure 4. If all pixels have been processed, processing flows to 230, wherein the method finishes. Alternatively, if there are still pixels to be processed, processing flows to 232. At 232, the next pixel is obtained in an analogous way to 116 of Figure 4. Once the next pixel has been obtained processing flows back to 204, wherein the method continues as described above.
  • one or more pixels of the image 2 are processed to detect the presence of interlacing artifacts.
  • the pixel being processed is deinterlaced.
  • the method is adaptive because it adapts the performance of deinterlacing in dependence on the presence of interlacing artifacts. Therefore, deinterlacing is performed in respect of pixels which do, or are likely to, contain interlacing artifacts.
  • the method detects the pixels of an image in which there are interlacing artifacts and only deinterlaces those pixels. Accordingly, the perceptual visual quality of the image is improved. In various embodiments, the perceptual quality of video having motion can be improved. Also, coding efficiency of the image is improved since interlacing artifacts are removed.
  • the above-described method is adaptive because it can adapt the amount of artifact detection performed in dependence on the noticeability of artifacts. For example, if an artifact is detected using the first pixel pair, no further detection is performed. Otherwise, the second pixel pair is used for artifact detection. Again, if an artifact is detected then no further detection is performed. Otherwise, the third pixel pair is used for artifact detection, and so on. Accordingly, the amount of processing involved in detecting an artifact is adaptive to how noticeable the artifact is. This is achieved by prioritizing the use of some pixel pairs over the use of other pixel pairs. Specifically, pixel pairs may be prioritized based on the type of interlacing artifact they are suited to detecting.
  • a pixel pair suited to detecting an interlacing artifact on a vertical edge may be prioritized over a pixel pair suited to detecting an interlacing artifact on a diagonal edge (the second or third pixel pairs). This may be because artifacts on vertical edges may be more noticeable than those on diagonal edges.
  • a pixel pair suited to detecting an interlacing artifact on a top-left to bottom-right diagonal edge may be prioritized over a pixel pair suited to detecting an interlacing artifact on a top-right to bottom-left diagonal edge (the third pixel pair).
  • top- left to bottom-right diagonal edges may be more noticeable than those on top-right to bottom-left diagonal edges. Therefore, certain pixel pairs are prioritized over other pixel pairs because they are better at detecting more noticeable interlacing artifacts. Accordingly, unnecessary checking can be avoided.
  • a further advantage of the above-described method may be that multiple pixel pairs may be processed. Furthermore, multiple pixel pairs may be assigned a particular order and they may be processed in accordance with that order. For example, a vertically aligned pixel pair (e.g. the first pixel pair) may be particularly effective at detecting interlacing artifacts on vertical or near vertical edges. However, a diagonally aligned pixel pair (e.g. the second or third pixel pair) may be particularly effective at detecting interlacing artifacts on diagonal or near diagonal edges. Therefore, it may be advantageous to process both a vertically aligned pixel pair and a diagonally aligned pixel pair.
  • An advantage of the above-described embodiment may be that deinterlacing can be performed using a simple averaging technique. Stated differently, the embodiment is computationally efficient. Therefore, the embodiment can be implemented on computing devices having relatively low processing capabilities.
  • a first set of experiments were conducted on seven 704x576 (4CIF) resolution test sequences in order to evaluate the performance of the embodiment described above with reference to Figure 4.
  • These test sequences are commonly used in video de-interlacing research and include: Mobile, Shields and Michael Schumacher.
  • three 4CIF video sequences were captured using a Panasonic BB- MCH581 camera for use as test sequences. All sequences have various visually detectable interlacing artifacts.
  • parts of a test sequence containing interlacing artifacts are compared with corresponding deinterlaced versions.
  • Figures 8 to 13 illustrate the results of the first set of experiments.
  • the embodiment is compared against the following five conventional deinterlacing algorithms: line averaging (LA), line doubling (LD), vertical temporal median filtering (VTMed), spatio-temporal edge-based median filtering (STMed) and edge-based line averaging (ELA).
  • LA line averaging
  • LD line doubling
  • VTMed vertical temporal median filtering
  • STMed spatio-temporal edge-based median filtering
  • ELA edge-based line averaging
  • the average processing time for each algorithm over all test sequences is specified in Figure 14.
  • the embodiment under test is much faster than ELA, VTMed and STMed, but is a little slower than the most simplest algorithms, LA and LD. All the algorithms are implemented in C code, running on 2.4 GHz CPU machine for this experiment.
  • Figure 8 shows the differences between (a) an LA deinterlacing method and (b) the embodiment under test.
  • LA and the embodiment under test include a similar algorithm except that LA performs deinterlacing for the whole frame.
  • the embodiment indentifies areas of the frame with interlacing artifacts and performs deinterlacing on only the identified areas. As can be seen, the embodiment provides better deinterlacing results on moving diagonal edges and the embodiment better preserves horizontal edges on static areas.
  • Figure 9 shows the differences between (a) an LD deinterlacing method and (b) the embodiment under test.
  • LD provides a simple and fast algorithm. As seen more particularly on Figure 9, the LD algorithm introduces artifacts on edges which are not introduced by the embodiment.
  • Figure 10 shows the differences between (a) a VTMed algorithm and (b) the embodiment under test. The embodiment can be seen to reduce more interlacing artifacts from moving areas. It is also noted that the embodiment is 34% faster than VTMed.
  • Figure 11 shows the difference between (a) an STMed algorithm and (b) the embodiment under test. While both the embodiment and STMed reduce interlacing artifacts effectively, the embodiment operates 64.7% faster than STMed. Additionally, STMed appears to introduce some flickering artifacts in some strong texture areas which are not introduced by the embodiment.
  • Figure 12 shows the difference between (a) an ELA algorithm and (b) the embodiment under test.
  • ELA appears to perform better on some diagonal edges; however, ELA seems to cause content detail loss in some texture areas and tends to introduce flickering artifacts, where the embodiment does not. It is also noted that the embodiment runs 34% faster than ELA. [0081] In view of the above, a subjective evaluation performed by viewing the example videos shows that the embodiment under test can reduce interlacing artifacts from most parts of the video and can yield better overall visual quality compared to the other algorithms.
  • FIG. 15 illustrates the results of the second set of experiments, wherein (a) illustrates the embodiment under test, (b) illustrates a known line averaging algorithm, and (c) illustrates the original frame before deinterlacing. In Figure 15, small details on the pale column behind the man in the image are better preserved with the embodiment under test.
  • the embodiment reduces unnecessary deinterlacing, i.e. it does not deinterlace parts of the image without interlacing artifacts.
  • the embodiment is capable of operating in this way because the embodiment can distinguish candidate pixels with interlacing artifacts from candidate pixels without interlacing artifacts. For example, in static scenes, there may be almost no visible interlacing artifacts. Accordingly, the embodiment can bypass these candidate pixels without interlacing artifacts to maintain video quality.
  • Some embodiments provide an efficient, simple and adaptive deinterlacing algorithm which determines whether discemable interlacing visual artifacts occur around a candidate pixel before actually performing deinterlacing in respect of the candidate pixel.
  • deinterlacing is carried out only for a candidate pixel assumed to have artifacts on its surroundings. Accordingly, unnecessary averaging or deinterlacing performed by current line averaging methods may be reduced.
  • the image may include more or less than 5x5 pixels.
  • the height and width of the image may be different; for example, the height may be greater than or less than the width.
  • each pixel may be a shape other than a square; for example, each pixel may have rectangular shape or a triangular shape.
  • the image may be an image portion of a larger image.
  • the image may be a frame, or part of a frame, of a video sequence.
  • the image may comprise a plurality of pixels which are not aligned vertically and/or horizontally.
  • the pixels of a row and/or a column may be offset from the pixels of an adjacent row and/or a column.
  • the pixels may not form a grid.
  • the plurality of pixels of the image may be arranged in any formation, such as, for example, a circular formation or an irregularly shaped formation.
  • the image comprises a plurality of pixels in a two-dimensional formation.
  • the plurality of pixels may be arranged in a different formation, such as, for example, a three-dimensional formation.
  • each pixel pair comprises two pixels which are adjacent to the candidate pixel (e.g. the first pixel).
  • one or more of the pixels of one or more pixel pairs may be spaced from the candidate pixel by one or more pixels.
  • a pixel pair may be vertically aligned with the candidate pixel but may comprise a pixel which is spaced from the candidate pixel by one pixel.
  • a pixel pair may be diagonally aligned with the candidate pixel but may comprise one pixel which is spaced from the candidate pixel by two pixels and another pixel which is spaced from the candidate pixel by one pixel.
  • a candidate pixel is compared to each of a pair of pixels to detect interlacing artifacts.
  • the candidate pixel may be compared to only one other pixel or more than two other pixels to detect interlacing artifacts.
  • Figure 16 depicts an example computing device 1000 that may be utilized to implement a method for deinterlacing an image according to an embodiment.
  • the computing device 1000 is an apparatus for deinterlacing an image.
  • the following description of computing device 1000 is provided by way of example only and is not intended to be limiting.
  • example computing device 1000 includes a processor 1004 for executing software routines. Although a single processor is shown for the sake of clarity, computing device 1000 may also include a multi-processor system. Processor 1004 is connected to a communication infrastructure 1006 for communication with other components of computing device 1000. Communication infrastructure 1006 may include, for example, a communications bus, cross-bar, or network. [0092] Computing device 1000 further includes a main memory 1008, such as a random access memory (RAM), and a secondary memory 1010. Secondary memory 1010 may include, for example, a hard disk drive 1012 and/or a removable storage drive 1014, which may include a floppy disk drive, a magnetic tape drive, an optical disk drive, or the like.
  • RAM random access memory
  • Removable storage drive 1014 reads from and/or writes to a removable storage unit 1018 in a well known manner.
  • Removable storage unit 1018 may include a floppy disk, magnetic tape, optical disk, or the like, which is read by and written to by removable storage drive 1014.
  • removable storage unit 1018 includes a computer readable storage medium having stored therein computer executable program code instructions and/or data.
  • secondary memory 1010 may include other similar means for allowing computer programs or other instructions to be loaded into computing device 1000.
  • Such means can include, for example, a removable storage unit 1022 and an interface 1020.
  • a removable storage unit 1022 and interface 1020 include a program cartridge and cartridge interface (such as that found in video game console devices), a removable memory chip (such as an EPROM or PROM) and associated socket, and other removable storage units 1022 and interfaces 1020 which allow software and data to be transferred from the removable storage unit 1022 to computer system 1000.
  • Computing device 1000 also includes at least one communication interface 1024.
  • Communication interface 1024 allows software and data to be transferred between computing device 1000 and external devices via a communication path 1026.
  • communication interface 1024 permits data to be transferred between computing device 1000 and a data communication network, such as a public data or private data communication network.
  • Examples of communication interface 1024 can include a modem, a network interface (such as Ethernet card), a communication port, and the like.
  • Software and data transferred via communication interface 1024 are in the form of signals which can be electronic, electromagnetic, optical or other signals capable of being received by communication interface 1024. These signals are provided to the communication interface via communication path 1026.
  • computing device 1000 further includes a display interface 1002 which performs operations for rendering images to an associated display 1030 and an audio interface 1032 for performing operations for playing audio content via associated speaker(s) 1034.
  • the term "computer program product” may refer, in part, to removable storage unit 1018, removable storage unit 1022, a hard disk installed in hard disk drive 1012, or a carrier wave carrying software over communication path 1026 (wireless link or cable) to communication interface 1024.
  • a computer readable medium can include magnetic media, optical media, or other recordable media, or media that transmits a carrier wave or other signal.
  • Computer programs are stored in main memory 1008 and/or secondary memory 1010. Computer programs can also be received via communication interface 1024. Such computer programs, when executed, enable the computing device 1000 to perform one or more features of embodiments discussed herein. In various embodiments, the computer programs, when executed, enable the processor 1004 to perform features of the above-described methods for deinterlacing an image. Accordingly, such computer programs represent controllers of the computer system 1000.
  • Software may be stored in a computer program product and loaded into computing device 1000 using removable storage drive 1014, hard disk drive 1012, or interface 1020. Alternatively, the computer program product may be downloaded to computer system 1000 over communications path 1026. The software, when executed by the processor 1004, causes the processor 1004 to perform functions of embodiments described herein. [0099] It is to be understood that the embodiment of Figure 16 is presented merely by way of example. Therefore, in some embodiments one or more features of the computing device 1000 may be omitted. Also, in some embodiments, one or more features of the computing device 1000 may be combined together. Additionally, in some embodiments, one or more features of the computing device 1000 may be split into one or more component parts.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Television Systems (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Selon des modes de réalisation fournis à titre d'exemple, la présente invention se rapporte à un procédé, à un appareil et à un produit programme informatique adaptés pour désentrelacer une image ayant une pluralité de pixels. Le procédé selon l'invention consiste : à calculer une différence entre un premier pixel de l'image et chaque pixel d'au moins une paire de pixels, chaque paire de pixels comprenant un pixel qui est placé en dessus du premier pixel et un autre pixel qui est placé en dessous du premier pixel ; et à désentrelacer le premier pixel uniquement si au moins une différence correspondant à une paire de pixels dépasse un seuil prédéfini. La présente invention se rapporte d'autre part à un appareil et à un produit programme informatique correspondants.
PCT/SG2011/000364 2010-10-20 2011-10-20 Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels WO2012053978A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
SG2013019054A SG188546A1 (en) 2010-10-20 2011-10-20 A method, an apparatus and a computer program product for deinterlacing an image having a plurality of pixels
US13/879,686 US9171370B2 (en) 2010-10-20 2011-10-20 Method, an apparatus and a computer program product for deinterlacing an image having a plurality of pixels

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US39479410P 2010-10-20 2010-10-20
US61/394,794 2010-10-20

Publications (1)

Publication Number Publication Date
WO2012053978A1 true WO2012053978A1 (fr) 2012-04-26

Family

ID=45975482

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/SG2011/000364 WO2012053978A1 (fr) 2010-10-20 2011-10-20 Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels

Country Status (3)

Country Link
US (1) US9171370B2 (fr)
SG (1) SG188546A1 (fr)
WO (1) WO2012053978A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012053978A1 (fr) * 2010-10-20 2012-04-26 Agency For Science, Technology And Research Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels
US9819900B2 (en) * 2015-12-30 2017-11-14 Spreadtrum Communications (Shanghai) Co., Ltd. Method and apparatus for de-interlacing television signal
US11039081B2 (en) 2019-07-17 2021-06-15 Pixart Imaging Inc. Image sensor capable of eliminating rolling flicker and adjusting frame rate

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5929935A (en) * 1996-12-02 1999-07-27 Motorola, Inc. Method for reducing flicker and circuit therefor
US20070200950A1 (en) * 2006-02-28 2007-08-30 Samsung Electronics Co., Ltd. Video Image Deinterlacing Apparatus and Methods of Performing Video Image Deinterlacing

Family Cites Families (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR910009880B1 (ko) * 1983-07-25 1991-12-03 가부시기가이샤 히다찌세이사꾸쇼 인터레이스된 텔레비젼 신호중의 화상의 움직임 검출회로
JP2687670B2 (ja) * 1990-04-19 1997-12-08 松下電器産業株式会社 動き検出回路および手ぶれ補正装置
WO1997005598A1 (fr) * 1995-07-27 1997-02-13 Magix Systeme pour diminuer le scintillement dans l'affichage entrelace d'un ordinateur graphique
JP2002064792A (ja) * 2000-08-14 2002-02-28 Sony Corp 画像信号処理装置およびその方法
US6633612B2 (en) * 2000-12-13 2003-10-14 Genesis Microchip Inc. Method and apparatus for detecting motion between odd and even video fields
JP4355577B2 (ja) * 2002-03-27 2009-11-04 トムソン ライセンシング ノイズ抑制対角線方向補正によるアップコンバージョン
US20030218621A1 (en) * 2002-05-24 2003-11-27 Jiande Jiang Method and system for edge-adaptive interpolation for interlace-to-progressive conversion
KR100580172B1 (ko) * 2003-06-27 2006-05-16 삼성전자주식회사 디-인터레이싱 방법, 그 장치, 그 비디오 디코더 및 그재생 장치
TWI253848B (en) * 2004-04-09 2006-04-21 Mstar Semiconductor Inc Pixel interpolation method and related pixel interpolation device
US7538799B2 (en) * 2005-01-14 2009-05-26 Freescale Semiconductor, Inc. System and method for flicker detection in digital imaging
US7477319B2 (en) * 2005-06-17 2009-01-13 Lsi Corporation Systems and methods for deinterlacing video signals
US7924345B2 (en) * 2005-10-20 2011-04-12 Broadcom Corp. Method and system for deinterlacing using polarity change count
JP4891103B2 (ja) * 2007-01-25 2012-03-07 キヤノン株式会社 動き量検出装置及びそれを利用した画像処理装置、並びにそれらの制御方法
JP4887275B2 (ja) * 2007-12-27 2012-02-29 富士フイルム株式会社 撮像装置及びそのシャッタ駆動モード選択方法
US8593572B2 (en) * 2008-01-30 2013-11-26 Csr Technology Inc. Video signal motion detection
JP5311916B2 (ja) * 2008-07-31 2013-10-09 株式会社エルモ社 撮像装置
WO2012053978A1 (fr) * 2010-10-20 2012-04-26 Agency For Science, Technology And Research Procédé, appareil et produit programme informatique pour désentrelacer une image ayant une pluralité de pixels
US8471962B2 (en) * 2011-06-30 2013-06-25 Stmicroelectronics Asia Pacific Pte Ltd. Apparatus and method for local video detector for mixed cadence sequence

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5929935A (en) * 1996-12-02 1999-07-27 Motorola, Inc. Method for reducing flicker and circuit therefor
US20070200950A1 (en) * 2006-02-28 2007-08-30 Samsung Electronics Co., Ltd. Video Image Deinterlacing Apparatus and Methods of Performing Video Image Deinterlacing

Also Published As

Publication number Publication date
US9171370B2 (en) 2015-10-27
SG188546A1 (en) 2013-04-30
US20130287315A1 (en) 2013-10-31

Similar Documents

Publication Publication Date Title
CA2387701C (fr) Systeme et procede de compensation du mouvement et de conversion de la frequence des trames
US7423691B2 (en) Method of low latency interlace to progressive video format conversion
US7259794B2 (en) De-interlacing device and method therefor
US10025988B2 (en) Anomalous pixel detection
US7336316B2 (en) De-interlacing of video data
US20080174694A1 (en) Method and apparatus for video pixel interpolation
EP1964395B1 (fr) Procedes et appareil pour le balayage progressif d'une video entrelacee
US8593572B2 (en) Video signal motion detection
US9171370B2 (en) Method, an apparatus and a computer program product for deinterlacing an image having a plurality of pixels
EP2178289B1 (fr) Procédé et unité de détection de mouvement basée sur un histogramme de différence
WO2006020532A2 (fr) Filtrage rapide par zone selectionnee, destine a la reduction d'artefacts de type pixel-bruit et analogue
US8031945B2 (en) Image-processing device, image-processing method, program of image-processing method, and recording medium recording program of image-processing method
US8704945B1 (en) Motion adaptive deinterlacer
US6384872B1 (en) Method and apparatus for interlaced image enhancement
CN105898179A (zh) 一种隔行视频的去隔行方法及装置
EP1576808B1 (fr) Procede a vecteurs 3d permettant la detection de mouvement inter-champs
EP2109310B1 (fr) Procédé de quantification d'un mouvement inter-champs dans une image vidéo
EP2509045B1 (fr) Procédé et appareil permettant de détecter des limites d'image dans des données vidéo
WO2006013510A1 (fr) Desentrelacement
CN105898177A (zh) 基于局部信息的去隔行方法及装置
Günyel et al. Method and apparatus for the detection and classification of occlusion regions
JPH08331516A (ja) フリッカー抑制装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 11834731

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 13879686

Country of ref document: US

122 Ep: pct application non-entry in european phase

Ref document number: 11834731

Country of ref document: EP

Kind code of ref document: A1