EP0989748B1 - Conversion de données d'images et processeur vidéo - Google Patents

Conversion de données d'images et processeur vidéo Download PDF

Info

Publication number
EP0989748B1
EP0989748B1 EP19990306636 EP99306636A EP0989748B1 EP 0989748 B1 EP0989748 B1 EP 0989748B1 EP 19990306636 EP19990306636 EP 19990306636 EP 99306636 A EP99306636 A EP 99306636A EP 0989748 B1 EP0989748 B1 EP 0989748B1
Authority
EP
European Patent Office
Prior art keywords
class
picture signal
signal
picture
class value
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Lifetime
Application number
EP19990306636
Other languages
German (de)
English (en)
Other versions
EP0989748A2 (fr
EP0989748A3 (fr
Inventor
Tetsujiro Kondo
Yasushi Tatehira
Nobuyuki Asakura
Masashi Uchida
Takuo Morimura
Kazutaka Ando
Hideo Nakaya
Tsutomu Watanabe
Satoshi Inoue
Wataru Niitsuma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Corp
Original Assignee
Sony Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Corp filed Critical Sony Corp
Publication of EP0989748A2 publication Critical patent/EP0989748A2/fr
Publication of EP0989748A3 publication Critical patent/EP0989748A3/fr
Application granted granted Critical
Publication of EP0989748B1 publication Critical patent/EP0989748B1/fr
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes
    • H04N7/0137—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes dependent on presence/absence of motion, e.g. of motion zones
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0117—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving conversion of the spatial resolution of the incoming video signal
    • H04N7/012—Conversion between an interlaced and a progressive signal
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0125—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level one of the standards being a high definition standard
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/01—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0135—Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level involving interpolation processes

Definitions

  • the present invention relates to a picture information conversion and video processor.
  • Illustrative embodiments of the invention relate to a picture information converting apparatus, a picture information converting method, and a video processor such as a television receiver.
  • a picture information converting process for generating an output scanning line structure with a scanning line structure different from an input picture signal.
  • classes are categorized corresponding to a three-dimensional (time-space) distribution of signal levels of the input picture signal. With reference to the obtained class values, pixels are predicted and generated. In such a process, it is more difficult to predict and generate pixels at positions of scanning lines of the input picture signal than pixels at other positions thereof.
  • Embodiments of the present invention seek to provide a picture information converting apparatus, a picture information converting method, and a video processor, for example, a television receiver that allow a proper number of classes to be assigned corresponding to the relation of positions of pixels predicted and generated and postions of scanning lines of an input picture signal, conditions such as the storage capacity of a memory disposed in the apparatus, and so forth.
  • Japanese Patent Application Publication No JP-A-09074543 (published 13 July 1997 ), which corresponds to United States Patent No US-A-5 946 044 (published 31 August 1999 ), discloses an image signal converting apparatus for converting a first input image signal to a second image signal different from said first image signal.
  • a class code deciding means decides the class of a noticed pixel in accordance with movement indicated by a change of at least one pixel in the vicinity of the noticed pixel between at least a first input image and a second input image of the first input image signal and patterns of at least one pixel in the vicinity of the noticed pixel in at least one of the first or second input images of the first input image signal, and generates a class code showing the class.
  • a predictive coefficient data storage means stores predictive coefficient data serving as information for converting the first input image signal to the second input image signal for each class code to output the predictive coefficient data corresponding to the class code
  • a predictive operating means generates the second image signal by using the predictive coefficient data output from the predictive coefficient data storage means corresponding to the class code and a plurality of pixels spatially and/or temporally present in the vicinity of said noticed pixel and thereby performs a predictive operation
  • the apparatus perform a classification considering various signal characteristics of input image data and converts low-resolution input image data to high-resolution image data by using not only a classifying means for classifying input images in accordance with the level distribution pattern of the input images, but also a classifying means for classifying input images in accordance with the movement of the input images and deciding the final class of input image signals from classes obtained by both the classifying means.
  • the present invention provides a picture information converting apparatus according to claim 1 hereof, a television receiver according to claim 5 hereof, and a picture information converting method according to claim 6 hereof.
  • the number of classes for each of considered points (pixels) is properly assigned corresponding to a relation that represents whether or not each considered point is present at a position on a scanning line of an input picture signal or corresponding to a condition such as the storage capacity of a memory of the apparatus.
  • a digital picture signal with standard resolution (hereinafter this signal is referred to as SD signal) is converted into a picture signal with high resolution (sometimes this signal is referred to as HD signal) and the resultant signal is output.
  • SD signal is an interlace picture signal with 525 scanning lines (hereinafter this signal is referred to as 525i signal).
  • a example of the HD signal is a progressive signal with 525 scanning lines (hereinafter this signal is referred to as 525p signal).
  • the number of pixels in the horizontal direction of the output picture signal is twice as many as that of the input picture signal.
  • the resolution of the input picture signal is improved by a class categorization adaptive process.
  • the class categorization adaptive process is different from a conventional method of which a signal with high resolution is obtained by an interpolating process.
  • classes are categorized corresponding to a three-dimensional (time-space) distribution of signal levels of the input SD signal. Predicted values for individual classes are leant and stored to a particular storing portion. An optimum estimated value is calculated and obtained corresponding to a predictive expression.
  • an output signal with higher resolution than an input SD signal can be obtained.
  • Fig. 1 shows an example of the arrangement of pixels of one field (odd field) in a picture information converting process for converting a 525i signal as an input SD signal into a 525p signal as an output picture signal.
  • large dots represent pixels of the 525i signal
  • small dots represent pixels of 525p signal.
  • the scanning lines of the 525i signal of the odd field spatially deviates by 0.5 lines from that of the even field.
  • line data y1 black small dots
  • line data y2 white small dots
  • y1 represents points that exist
  • y2 represents points that are newly predicted and generated.
  • the memory in the apparatus does not have sufficient storage capacity or the picture information converting process is performed with a small number of classes, the total number of classes should be decreased.
  • the number of classes are properly assigned so that y1 and y2 can be effectively and precisely predicted and generated.
  • a picture information converting process is performed.
  • a 525i signal as an input SD signal is converted into a 525p signal as an output picture signal.
  • Fig. 2 is a block diagram showing an example of the structure of a picture information converting process system according to the first embodiment of the present invention.
  • the input SD signal (525i signal) is supplied to tap selecting circuits 1, 10, 11, and 20.
  • the tap selecting circuit 1 extracts a region of a plurality of pixels necessary for a calculating process (corresponding to formula (1) (that will be described later) for predicting and estimating line data y1 and y2 and selects SD pixels (hereinafter referred to as predictive taps) necessary for predicting and estimating the line data y1 and y2 from the extracted region.
  • the selected predictive taps are supplied to estimation predictive calculating circuits 4 and 5.
  • the predictive taps may be the same as spatial class taps (that will be described later). To improve the predictive accuracy, the.predictive taps are selected with predictive tap position information corresponding to classes.
  • a coefficient memory 41 supplies predictive coefficients necessary for predicting and estimating the line data y1 and y2 to the estimation predictive calculating circuits 4 and 5, respectively.
  • the estimation predictive calculating circuits 4 and 5 successively predict and generate pixel values y with the predictive taps received from the tap selecting circuit 1 and the predictive coefficients received from the coefficient memory 41 corresponding to the following formula (1).
  • y w ⁇ 1 x x ⁇ 1 + x ⁇ 2 x x ⁇ 2 + ... + wn x xn where x1, ... , and xn represent predictive taps; and w1, ..., and wn are predictive coefficients.
  • formula (1) is an expression for predicting and generating a pixel value y with n predictive taps.
  • the estimation predictive calculating circuit 4 predicts and generates line data y1 as a sequence of pixel values y.
  • the estimation predictive calculating circuit 5 predicts and generates line data y2.
  • the predictive coefficients w1, ..., and wn of the line data y1 are different from those of the line data y2.
  • the estimation predictive calculating circuits 4 and 5 supply the line data y1 and y2 to a line sequence converting circuit 6, respectively.
  • the line sequence converting circuit 6 performs a line double speed process for the received line data y1 and y2 and generates a signal with high resolution (referred to as high resolution signal).
  • the high resolution signal is a final output picture signal of the picture information converting process system according to the first embodiment of the present invention.
  • the output picture signal is supplied to a CRT display (not shown).
  • the CRT display has a synchronous system that allows the output picture signal (525p signal) to be displayed.
  • Examples of the input SD signal are a broadcast signal and a reproduction signal of a reproducing apparatus such as a VCR.
  • the picture information converting process system according to the first embodiment of the present invention can be built in a TV receiver or the like.
  • Each of the tap selecting circuits 10 and 11 selects SD pixels necessary for detecting spatial classes of the line data y1 and y2 (these pixels are referred to as spatial class taps).
  • the tap selecting circuit 20 selects SD pixels necessary for detecting a motion class of the input SD signal (these pixels are referred to as motion class taps).
  • Output data of the tap selecting circuit 10 is supplied to a spatial class detecting circuit 12.
  • Output data of the tap selecting circuit 11 is supplied to a spatial class detecting circuit 13.
  • Output data of the tap selecting circuit 20 is supplied to a motion class detecting circuit 21.
  • the spatial class detecting circuits 12 and 13 detect spatial class values of the line data y1 and y2 corresponding to the received spatial class taps and supply the detected spatial class values to class combining circuits 30 and 31, respectively.
  • the motion class detecting circuit 21 detects a motion class value corresponding to the received motion class taps and supplies the detected motion class value to the class combining circuits 30 and 31.
  • Each of the class combining circuits 30 and 31 combines the received spatial class value and the received motion class value.
  • Output data of the class combining circuit 30 is supplied to a class value converting circuit 32.
  • Output data of the class combining circuit 31 is supplied to a class value converting circuit 33.
  • the class value converting circuits 32 and 33 perform a class value converting process (that will be described later) for output data of the class value combining circuits 32 and 33, respectively.
  • the class value converting process allows the total number of classes to be decreased.
  • the converted class values are supplied to a coefficient memory 41.
  • the coefficient memory 41 stores predictive coefficients that have been learnt and supplies predictive coefficients corresponding to the class values received from the class value converting circuits 32 and 33 to the estimation predictive calculating circuits 4 and 5. To do that, for example, a method for storing predictive coefficients corresponding to addresses designated with class values obtained in the class value converting process may be used.
  • each spatial class detecting circuit detects a spatial pattern of a level distribution of picture data corresponding to a pattern of a level distribution of spatial class taps and generates a spatial class value corresponding to the detected spatial pattern.
  • each input pixel is compressed to data with less than eight bits.
  • ADRC Adaptive Dynamic Range Coding
  • DPCM Differential Pulse Code Modulation
  • VQ Vector Quantization
  • the ADRC method is an adaptively requantizing method developed for a high efficient encoding process for use with a VCR (Video Cassette Recorder). Since the ADRC method allows a local pattern of a signal level to be effectively represented with a short word length, according to the first embodiment of the present invention, the ADRC method is used to generate a spatial class categorized code. In the ADRC method, the length between the maximum value MAX and the minimum value MIN is equally divided by a designated bit length and re-quantized corresponding to the following formula (1).
  • DR MAX - MIN + 1
  • Q L - MIN + 0.5 ⁇ x 2 / DR
  • DR represents the dynamic range of spatial class taps
  • L represents the data level of the pixel of each spacial class tap
  • Q represents a re-quantized code
  • ⁇ ⁇ represents a truncating process
  • Fig. 3 shows an example of the arrangement of spatial class taps used in the spatial class detecting process for the line data y1.
  • Fig. 3 when time elapses in the horizontal direction (from left to right), pixels in individual fields are arranged in the vertical direction.
  • Fields shown in Fig. 3 are denoted by F-1/o (at the leftmost position), F-1/e, F/o, and F/e (at the rightmost position).
  • the field F-1/o represents "a field composed of odd-numbered scanning lines of the (F-1)th frame”.
  • the field F-1/e represents "a field composed of even-numbered scanning lines of the (F-1)th frame”.
  • the field F/o represents "a field composed of odd-numbered scanning lines of the F-th frame”.
  • the field F/e represents "a field composed of even-numbered scanning lines of the F-th frame”.
  • spatial class taps of the line data y1 of the field F/o a total of five pixels - pixels T4 and T5 of the field F-1/e and pixels T1, T2, and T3 of the field F/o are used.
  • other spatial class taps may be used.
  • a plurality of input pixels in the horizontal direction may be used.
  • Fig. 4 shows an example of the arrangement of spatial class taps used in the spatial class detecting process for the line data y2.
  • the notation of Fig. 4 is the same as the notation of Fig. 3 .
  • spatial class taps of the line data y2 of the field F/o a total of six pixels - a pixel T6 of the field F-1/e, pixels T2, T3, T4, and T5 of the field F/o, and a pixel T1 of the field F/e are used.
  • other spatial class taps may be used.
  • a plurality of input pixels in the horizontal direction may be used.
  • the motion class detecting circuit 21 calculates the average value param of the absolute values of frame differences with the received motion class taps corresponding to the following formula (3).
  • n represents the number of motion class taps. For example, 6 is assigned to n (see Fig. 5 ).
  • a motion class value as an index of a motion is determined.
  • the motion class value is generated as follows.
  • the motion class value When the motion class value is 0, the motion is minimum (still picture). As the motion class value becomes large, the motion of the picture becomes large.
  • the motion class may be determined corresponding to a motion vector.
  • Fig. 5 shows an example of the arrangement of motion class taps used in the motion class detecting process.
  • motion class taps used for predicting the line data y1 and y2 of the field F/o pixels n1, n3, and n5 of the field F/o
  • pixels n2, n4, and n6 of the field F/e pixels m2, m4, and m6 of the field F-/e
  • pixels m1, m3, and m5 of the field F-1/e are used.
  • the vertical positions of the pixels m1, m2, .., and m6 match those of the pixels n1, n2, ..., and n6, respectively.
  • other motion class taps may be used.
  • Fig. 6 shows a process for decreasing the total number of classes. Temporary predictive coefficients shown in Fig. 6 will be described later.
  • FIG. 7 shows an example of the structure of a process system that generates a class value conversion table as a first learning stage.
  • a known signal for example, a 525p signal
  • the thin-out filter 51 thins out the number of pixels by 1/2 in each of the horizontal and vertical directions.
  • the thin-out filter 51 generates an SD signal (for example, a 525i signal) with pixels that are 1/4 as small as those of the input signal.
  • a vertical thin-out filter thins out pixels of the input picture signal so that the frequency in the vertical direction of the input picture signal is halved.
  • a horizontal thin-out filter thins out pixels of the input picture signal so that the frequency in the horizontal direction of the input picture signal is halved.
  • Fig. 8 shows the spatial relation between pixels of a 525p signal as an example of the SD signal that is supplied to the thin-out filter 51 and pixels of a 525i signal as an example of the output picture signal of the thin-out filter 51.
  • odd-numbered field of the 525p signal even numbered lines are thinned out. On each odd-numbered line, pixels are alternately thinned out in the horizontal direction.
  • Fig. 8 the first field and the third field that have been thinned out are shown.
  • odd-numbered field of the 525p signal odd-numbered lines are thinned out.
  • pixels are alternately thinned out in the horizontal direction.
  • Fig. 8 also shows the second field of which pixels have been thinned out.
  • the thin-out filter 51 generates a interlace picture signal and supplies it to tap selecting circuits 101, 110, 111, and 120.
  • the tap selecting circuit 101 selects predictive taps used in a calculation of a normal equation (that will be described later).
  • the tap selecting circuit 101 supplies the selected predictive taps to normal equation adding circuits 61 and 62.
  • the tap selecting circuits 110 and 111 select spatial class taps of the line data y1 and y2 and supply the selected spatial taps to spatial class detecting circuits 112 and 113, respectively.
  • the tap selecting circuit 120 selects motion class taps and supplies the selected motion class taps to a motion class detecting circuit 121.
  • the spatial class detecting circuits 112 and 113 compress data of the spatial class taps corresponding to the ADRC method and detect spatial class values of the line data y1 and y2, respectively.
  • the motion class detecting circuit 121 detects motion class values. Output data of the spatial class detecting circuit 112 and the motion class detecting circuit 121 is supplied to a class combining circuit 130. In addition, output data of the spatial class detecting circuit 113 and the motion class detecting circuit 121 are supplied to a class combining circuit 131.
  • the class combining circuit 130 combines the spatial class value and the motion class value of the line data y1.
  • the class combining circuit 131 combines the spatial class value and the motion class value of the line data y2. Output data of the class combining circuit 130 and the class combining circuit 131 are supplied to the normal equation adding circuits 61 and 62, respectively.
  • the normal equation adding circuit 61 performs an adding operation for obtaining data of a normal equation having a solution of predictive coefficients for predicting and generating the line data y1 corresponding to the input SD signal, the output data of the tap selecting circuit 101, and the output data of the class combining circuit 130.
  • the normal equation adding circuit 62 performs an adding operation for obtaining data of a normal equation having a solution of predictive coefficients for predicting and generating the line data y2 corresponding to the input SD signal, the output signal of the tap selecting circuit 101, and the output data of the class combining circuit 131.
  • the normal equation adding circuits 61 and 62 supply their calculated data to a temporary predictive coefficient determining circuit 63.
  • the temporary predictive coefficient determining circuit 63 performs a calculating process for solving the normal equation corresponding to the received data, calculates temporary predictive coefficients for predicting and generating the line data y1 and y2 and supplies the calculated temporary predictive coefficients to a adjacent coefficient class integrating process portion 64.
  • the temporary predictive coefficients are not final predictive coefficients because they are determined as the result of the class integrating process (that will be described later).
  • the calculating process for solving the normal equation is the same as that of a predictive coefficient determining circuit 263 (that will be described later with reference to Fig. 9 ).
  • the adjacent coefficient class integrating process portion 64 calculates the distance d between the temporary predictive coefficients corresponding to the following formula (4).
  • kl and k1' represent predictive coefficients of different classes
  • 1 represents a predictive tap number
  • n represents the total number of predictive taps.
  • the adjacent coefficient class integrating process portion 64 compares the distances of predictive coefficients of different classes with a predetermined threshold value and integrates a plurality of classes whose distance is smaller than the threshold value into one class. By integrating a plurality of classes, the total number of classes can be decreased. When distances of particular classes are smaller than the threshold value, classes whose distance is minimum are integrated. As the threshold value becomes large, the number of classes that can be deleted becomes large. Thus, by varying the threshold value, the number of classes obtained in the class value converting process can be adjusted.
  • the memory resource can be effectively used.
  • the picture information converting process can be effectively performed.
  • the adjacent coefficient class integrating process portion 64 supplies information for integrating classes to a class value converting table generating portion 65.
  • the class value converting table 65 generates a class value conversion table corresponding to the received information.
  • the class value conversion table is supplied to the class value converting circuits 32 and 33 (see Fig. 2 ) and class value converting circuits 81 and 82 (that will be described later with reference to Fig. 9 ).
  • the class value converting circuits 32 and 33 and the class value converting circuits 81 and 82 store and reference the class value conversion table.
  • FIG. 9 shows an example of the structure of a predictive coefficient calculating process system that calculates predictive coefficients.
  • a known signal for example, a 525p signal
  • the thin-out filter 251 thins out the number of pixels by 1/2 in each of the horizontal and vertical directions.
  • the thin-out filter 251 generates an SD signal (for example, a 525i signal) with pixels that are 1/4 as small as those of the input signal.
  • a vertical thin-out filter thins out pixels of the input picture signal so that the frequency in the vertical direction of the input picture signal is halved.
  • a horizontal thin-out filter thins out pixels of the input picture signal so that the frequency in the horizontal direction of the input picture signal is halved.
  • the thin-out filter 251 supplies the SD signal to tap selecting circuits 201, 210, 211, and 220. By varying the characteristics of the thin-out filter 251, the learning characteristics are varied. Thus, the picture quality of the picture to be converted can be controlled.
  • the tap selecting circuit 201 selects predictive taps and supplies the selected predictive taps to normal equation adding circuits 261 and 262.
  • the tap selecting circuits 210 and 211 select spatial class taps for the line data y1 and y2 and supplies the selected spatial class taps to spatial class detecting circuits 212 and 213, respectively.
  • a tap selecting circuit 220 selects predictive taps and supplies the selected predictive taps to a motion class detecting circuit 221.
  • the spatial class detecting circuits 212 and 213 detect spatial class values for the line data y1 and y2 corresponding to the received spatial class taps, respectively.
  • the motion class detecting circuit 221 detects a motion class value corresponding to the received motion class taps.
  • Output data of the spatial class detecting circuit 212 and the motion class detecting circuit 212 is supplied to a class combining circuit 230.
  • output data of the spatial class detecting circuit 212 and the motion class detecting circuit 221 is supplied to a class combining circuit 231.
  • the class combining circuits 230 and 231 combine class values for the line data y1 and y2 and supply the combined class values to class value converting process portions 81 and 82, respectively.
  • the class value converting process portions 81 and 82 store the above-mentioned class value conversion table. With reference to the class value conversion table, the class value converting process portions 81 and 82 convert the class values received from the class combining circuits 230 and 231, respectively. The class value converting process portions 81 and 82 supply the resultant class values to the normal equation adding circuits 261 and 262, respectively.
  • the normal equation adding circuits 261 and 262 calculate data used in a calculating process for solving a normal equation having a solution of predictive coefficients.
  • the normal equation adding circuit 261 performs an adding process corresponding to the input SD signal, the output data of the tap selecting circuit 201, and the output data of the class value converting circuit 81 and obtains data necessary for solving a normal equation having a solution of predictive coefficients for predicting and generating the line data y1.
  • the normal equation adding circuit 262 performs an adding process corresponding to the input SD signal, the output'data of the tap selecting circuit 201, and the output data of the class value converting circuit 82 and obtains data necessary for solving a normal equation having a solution of predictive coefficients for predicting and generating the line data y2.
  • Data obtained by the normal equation adding circuits 261 and 262 is supplied to a predictive coefficient determining circuit 263.
  • the predictive coefficient determining circuit 263 performs a calculating process for solving a normal equation corresponding to the received data and obtains predictive coefficients for predicting and generating the line data y1 and y2.
  • the obtained predictive coefficients are supplied to a coefficient memory 84.
  • the coefficient memory 84 stores the predictive coefficients.
  • predictive coefficients w1, ..., and wn are initially indeterminate coefficients.
  • a plurality of pieces of signal data are learned for individual classes.
  • the following formula (5) is given corresponding to formula (1).
  • yk w ⁇ 1 x xk ⁇ 1 + w ⁇ 2 x xk ⁇ 2 + ... + wn x xkn
  • k 1, 2, ..., m
  • elements ek of an error vector e are defined by the following formula (6).
  • Predictive coefficients that minimize the error vector e defined by formula (7) are obtained.
  • predictive coefficients are uniquely obtained by so-called method of least squares.
  • ek yk - w ⁇ 1 x xk ⁇ 1 + w ⁇ 2 x xk ⁇ 2 + ... + wn x xkn
  • Each predictive coefficient wi is defined so that the partial differential value of each value of i becomes 0.
  • formula (8) can be written as a matrix of formula (11).
  • Formula (11) is referred to as normal equation.
  • the normal equation adding circuits 261 and 262 calculate data of the normal equation (namely, Xji and Yi of formulas (9) and (10)) corresponding to the class values received from the class value converting circuits 81 and 82, the predictive taps received from the predictive tap selecting circuit 201, and the known picture signal in the same signal format as the input picture signal and supply the obtained data of the normal equation to a predictive coefficient determining portion 263.
  • the predictive coefficient determining portion 263 performs a calculating process for solving the normal equation corresponding to a conventional matrix solving method such as sweep-out method with the data of the normal equation and obtains predictive coefficients wi.
  • the predictive coefficients generated in such a manner are calculated corresponding to data of which class values have been converted.
  • temporary predictive coefficients described with reference to Fig. 7 are obtained corresponding to data of which class values have not been converted. Except for such a difference, the predictive coefficients and the temporary predictive coefficients are obtained in the same calculating process as a solution of the normal equation.
  • the process for determining the temporary predictive coefficients is the same as the process performed in the normal equation adding circuits 61 and 62 and the predictive coefficient determining circuit 63.
  • the normal equation adding circuits 61 and 62 calculate data of the normal equation (namely, the values of Xji and Yi of formulas (9) and (10)) with the class values received from the class combining circuits 130 and 131, the predictive taps received from the predictive tap selecting circuit 101, and the known input picture signal (for example, a 525p signal) in the same signal format as the output picture signal and supply the calculated data of the normal equation to the temporary predictive coefficient determining portion 63.
  • the temporary predictive coefficient determining portion 63 performs a calculating process for solving the normal equation corresponding to a conventional matrix solving method such as sweep-out method with the data of the normal equation.
  • the picture signal converting process system shown in Fig. 5 , the class value conversion table generating process system shown in Fig. 7 , and the predictive coefficient process system shown in Fig. 9 may be separately disposed. Alternatively, with a switch or the like, these systems may be used in common. However, when such process systems are separately disposed, structural elements having the same function should have the same operating characteristics so as to secure the validity of the learning process. For example, the tap selecting circuits 1, 51, and 201 should have the same operating characteristics. When the structural elements are used in common, although the process of the apparatus becomes complicated due to the switching operations, the circuit scale of the apparatus is reduced. Thus, the cost of the apparatus can be reduced.
  • the number of predictive taps that are output from the tap selecting circuit 201 is larger than the number of predictive taps used in the picture information conversing process system.
  • the predictive coefficient determining portion 263 shown in Fig. 9 more coefficients for individual classes are obtained. Predictive coefficients are selected in the order of the largest absolute value.
  • the horizontal period of the 525p signal generated by the estimation predictive calculating circuit 4 and 5 is the same as the horizontal period of the 525i signal of which the picture information converting process has not been performed.
  • the line sequence converting circuit 6 has a line memory and performs a line double speed process for doubling the horizontal period therewith.
  • Fig. 10 shows waveforms of analog signals of which the line double speed process is performed.
  • the estimation predictive calculating circuits 4 and 5 predict and generate the line data y1 and y2, respectively.
  • the line data y1 contains lines a1, a2, a3, and so forth in the order.
  • the line data y2 contains lines b1, b2, b3, and so forth in the order.
  • the line sequence converting circuit 6 has a line doubler and a switching circuit (not shown).
  • the line doubler compresses data of each line by 1/2 in the time axis direction and supplies the compressed data to the switching circuit.
  • the switching circuit alternately selects received data and outputs the selected data.
  • line sequence output data (a0, b0, a1, b1, and so forth) is formed.
  • an output pixel value (line data L1) of the current line and an output pixel value (line data L2) of a new line are generated at a time.
  • the line double speed process may be performed for the line data y1 and y2.
  • the arrangement of spatial class taps of the line data y1 is different from that of the line data y2 (the number of spatial class taps of the line data y1 is 5, whereas the number of spatial class taps of the line data y2 is 6).
  • the number of spatial class taps of the line data y1 may be the same as that of the line data y2.
  • Fig. 11 shows the arrangement of spatial class taps of the line data y1 and y2 according to the second embodiment of the present invention. The notation of Fig. 11 is the same as the notation of Fig. 3 .
  • the line data y1 and y2 of the field F/o As spatial class taps of the line data y1 and y2 of the field F/o, a total of six pixels - pixels T5 and T6 of the field F-1/e, pixels T2, T3, and T4 of the field F/o, and a pixel T6 of the field F/e are used.
  • Fig. 12 shows the state of which number of classes is decreased in the class value converting process.
  • the number of classes of each of the line data y1 and y2 is 256.
  • the number of classes of the line data y1 is smaller than that of the line data y2. This is because it is more difficult to predict and generate the line data y2 than the line data y1.
  • a spatial class detecting circuit 16 and a tap selecting circuit 15 are disposed.
  • the tap selecting circuit 15 selects a spatial class.
  • the tap selecting circuit 15 is disposed upstream of the spatial class detecting circuit 16.
  • a spatial class value that is output from the spatial class detecting circuit 16 and a motion class value that is output from a motion class detecting circuit 21 are supplied to a class combining circuit 17.
  • the class combining circuit 17 combines the spatial class value and the motion class value.
  • Output data of the class combining circuit 17 is supplied to class value converting circuits 18 and 19.
  • the class value converting circuits 18 and 19 perform a class value converting process for the output data of the class combining circuit 17 and generate converted class values of the line data y1 and y2, respectively.
  • the class value converting process is performed so that classes are obtained corresponding to the difficulty for predicting and generating the line data y1 and y2.
  • Output data of the class value converting circuits 18 and 19 is supplied to a coefficient memory 41.
  • the structure of the spatial class detecting portion for the line data y1 is in common with that for the line data y2, the structure of the apparatus can be simplified.
  • the class value converting process the number of classes for the line data y1 is more reduced than that for the line data y2.
  • the structure for detecting spatial classes for the line data y1 and the structure for detecting spatial classes for the line data y2 may be separately disposed as with the structure shown in Fig. 2 .
  • a 525i signal is converted into a 525p signal.
  • the number of scanning lines is not limited to 525.
  • an output picture signal composing lines having pixels that are not twice those in the horizontal direction of an input SD signal may be predicted and generated.
  • the present invention can be applied for a picture information converting process for predicting and generating an interlace picture signal (a 1050i signal) as an output picture signal.
  • Fig. 14 shows the arrangement of pixels of a 521 signal and a 1050i signal as a picture of one field.
  • large dots represent pixels of the 525i signal, whereas small dots represent pixels of the 1050i signal.
  • solid lines represent lines of an odd field of a particular frame, whereas doted lines represent another field (even field) of the frame. Each line of the even field deviates from each line of the odd field by 0.5 lines. On the lines of the even field, pixels of line data y1' and y2' are formed.
  • Fig. 15 shows the arrangement of spatial class taps in such a case.
  • the notation of Fig. 15 is the same as the notation of Fig. 3 .
  • spatial class taps for the line data y1 and y2 of the field F/o a total of seven pixels - pixels T4, T5, T6, and T7 of the field F-1/e and pixels T1, T2, and T3 of the field F/o are used.
  • a picture information converting process can be performed with the structure shown in Fig. 13 .
  • spatial class taps may be arranged in a different manner.
  • the arrangement of spatial class taps for the line data y1 may be different from that for the line data y2.
  • a plurality of input pixels in the horizontal direction may be used as spatial class taps.
  • Fig. 16 shows an example of the arrangement of motion class taps in the case shown in Fig. 15 .
  • pixels n1, n3, and n5 of the field F/o pixels n2, n4, and n6 of the field F/e
  • pixels m2, m4, and m6 of the field F-1/e pixels m1, m3, and m5 of the field F-1/o
  • the vertical positions of the pixels m1, m2, ..., and m6 match those of the pixels n1, n2, ..., and n6, respectively.
  • the arrangement of the motion class taps is not limited to that shown in Fig. 13 .
  • the number of classes determined corresponding to each of a plurality of'considered points at non-scanning-line positions of the input picture signal is decreased corresponding to the relation of positions of the considered points to the scanning lines of the input picture signal (for example, many classes are assigned to y2 that is difficult to predict and generate).
  • the present invention can be effectively applied for the case that the process is performed with a small number of classes.
  • classes can be properly assigned for each considered point corresponding to the difficulty (for example, y1 and y2) for predicting and generating pixels due to the difference between positions of a considered point and a scanning line of the input picture signal.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Graphics (AREA)
  • Television Systems (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Claims (9)

  1. Dispositif de conversion d'informations d'image destiné à générer un signal d'image de sortie avec une structure de lignes de balayage différente de celle d'un signal d'image d'entrée, le dispositif comprenant :
    un premier moyen de sélection de données d'image (10, 11) destiné à sélectionner, à partir du signal d'image d'entrée, des pixels adjacents avec des relations de positionnement prédéterminées à chacun d'une pluralité de points considérés, lesdites relations étant différentes en fonction de la relation entre la position du point considéré et une ligne de balayage du signal d'image d'entrée ;
    un moyen de détection de classe spatiale (12, 13) destiné à détecter un motif d'une distribution de niveaux pour des données d'image sélectionnées par ledit premier moyen de sélection de données d'image (10, 11), et à déterminer une première valeur de classe spatiale qui représente une classe spatiale de chaque point considéré correspondant au motif détecté ;
    un moyen de conversion de valeurs de classe (32, 33) destiné à mettre en oeuvre un processus de conversion de valeurs de classe visant à réduire le nombre de classes pour chaque dite première valeur de classe et générer une seconde valeur de classe correspondant à des données de coefficient de prédiction, dans lequel ledit moyen de conversion de valeurs de classe (32, 33) est exploitable de manière à mettre en oeuvre le processus de conversion de valeurs de classe en référence à une table de conversion de valeurs de classe générée par un processus d'apprentissage visant à comparer les distances de coefficients de prédiction de différentes classes à une valeur seuil prédéterminée et à intégrer une pluralité de classes dont la distance est inférieure à la valeur seuil dans une classe pour un signal d'image particulier dont le format de signal est le même que celui du signal d'image de sortie ;
    un moyen de stockage (41) destiné à stocker et délivrer en sortie des données de coefficient de prédiction prédéterminées correspondant aux secondes valeurs de classe ;
    un deuxième moyen de sélection de données d'image (1) destiné à sélectionner, à partir du signal d'image d'entrée, des pixels adjacents avec des relations de positionnement prédéterminées à chaque point considéré ; et
    un moyen de traitement de calcul (4, 5) destiné à mettre en oeuvre un processus de calcul visant à prédire et à générer des pixels à des positions ayant des relations de positionnement prédéterminées aux points considérés, sur la base des données d'image obtenues par ledit deuxième moyen de sélection de données d'image (1) et desdites données de coefficient de prédiction prédéterminées.
  2. Dispositif de conversion d'informations d'image selon la revendication 1, comprenant :
    un troisième moyen de sélection de données d'image (20) pour sélectionner des pixels adjacents ayant des relations de positionnement prédéterminées à chaque point considéré à partir d'une pluralité de trames du signal d'image d'entrée ;
    un moyen de détection de classe de mouvement (21) pour calculer la somme des valeurs absolues des différences de trames avec les données d'image sélectionnées par ledit troisième moyen de sélection de données d'image, et pour déterminer des valeurs de classe de mouvement correspondant au résultat calculé, les valeurs de classe de mouvement représentant des mouvements ; et
    un moyen de combinaison de classes (30, 31) pour combiner les valeurs de classe spatiale et les valeurs de classe de mouvement de manière à générer une première valeur de classe ;
    dans lequel ledit moyen de conversion de valeurs de classe (32, 33) est exploitable de manière à réduire le nombre de classes pour chaque première valeur de classe et à générer une seconde valeur de classe correspondant aux données de coefficient de prédiction.
  3. Dispositif de conversion d'informations d'image selon la revendication 1 ou 2, dans lequel le signal d'image d'entrée est un signal d'image entrelacée présentant 525 lignes de balayage, et le signal d'image de sortie est un signal d'image progressive présentant 525 lignes de balayage.
  4. Dispositif de conversion d'informations d'image selon la revendication 1 ou 2, dans lequel le signal d'image d'entrée est un signal d'image entrelacée présentant 525 lignes de balayage, et le signal d'image de sortie est un signal d'image progressive présentant 1 050 lignes de balayage.
  5. Récepteur de télévision intégrant un dispositif de conversion d'informations d'image selon l'une quelconque des revendications précédentes.
  6. Procédé de conversion d'informations d'image destiné à générer un signal d'image de sortie avec une structure de lignes de balayage différente de celle d'un signal d'image d'entrée, le procédé comprenant les étapes ci-dessous consistant à :
    (a) sélectionner (10, 11), à partir du signal d'image d'entrée, des pixels adjacents avec des relations de positionnement prédéterminées à chacun d'une pluralité de points considérés, lesdites relations étant différentes en fonction de la relation entre la position du point considéré et une ligne de balayage du signal d'image d'entrée ;
    (b) détecter (12, 13) un motif d'une distribution de niveaux pour des données d'image sélectionnées à l'étape (a), et déterminer une première valeur de classe spatiale qui représente une classe spatiale de chaque point considéré correspondant au motif détecté ;
    (c) mettre en oeuvre (32, 33) un processus de conversion de valeurs de classe visant à réduire le nombre de classes pour chaque dite première valeur de classe et générer une seconde valeur de classe correspondant à des données de coefficient de prédiction, dans lequel ledit processus de conversion de valeurs de classe de l'étape (c) est mis en oeuvre en référence à une table de conversion de valeurs de classe générée par un processus d'apprentissage visant à comparer les distances de coefficients de prédiction de différentes classes à une valeur seuil prédéterminée et à intégrer une pluralité de classes dont la distance est inférieure à la valeur seuil dans une classe pour un signal d'image particulier dont le format de signal est le même que celui du signal d'image de sortie ;
    (d) stocker (41) et délivrer en sortie des données de coefficient de prédiction prédéterminées correspondant aux secondes valeurs de classe ;
    (e) sélectionner (1), à partir du signal d'image d'entrée, des pixels adjacents ayant des relations de positionnement prédéterminées à chaque point considéré ; et
    (f) mettre en oeuvre (4, 5) un processus de calcul visant à prédire et générer des pixels à des positions présentant des relations de positionnement prédéterminées aux points considérés, sur la base des données d'image obtenues à l'étape (e) et desdites données de coefficient de prédiction prédéterminées.
  7. Procédé de conversion d'informations d'image selon la revendication 6, comprenant les étapes ci-dessous consistant à :
    (g) sélectionner (20) des pixels adjacents présentant des relations de positionnement prédéterminées à chaque point considéré à partir d'une pluralité de trames du signal d'image d'entrée ;
    (h) calculer (21), la somme des valeurs absolues des différences de trames avec les données d'image sélectionnées à l'étape (g) et déterminer des valeurs de classe de mouvement correspondant au résultat calculé, les valeurs de classe de mouvement représentant des mouvements ; et
    (i) combiner (30, 31) les valeurs de classe spatiale et les valeurs de classe de mouvement de manière à générer une première valeur de classe ;
    dans lequel le processus de conversion de valeurs de classe de l'étape (c) réduit le nombre de classes pour chaque première valeur de classe et génère une seconde valeur de classe correspondant aux données de coefficient de prédiction.
  8. Procédé de conversion d'informations d'image selon la revendication 6 ou 7, dans lequel le signal d'image d'entrée est un signal d'image entrelacée présentant 525 lignes de balayage, et le signal d'image de sortie est un signal d'image progressive présentant 525 lignes de balayage.
  9. Procédé de conversion d'informations d'image selon la revendication 6 ou 7, dans lequel le signal d'image d'entrée est un signal d'image entrelacée présentant 525 lignes de balayage, et le signal d'image de sortie est un signal d'image progressive présentant 1 050 lignes de balayage.
EP19990306636 1998-08-24 1999-08-20 Conversion de données d'images et processeur vidéo Expired - Lifetime EP0989748B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP23731398 1998-08-24
JP23731398A JP4147632B2 (ja) 1998-08-24 1998-08-24 画像情報変換装置、画像情報変換方法、およびテレビジョン受像機

Publications (3)

Publication Number Publication Date
EP0989748A2 EP0989748A2 (fr) 2000-03-29
EP0989748A3 EP0989748A3 (fr) 2001-09-12
EP0989748B1 true EP0989748B1 (fr) 2012-10-03

Family

ID=17013524

Family Applications (1)

Application Number Title Priority Date Filing Date
EP19990306636 Expired - Lifetime EP0989748B1 (fr) 1998-08-24 1999-08-20 Conversion de données d'images et processeur vidéo

Country Status (3)

Country Link
US (1) US6483546B1 (fr)
EP (1) EP0989748B1 (fr)
JP (1) JP4147632B2 (fr)

Families Citing this family (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2000065830A1 (fr) * 1999-04-23 2000-11-02 Sony Corporation Dispositif et procede de conversion d'image
EP1197946B1 (fr) * 2000-02-10 2007-04-04 Sony Corporation Dispositif et procede de traitement d'images et support d'enregistrement
JP4670169B2 (ja) 2000-11-15 2011-04-13 ソニー株式会社 情報信号処理装置、情報信号処理方法、画像信号処理装置およびそれを使用した画像表示装置、それに使用される係数種データ生成装置、並びに情報記録媒体
JP4696388B2 (ja) 2000-11-15 2011-06-08 ソニー株式会社 情報信号処理装置、情報信号処理方法、画像信号処理装置およびそれを使用した画像表示装置、それに使用される係数種データ生成装置、係数データ生成装置、並びに情報記録媒体
US7095445B2 (en) * 2000-12-20 2006-08-22 Samsung Electronics Co., Ltd. Method of detecting motion in an interlaced video sequence based on logical operation on linearly scaled motion information and motion detection apparatus
JP4517264B2 (ja) * 2000-12-26 2010-08-04 ソニー株式会社 情報信号処理装置、情報信号処理方法、画像信号処理装置およびそれを使用した画像表示装置、並びに情報提供媒体
DE60234965D1 (de) * 2001-01-22 2010-02-11 Sony Corp Koeffizientengenerierungsvorrichtung und -verfahren zur Umwandlung von Informationssignalen
EP1871107A3 (fr) * 2002-01-30 2009-12-09 Sony Corporation Dispositif de production de données de type coefficient et données à coefficient utilisé dans un appareil d'affichage d'image
JP4534422B2 (ja) * 2002-03-08 2010-09-01 ソニー株式会社 半導体装置
US7865065B2 (en) 2002-03-08 2011-01-04 Sony Corporation Apparatus and method of recording information on a recording medium
US7689105B2 (en) 2002-03-08 2010-03-30 Sony Corporation Record medium and its reproducer, reproducing method, record medium manufacturing apparatus, record medium manufacturing method, and recorder
US7777321B2 (en) * 2002-04-22 2010-08-17 Gann Keith D Stacked microelectronic layer and module with three-axis channel T-connects
JP4055203B2 (ja) 2002-09-12 2008-03-05 ソニー株式会社 データ処理装置およびデータ処理方法、記録媒体、並びにプログラム
JP4055655B2 (ja) 2003-05-29 2008-03-05 ソニー株式会社 係数の生成装置および生成方法、クラス構成の生成装置および生成方法、情報信号処理装置、並びに各方法を実行するためのプログラム
JP3885786B2 (ja) 2003-09-12 2007-02-28 ソニー株式会社 情報信号の処理装置および処理方法、並びにその処理方法を実行させるためのプログラム
US7542095B2 (en) * 2005-01-20 2009-06-02 Samsung Electronics Co., Ltd. Method and system of noise-adaptive motion detection in an interlaced video sequence

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4941045A (en) * 1988-10-11 1990-07-10 Scientific-Atlanta, Inc. Method and apparatus for improving vertical definition of a television signal by scan conversion
JP2830111B2 (ja) 1989-07-21 1998-12-02 ソニー株式会社 高能率符号化装置
JP3271108B2 (ja) * 1993-12-03 2002-04-02 ソニー株式会社 ディジタル画像信号の処理装置および方法
KR100248452B1 (ko) 1994-09-09 2000-03-15 이데이 노부유끼 디지털 신호 처리용 집적 회로
JP3794505B2 (ja) 1995-03-22 2006-07-05 ソニー株式会社 信号変換装置及び信号変換方法
JP3669530B2 (ja) * 1995-06-30 2005-07-06 ソニー株式会社 画像信号変換装置及び画像信号変換方法
US5946044A (en) * 1995-06-30 1999-08-31 Sony Corporation Image signal converting method and image signal converting apparatus
WO1997046014A1 (fr) 1996-05-30 1997-12-04 Sony Corporation Dispositif et procede de conversion d'informations d'images, et circuit et procede permettant d'effectuer la somme de produits
KR100504976B1 (ko) 1996-12-11 2005-11-29 소니 가부시끼 가이샤 신호변환장치및방법
US6330344B1 (en) 1997-02-14 2001-12-11 Sony Corporation Image processing device and method employing motion detection to generate improved quality image from low resolution image

Also Published As

Publication number Publication date
JP2000069435A (ja) 2000-03-03
EP0989748A2 (fr) 2000-03-29
JP4147632B2 (ja) 2008-09-10
US6483546B1 (en) 2002-11-19
EP0989748A3 (fr) 2001-09-12

Similar Documents

Publication Publication Date Title
EP0975156B1 (fr) Appareil de traitement de signal d'information, appareil de conversion d'information d'image et appareil d'affichage d'image
EP0989748B1 (fr) Conversion de données d'images et processeur vidéo
EP0571180B1 (fr) Equipement de conversion de données digitales
US6597737B1 (en) Motion determining apparatus, method thereof, and picture information converting apparatus
US6201833B1 (en) Motion determining apparatus, method thereof, and picture information converting apparatus
US6433828B1 (en) Picture conversion using field-by-field vertical inversion dependent upon the type of picture signal to be outputted
JP4300603B2 (ja) 画像情報変換装置および方法
JP3723995B2 (ja) 画像情報変換装置および方法
US6707502B1 (en) Apparatus and method for converting a field frequency of a picture signal
JP4099555B2 (ja) 画像情報変換装置
JP4123587B2 (ja) 画像情報処理装置および方法
JP4078719B2 (ja) 画像情報変換装置、変換方法および画像表示装置
JP3800638B2 (ja) 画像情報変換装置および方法
JP4193236B2 (ja) 画像情報変換装置、画像情報変換方法、およびテレビジョン受像機
JP3587188B2 (ja) ディジタル画像信号処理装置および処理方法
JP2000244879A (ja) 画像情報変換装置および方法
JP4154768B2 (ja) 画像情報変換装置および画像表示装置
JP2000092454A (ja) 画像情報変換装置および画像情報変換方法
JPH07193790A (ja) 画像情報変換装置
JP2002223420A (ja) 画像処理装置および画像処理方法
JPH07193791A (ja) 画像情報変換装置
JP2000083223A (ja) 画像情報変換装置および画像情報変換方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

Kind code of ref document: A2

Designated state(s): DE FR GB

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AT BE CH CY DE DK ES FI FR GB GR IE IT LI LU MC NL PT SE

AX Request for extension of the european patent

Free format text: AL;LT;LV;MK;RO;SI

17P Request for examination filed

Effective date: 20020227

AKX Designation fees paid

Free format text: DE FR GB

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): DE FR GB

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 69944419

Country of ref document: DE

Effective date: 20121129

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20130704

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 69944419

Country of ref document: DE

Effective date: 20130704

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20130820

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20140301

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 69944419

Country of ref document: DE

Effective date: 20140301

REG Reference to a national code

Ref country code: FR

Ref legal event code: ST

Effective date: 20140430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130820

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20130902