EP2648409B1 - Dispositif et procédé de commande de quantification et programme de commande de quantification - Google Patents

Dispositif et procédé de commande de quantification et programme de commande de quantification Download PDF

Info

Publication number
EP2648409B1
EP2648409B1 EP12754501.0A EP12754501A EP2648409B1 EP 2648409 B1 EP2648409 B1 EP 2648409B1 EP 12754501 A EP12754501 A EP 12754501A EP 2648409 B1 EP2648409 B1 EP 2648409B1
Authority
EP
European Patent Office
Prior art keywords
intra slice
picture
slice region
similarity
intra
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
EP12754501.0A
Other languages
German (de)
English (en)
Other versions
EP2648409A4 (fr
EP2648409A1 (fr
Inventor
Masaki Kitahara
Atsushi Shimizu
Naoki Ono
Seisuke Kyochi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Telegraph and Telephone Corp
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Publication of EP2648409A1 publication Critical patent/EP2648409A1/fr
Publication of EP2648409A4 publication Critical patent/EP2648409A4/fr
Application granted granted Critical
Publication of EP2648409B1 publication Critical patent/EP2648409B1/fr
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques

Definitions

  • the present invention relates to a quantization control apparatus, a quantization control method, and a quantization control program, which are used in video encoding that realizes low delay by employing intra slices.
  • delay indicates a time measured from when a video sending side of a system acquires a picture using a camera or the like, to when a video decoding side of the system displays this acquired picture.
  • a "picture” denotes a frame or a field in input video.
  • a video coding technique is applied so as to reduce the bandwidth frequency required for video transmission.
  • video coding technique mainly affects (i) time required for encoding executed on the video sending side, (ii) time required for buffering of encoded data on the video receiving side, and (iii) time required for decoding of the encoded data on the video receiving side.
  • speed for these processes should be increased.
  • employment of intra slices (explained later) and a quantization control technique suitable therefor is effective.
  • an intra-type picture is inserted regularly for a purpose of refreshing.
  • the intra-type picture has a very large amount of code, which requires a long buffering time for the encoded data on the video receiving side, thereby the delay time is increased.
  • the video encoding that employs intra slices realizes refreshing by moving, in each picture, a band in a longitudinal or lateral direction, instead of using intra pictures, where the band forcibly subjects all macroblocks in the picture to intra encoding.
  • inter encoding may be performed in each region (in the picture) other than the intra slices, where such a region will be called a "non intra slice region".
  • the amount of code generated for each picture can almost be constant, thereby the buffering time can be reduced.
  • intra slice When arranging the above-described band in a longitudinal direction, it may not be called the “intra slice” (and may be called an “intra column”). However, for convenience of explanation, the present specification uses "intra slice” for either case of moving the band in a longitudinal or lateral direction.
  • Fig. 9 is a diagram showing a concept of a case that provides an intra slice arranged in a longitudinal direction.
  • the intra slice moves from the left end to the right end in the relevant picture, where time for the number of pictures among which the position of the intra slice moves in one period (i.e., the number of pictures for which the intra slice moves from the left end to the right end for the pictures) is called an "intra slice period".
  • the amount of generated code considerably differ between the intra slice region and the non intra slice region. Therefore, a quantization control method that can allocate an appropriate amount code to each region is required.
  • Patent-Document 1 discloses a technique that implements such a technique.
  • encoding information for a non intra slice region in the picture immediately before the encoding target picture is used to compute complexity index Xp for the relevant region.
  • a target amount T of code for the encoding target picture is divided into a target amount Ti of code for the intra slice region and a target amount Tp of code for the non intra slice region. Then, based on Ti and Tp, the quantization step for each macroblock in the picture is determined.
  • the intra slice period is set to 0.5 sec or longer.
  • the area of the intra slice becomes considerably small. For example, if the intra slice period is 0.5 sec and 30 pictures are present in each second, then the area of the intra slice region is one-fourteenth of the area of the non intra slice region.
  • the property of video may considerably differ between the intra slice in an encoding target picture and the intra slice in a picture immediately before the encoding target picture.
  • the complexity index Xi is not accurate, which degrades the image quality.
  • Patent-Document 2 may be combined with the technique of Patent-Document 1.
  • Patent-Document 2 is not applied to a case using the intra slice, it corrects a complexity index of the encoding target picture when computing the complexity index by using encoding information of a picture immediately before the encoding target picture, where the correction is performed based on activity measures of the encoding target picture and the picture immediately before.
  • the activity measure represents a characteristic feature of each image in a manner such that the finer (i.e., including edge portions) the texture of each target block, the larger the activity measure.
  • the relevant technique is effective when the property slightly differs between the encoding target picture and the picture immediately before, for example, during a fade-in or fade-out period.
  • Document WO2006099229 discloses a scheme for quantization parameter prediction, predicting the quantization parameter of a current inter block from the quantization parameter of a reference picture, which is referenced by a motion vector; with respect to intra blocks, it is proposed to predict the quantization parameter of a current block from its spatially neighboring blocks.
  • Document US2008304562 discloses a scheme for quantization parameter prediction, coding a delta quantization parameter of a current B or P picture, based on an initial quantization step, and based on the temporal and spatial complexity of a group of frames.
  • Such a phenomenon often occurs in an encoding method using intra prediction, such as H.264.
  • prediction is performed using decoded neighbor pixels of an encoding target block together with switching between a plurality of prediction models such as multidirectional extrapolation or mean value prediction. That is, in a processing target picture, highly accurate prediction can be performed in a region that has a texture which coincides with the relevant prediction model, thereby the amount of generated code is small.
  • an object of the present invention is to provide a quantization control apparatus, a quantization control method, and a quantization control program in video encoding, which implement low delay by utilizing the intra slices.
  • the present invention provides a quantization control apparatus provided in a video encoding apparatus which utilizes intra slices, the quantization control apparatus comprising:
  • the quantization control apparatus further comprises a threshold determination device that determines whether or not the similarity of the picture selected by the picture selection device is higher than a predetermined threshold, wherein according to a result of the determination performed by the threshold determination device:
  • the similarity computation device that computes the similarity based on a difference absolute value for a total of activities for respective blocks of each intra slice.
  • those which each have an intra slice whose position on an image plane is relatively close to that of the intra slice of the encoding target picture are subjected to the similarity computation by the similarity computation device.
  • the present invention also provides a quantization control method of controlling quantization in a quantization control apparatus provided in a video encoding apparatus which utilizes intra slices, the quantization control method comprising:
  • the quantization control method further comprises a threshold determination step that determines whether or not the similarity of the picture selected by the picture selection step is higher than a predetermined threshold, wherein according to a result of the determination performed by the threshold determination step:
  • the similarity computation step that computes the similarity based on a difference absolute value for a total of activities for respective blocks of each intra slice.
  • those which each have an intra slice whose position on an image plane is relatively close to that of the intra slice of the encoding target picture are subjected to the similarity computation by the similarity computation step.
  • the present invention also provides a quantization control program of making a computer, that is provided at a video encoding apparatus which utilizes intra slices, execute a quantization control operation, the operation including:
  • the present invention has a distinctive feature such that computation of a complexity index for an intra slice of an encoding target picture can utilize a previously-encoded intra slice whose property is akin to that of the intra slice of the encoding target picture, thereby the image quality is improved.
  • Fig. 1 is a block diagram showing the structure of the present embodiment.
  • the video encoding apparatus 1 shown in Fig. 1 has a structure that generates encoded data based on H.264, and outputs encoded data by receiving an input picture and an input picture number.
  • a quantization control unit 2 has a distinctive feature in comparison with a corresponding known structure.
  • Fig. 2 shows the structure of the quantization control unit 2 in Fig. 1 .
  • the quantization control unit 2 of the present embodiment has a similarity computation unit 21, a picture selection unit 22, an allocated code amount determination unit 23, and a quantization step determination unit 24.
  • FIG. 1 a basic operation of the video encoding apparatus 1 in Fig. 1 will be explained.
  • the video encoding apparatus 1 Since the video encoding apparatus 1 generates encoded data based on H.264, many processes are each performed on a block basis.
  • a difference from a predicted image block, that is output from an intra prediction unit 3 or an inter prediction unit 4 is computed.
  • a difference image block formed by the computed difference is input into an orthogonal transformation and quantization unit 5, which subjects the difference image block to orthogonal transformation, and quantizes orthogonal transformation coefficients. Obtained quantized values of the orthogonal transformation coefficients are subjected, by a variable length encoding unit 6, to variable length encoding.
  • the quantized orthogonal transformation coefficients are also input into an inverse quantization and inverse orthogonal transformation unit 7, which subjects the quantized orthogonal transformation coefficients to inverse quantization and also subjects the inverse-quantized transformation coefficients to inverse orthogonal transformation.
  • the quantization and the inverse quantization are performed using the quantization step (value), that is determined for each block and output from the quantization control unit 2. The operation of the quantization control will be explained later.
  • the inverse-orthogonal-transformed block is input into the intra prediction unit 3 so as to be used in intra prediction of another block.
  • the intra prediction unit 3 subjects the encoding target block in the input picture to intra prediction using pixels in the block that is output from the inverse quantization and inverse orthogonal transformation unit 7.
  • the block output from the inverse quantization and inverse orthogonal transformation unit 7 is also input into a loop filtering unit 8.
  • the loop filtering unit 8 subjects the blocks to a loop filtering operation (i.e., filtering operation within an encoding loop).
  • a loop filtering operation i.e., filtering operation within an encoding loop.
  • Such a picture to which the loop filtering operation has been applied equals a decoded picture obtained on the decoding side.
  • the decoded picture obtained here is stored in a decoded picture memory 9.
  • the inter prediction unit 4 sets the decoded picture stored in the decoded picture memory 9 to a reference image in the inter prediction of the encoding target block in the input picture.
  • An intra slice control unit 10 determines at which position an intra slice is inserted and what size the width of the intra slice is (i.e., what blocks of the picture are included in the intra slice).
  • an input picture number that indicates the input order of the input pictures is used.
  • An intra/inter switch 11 is informed of whether or not the encoding target block is included in the intra slice.
  • the intra/inter switch 11 always chooses the intra prediction. Instead, if the encoding target block is not included in the intra slice, the intra/inter switch 11 always chooses the inter prediction.
  • the switching between the intra prediction and the inter prediction may be performed in accordance with prediction efficiency.
  • the inter prediction is always chosen here.
  • Fig. 4 shows an example of the intra slice insertion utilized in the present embodiment.
  • the intra slice period is formed by six pictures, and the intra slice in each picture has the same width.
  • all pictures except for the head picture are each encoded as a "P picture". That is, unlike in the case of including B pictures, encoding is executed in the order of inputting pictures.
  • the quantization control unit 2 receives an encoding target picture and a plurality of previously-encoded pictures (which are not decoded images but original images).
  • previously-encoded pictures within one intra slice period i.e., six pictures
  • the similarity computation unit 21 also receives intra slice information that designates at which position the intra slice is present in each picture.
  • the similarity is an index value that indicates a degree of similarity between images of two different intra slice regions.
  • the similarity may utilize the sum of absolute differences (called "SAD” below), where each difference is between corresponding pixel values (of the compared intra slices) which have the same relative coordinates. In this case, the smaller the value of C(i), the closer the two intra slices are to each other.
  • the picture selection unit 22 determines a previously-encoded picture "i_select" that has the highest similarity for the intra slice (see step S2). Specifically, C(i_select) that has the smallest SAD is obtained. Then, the previously-encoded picture i-select is determined to be a selected picture, and informs the allocated code amount determination unit 23 of the index value "i_select" (selected picture information) of the selected picture.
  • the allocated code amount determination unit 23 computes complexity indexes for the intra slice region and the non intra slice region.
  • the complexity index Xi for the intra slice region is computed (see step S4). Specifically, Xi is computed utilizing the encoding information (the amount of generated code and the average quantization step) for the intra slice region of the previously-encoded picture i_select (the selected picture) selected by the picture selection unit 22.
  • the allocated code amount determination unit 23 has received encoding information for the intra slice regions of the above-described previously-encoded pictures.
  • the quantization step determination unit 24 computes respective amounts Ti and Tp of code allocated to the intra slice region and the non intra slice region in the encoding target picture, and further computes the quantization step for each block by using the allocated amounts of code.
  • This process may be performed by a known method.
  • This formula has a known premise (see, for example, Patent-Document 1), and a detailed explanation will be omitted.
  • an amount of code allocated to each block in the intra slice region is determined based on Ti
  • an amount of code allocated to each block in the non intra slice region is determined based on Tp
  • the quantization step for each block is computed (see step S5).
  • the accuracy of the estimated complexity index is low.
  • similarities for the intra slice between the encoding target picture and a plurality of previously-encoded pictures are computed so as to select a previously-encoded picture utilized to compute the complexity index. Accordingly, encoding information for a previously-encoded intra slice region that has a texture akin to that of the intra slice region in the encoding target picture can be utilized. Therefore, a higher accuracy of the complexity index in comparison with the known method can be obtained, which implements code amount allocation conformed to the property of the intra slice region and higher image quality.
  • FIG. 5 a video encoding apparatus in accordance with a second embodiment of the present invention will be explained.
  • steps identical to those in Fig. 3 are given identical reference signs, and explanations thereof are omitted.
  • the operation shown in Fig. 5 has a distinctive feature of further having steps S6 and S7.
  • the similarity of the selected picture is compared with a predetermined threshold, so as to determine whether or not the similarity is higher than the threshold (see step 56).
  • the similarity C(i) is SAD as described above, the determination "whether or not the similarity is higher than a predetermined threshold" is actually "whether or not C(i_select) is smaller than a predetermined threshold”.
  • the index i_select of the selected picture is communicated to the allocated code amount determination unit 23, thereby steps S4 and S5 are executed as in the operation shown in Fig. 3 .
  • the picture selection unit 22 supplies null data to the allocated code amount determination unit 23.
  • the allocated code amount determination unit 23 initializes the complexity index Xi by a predetermined initial value (see step S7).
  • An initial value employed in MPEG-2 TM5 is a representative example therefor.
  • the intra slices of the selected picture and the encoding target picture have completely different complexities. For example, if the texture of the intra slice in the encoding target picture is extremely complex in comparison with the intra slice of the selected picture, the relevant allocated amount of code is small, which degrades the image quality. Therefore, when the intra slice of the selected picture has a low similarity, the complexity index is initialized using a predetermined initial value, thereby an extreme degradation in the image quality can be prevented.
  • FIG. 6 a video encoding apparatus in accordance with a third embodiment of the present invention will be explained.
  • steps identical to those in Fig. 3 are given identical reference signs, and explanations thereof are omitted.
  • the similarity computation unit 21 computes the similarity using the above-described activity (measure) of each picture.
  • total activity the total of activity measures (called “total activity") for the blocks in the relevant intra slice is computed, and a difference absolute value between the total activity for the intra slice of the encoding target picture and the total activity for the intra slice of each previously-encoded picture is determined to be a similarity (see step S11).
  • each block is computed by dividing the block into four sub-blocks, computing a variance of pixel values in each sub-block, and determining the minimum value in the four variances to be the activity.
  • the difference absolute values for the total activity are communicated to the picture selection unit 22.
  • the picture selection unit 22 determines a previously-encoded picture, that has the smallest difference absolute value for the total activity, to be the selected picture, and informs the allocated code amount determination unit 23 of the index "i_select" of the selected picture (see step S21).
  • the total sum of difference absolute values, each computed between activities in compared blocks may be determined to be a similarity.
  • the blocks j in the intra slices A and B have the same relative position within the intra slice.
  • the activity may be computed for each intra slice, and a difference absolute value between the activities of the relevant two intra slices may be determined to be the similarity.
  • the activity for each intra slice is obtained by dividing the intra slice into a plurality of blocks (e.g., four blocks having equal areas), computing a variance of pixel values in each block, and determining the minimum value in the variances of the blocks to be the activity of the relevant intra slice.
  • the relevant two intra slices include the same object, they have similar total values for the activities of the blocks in the intra slice. Therefore, the difference absolute value for the total of activities for the respective blocks is effectively utilized as a measure for the similarity.
  • the activity determined on a block basis is effective for mode determination executed other than the quantization control, and is known as shown in Patent Document 1. That is, when an activity has been computed for mode determination or the like, if the activity is also utilized for the similarity computation in the present invention, the amount of computation can be substantially reduced in comparison with SAD. A typical example will be shown below.
  • the amount of computation for SAD between two intra slices for one picture is determined in consideration that the number of times for the product-sum operation is 2N-1 and the number of times for the absolute value computation is N.
  • the activity on for each block the size of each block is assumed to be 16x16 pixels
  • the amount of computation required for the difference absolute value for the total of the activities for the blocks in the intra slice of one picture is determined in consideration that the number of times for the product-sum operation is N/256 and the number of times for the absolute value computation is 1.
  • the amount of computation can be reduced while securing required image quality, in comparison with a case in which a measure that utilizes the difference on a pixel basis (e.g., SAD) is employed as the similarity.
  • a measure that utilizes the difference on a pixel basis e.g., SAD
  • FIG. 7 a video encoding apparatus in accordance with a fourth embodiment of the present invention will be explained.
  • steps identical to those in Fig. 3 are given identical reference signs, and explanations thereof are omitted.
  • the operation shown in Fig. 7 has a distinctive feature of replacing step S1 in Fig. 3 with step S12 in Fig. 6 .
  • a difference absolute value between the coordinate value (on the image plane) of the intra slice in the encoding target picture and the coordinate value (on the image plane) of the intra slice in each previously-encoded picture is computed, and the intra slice of each previously-encoded picture, whose difference absolute value is smaller than or equal to a predetermined value, is defined as an adjacent intra slice.
  • the coordinate value of the intra slice, that is determined on the image plane may be set to the coordinate value of a midpoint between the left end and the right end of the intra slice in the horizontal direction on the image plane.
  • the intra slices of the previously-encoded pictures as shown in Fig. 8 are selected. That is, the shown example selects a previously-encoded picture immediately before the encoding target picture (i.e., "A" in Fig. 8 having a left-hand neighbor intra slice on the image plane), an earlier picture than the encoding target picture by one intra slice period (i.e., "B” in Fig. 8 having an intra slice at the same position on the image plane), and the next picture to the earlier picture than the encoding target picture by one intra slice period (i.e., "C" in Fig. 8 having a right-hand neighbor intra slice on the image plane).
  • a previously-encoded picture immediately before the encoding target picture i.e., "A" in Fig. 8 having a left-hand neighbor intra slice on the image plane
  • an earlier picture than the encoding target picture by one intra slice period i.e., "B” in Fig. 8 having an intra slice at the same position on the image plane
  • the similarity between the intra slice of the encoding target picture and the intra slice of each selected picture is computed and communicated to the picture selection unit 22 (see step S12).
  • an application e.g., videophone
  • intra slices generally handles a small amount of motion, and thus the possibility that an object included in the intra slice of the encoding target picture is also included in a previously-encoded intra slice, which is spatially close to the intra slice (of the encoding target picture) on the image plane, is high.
  • the similarity computation is applied only to the previously-encoded intra slices whose positions on the image plane are spatially close to the position of the encoding target intra slice. Therefore, it is possible to reduce the amount of computation while securing required image quality.
  • the intra slice period is 0.5 second, and all past pictures within one intra slice period are subjected to the similarity computation, each of 15 past pictures should be subjected to the similarity computation.
  • the similarity computation is limitedly applied to (i) a past picture that has an intra slice at the same position as that of the encoding target picture (i.e., earlier picture by one intra slice period) and (ii) past pictures that have left-hand and right-hand intra slices from that of the encoding target picture, three past pictures are each subjected to the similarity computation, which requires an amount of computation one fifth of that required for the above typical example.
  • the present invention computes a similarity between the encoding target picture and each of past previously-encoded pictures, and computes a complexity index for the intra slice of the encoding target picture by utilizing encoding information produced when a previously-encoded picture having the highest similarity (for the intra slice) was encoded.
  • a program for executing the functions of the units shown in Fig. 2 may be stored in a computer readable storage medium, and the program stored in the storage medium may be loaded and executed on a computer system, so as to perform the quantization control operation.
  • the computer system has hardware resources which may include an OS and peripheral devices.
  • the above computer readable storage medium is a storage device, for example, a portable medium such as a flexible disk, a magneto optical disk, a ROM, or a CD-ROM, or a memory device such as a hard disk built in a computer system.
  • the computer readable storage medium also includes a device for temporarily storing the program, such as a volatile memory (RAM) in a computer system which functions as a server or client and receives the program via a network (e.g., the Internet) or a communication line (e.g., a telephone line).
  • RAM volatile memory
  • the above program stored in a memory device or the like of a computer system, may be transmitted via a transmission medium or by using transmitted waves passing through a transmission medium to another computer system.
  • the transmission medium for transmitting the program has a function of transmitting data, and is, for example, a (communication) network such as the Internet or a communication line such (e.g., a telephone line).
  • the program may execute a part of the above-explained functions.
  • the program may also be a "differential" program so that the above-described functions can be executed by a combination program of the differential program and an existing program which has already been stored in the relevant computer system.
  • the present invention can be applied to purposes which should inevitably perform quantization control.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Claims (9)

  1. Appareil de commande de quantification fourni dans un appareil de codage vidéo qui utilise des intratranches, dans lequel une image cible de codage comprend au moins une zone d'intratranches et au moins une zone sans intratranches, dans lequel l'appareil de commande de quantification est caractérisé par :
    - un dispositif de calcul informatique de similarité (21) qui calcule informatiquement une similarité entre une zone d'intratranches de l'image cible de codage et une zone d'intratranches de chacune des images codées précédemment, où la similarité correspond à un indice déterminé de sorte que plus les images des deux intratranches distinctes sont similaires, plus la similarité est élevée ;
    - un dispositif de sélection d'image (22) qui sélectionne une image présentant la similarité la plus élevée parmi les images codées précédemment, sur la base de la similarité calculée informatiquement par le dispositif de calcul informatique de similarité ; et
    - un dispositif de détermination d'échelon de quantification (23, 24) qui :
    o calcule :
    ▪ une complexité pour la zone d'intratranches, en utilisant une quantité de code générée pour la zone d'intratranches de l'image sélectionnée par le dispositif de sélection d'image et un échelon de quantification moyen pour la zone d'intratranches de cette image ;
    ▪ une complexité pour la zone sans intratranches, en utilisant une quantité de code générée pour la zone sans intratranches de l'image codée immédiatement avant l'image cible de codage et un échelon de quantification moyen pour la zone sans intratranches de cette image ;
    ▪ des quantités de code respectives affectées à la zone d'intratranches de l'image cible de codage et à la zone sans intratranches de l'image cible de codage, en utilisant la complexité calculée informatiquement pour la zone d'intratranches et la complexité calculée informatiquement pour la zone sans intratranches ;
    o détermine un échelon de quantification de la zone d'intratranches sur la base de la quantité de code affectée à la zone d'intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone d'intratranches de l'image cible de codage ; et
    o détermine un échelon de quantification de la zone sans intratranches sur la base de la quantité de code affectée à la zone sans intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone sans intratranches de l'image cible de codage.
  2. Appareil de commande de quantification selon la revendication 1, comprenant en outre :
    - un dispositif de détermination de seuil qui détermine si la similarité de l'image sélectionnée par le dispositif de sélection d'image est supérieure ou non à un seuil prédéterminé ;
    dans lequel, selon un résultat de la détermination mise en oeuvre par le dispositif de détermination de seuil :
    si la similarité est supérieure au seuil prédéterminé, le dispositif de détermination d'échelon de quantification exécute la détermination des échelons de quantification de la zone d'intratranches et de la zone sans intratranches ; et
    si la similarité est inférieure ou égale au seuil prédéterminé, le dispositif de détermination d'échelon de quantification détermine chacun des échelons de quantification sur la base d'une valeur initiale prédéterminée, sans utiliser l'image sélectionnée.
  3. Appareil de commande de quantification selon la revendication 1, dans lequel :
    le dispositif de calcul informatique de similarité calcule informatiquement la similarité sur la base d'une valeur absolue de différence pour un total des activités pour des blocs respectifs de chaque intratranche.
  4. Appareil de commande de quantification selon la revendication 1, dans lequel :
    parmi les images codées précédemment, celles qui présentent individuellement une intratranche dont la position sur un plan d'image est relativement proche de celle de l'intratranche de l'image cible de codage sont soumises au calcul informatique de similarité par le dispositif de calcul informatique de similarité.
  5. Procédé de commande de quantification destiné à commander une quantification dans un appareil de commande de quantification fourni dans un appareil de codage vidéo qui utilise des intratranches, dans lequel une image cible de codage comprend au moins une zone d'intratranches et au moins une zone sans intratranches, dans lequel le procédé de commande de quantification est caractérisé par :
    - une étape de calcul informatique de similarité (S1) qui calcule informatiquement une similarité entre une zone d'intratranches de l'image cible de codage et une zone d'intratranches de chacune des images codées précédemment, où la similarité correspond à un indice déterminé de sorte que plus les images des deux intratranches distinctes sont similaires, plus la similarité est élevée ;
    - une étape de sélection d'image (S2) qui sélectionne une image présentant la similarité la plus élevée parmi les images codées précédemment, sur la base de la similarité calculée informatiquement au cours de l'étape de calcul informatique de similarité ; et
    - une étape de détermination d'échelon de quantification (S3 - S5) qui :
    o calcule :
    ▪ une complexité pour la zone d'intratranches, en utilisant une quantité de code générée pour la zone d'intratranches de l'image sélectionnée au cours de l'étape de sélection d'image et un échelon de quantification moyen pour la zone d'intratranches de cette image ;
    ▪ une complexité pour la zone sans intratranches, en utilisant une quantité de code générée pour la zone sans intratranches de l'image codée immédiatement avant l'image cible de codage et un échelon de quantification moyen pour la zone sans intratranches de cette image ;
    ▪ des quantités de code respectives affectées à la zone d'intratranches de l'image cible de codage et à la zone sans intratranches de l'image cible de codage, en utilisant la complexité calculée informatiquement pour la zone d'intratranches et la complexité calculée informatiquement pour la zone sans intratranches ;
    o détermine un échelon de quantification de la zone d'intratranches sur la base de la quantité de code affectée à la zone d'intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone d'intratranches de l'image cible de codage ; et
    o détermine un échelon de quantification de la zone sans intratranches sur la base de la quantité de code affectée à la zone sans intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone sans intratranches de l'image cible de codage.
  6. Procédé de commande de quantification selon la revendication 5, comprenant en outre :
    une étape de détermination de seuil qui détermine si la similarité de l'image sélectionnée au cours de l'étape de sélection d'image est supérieure ou non à un seuil prédéterminé ;
    dans lequel, selon un résultat de la détermination mise en oeuvre au cours de l'étape de détermination de seuil :
    si la similarité est supérieure au seuil prédéterminé, l'étape de détermination d'échelon de quantification exécute la détermination des échelons de quantification de la zone d'intratranches et de la zone sans intratranches ; et
    si la similarité est inférieure ou égale au seuil prédéterminé, l'étape de détermination d'échelon de quantification détermine chacun des échelons de quantification sur la base d'une valeur initiale prédéterminée, sans utiliser l'image sélectionnée.
  7. Procédé de commande de quantification selon la revendication 5, dans lequel :
    l'étape de calcul informatique de similarité calcule la similarité sur la base d'une valeur absolue de différence pour un total des activités pour des blocs respectifs de chaque intratranche.
  8. Procédé de commande de quantification selon la revendication 5, dans lequel :
    parmi les images codées précédemment, celles qui présentent individuellement une intratranche dont la position sur un plan d'image est relativement proche de celle de l'intratranche de l'image cible de codage sont soumises au calcul informatique de similarité au cours de l'étape de calcul informatique de similarité.
  9. Programme de commande de quantification destiné à amener un ordinateur, qui est fourni au niveau d'un appareil de codage vidéo lequel utilise des intratranches, à exécuter une opération de commande de quantification, dans lequel une image cible de codage comprend au moins une zone d'intratranches et au moins une zone sans intratranches, et dans lequel l'opération est caractérisée par :
    - une étape de calcul informatique de similarité (S1) qui calcule informatiquement une similarité entre une zone d'intratranches de l'image cible de codage et une zone d'intratranches de chacune des images codées précédemment, où la similarité correspond à un indice déterminé de sorte que plus les images des deux intratranches distinctes sont similaires, plus la similarité est élevée ;
    - une étape de sélection d'image (S2) qui sélectionne une image présentant la similarité la plus élevée parmi les images codées précédemment, sur la base de la similarité calculée informatiquement au cours de l'étape de calcul informatique de similarité ; et
    - une étape de détermination d'échelon de quantification (S3 - S5) qui :
    o calcule :
    ▪ une complexité pour la zone d'intratranches, en utilisant une quantité de code générée pour la zone d'intratranches de l'image sélectionnée au cours de l'étape de sélection d'image et un échelon de quantification moyen pour la zone d'intratranches de cette image ;
    ▪ une complexité pour la zone sans intratranches, en utilisant une quantité de code générée pour la zone sans intratranches de l'image codée immédiatement avant l'image cible de codage et un échelon de quantification moyen pour la zone sans intratranches de cette image ;
    ▪ des quantités de code respectives affectées à la zone d'intratranches de l'image cible de codage et à la zone sans intratranches de l'image cible de codage, en utilisant la complexité calculée informatiquement pour la zone d'intratranches et la complexité calculée informatiquement pour la zone sans intratranches ;
    o détermine un échelon de quantification de la zone d'intratranches sur la base de la quantité de code affectée à la zone d'intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone d'intratranches de l'image cible de codage ; et
    o détermine un échelon de quantification de la zone sans intratranches sur la base de la quantité de code affectée à la zone sans intratranches de l'image cible de codage, où l'échelon de quantification est utilisé en vue de coder chaque bloc dans la zone sans intratranches de l'image cible de codage.
EP12754501.0A 2011-03-10 2012-03-05 Dispositif et procédé de commande de quantification et programme de commande de quantification Active EP2648409B1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2011052872 2011-03-10
PCT/JP2012/055564 WO2012121211A1 (fr) 2011-03-10 2012-03-05 Dispositif et procédé de commande de quantification et programme de commande de quantification

Publications (3)

Publication Number Publication Date
EP2648409A1 EP2648409A1 (fr) 2013-10-09
EP2648409A4 EP2648409A4 (fr) 2014-05-21
EP2648409B1 true EP2648409B1 (fr) 2016-08-17

Family

ID=46798172

Family Applications (1)

Application Number Title Priority Date Filing Date
EP12754501.0A Active EP2648409B1 (fr) 2011-03-10 2012-03-05 Dispositif et procédé de commande de quantification et programme de commande de quantification

Country Status (11)

Country Link
US (1) US9906803B2 (fr)
EP (1) EP2648409B1 (fr)
JP (1) JP5410638B2 (fr)
KR (1) KR101522391B1 (fr)
CN (1) CN103329532B (fr)
BR (1) BR112013019415A2 (fr)
CA (1) CA2824164A1 (fr)
ES (1) ES2604202T3 (fr)
RU (1) RU2573279C2 (fr)
TW (1) TWI510050B (fr)
WO (1) WO2012121211A1 (fr)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6398569B2 (ja) * 2014-10-07 2018-10-03 株式会社ソシオネクスト 画像符号化装置、画像符号化方法および画像符号化プログラム
KR20160105203A (ko) 2015-02-27 2016-09-06 삼성전자주식회사 멀티미디어 코덱, 상기 멀티미디어 코덱을 포함하는 애플리케이션 프로세서, 및 상기 애플리케이션 프로세서의 동작 방법
KR102601732B1 (ko) * 2016-05-31 2023-11-14 삼성디스플레이 주식회사 영상 부호화 방법 및 영상 복호화 방법
JP6943628B2 (ja) 2017-05-24 2021-10-06 Ｎｔｔエレクトロニクス株式会社 映像符号化装置および映像符号化方法

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0795564A (ja) * 1993-09-20 1995-04-07 Toshiba Corp 動画像符号化装置
JP2009055262A (ja) * 2007-08-27 2009-03-12 Nippon Telegr & Teleph Corp <Ntt> 映像符号化制御装置、映像符号化制御方法、映像符号化制御プログラムおよびそのプログラムを記録したコンピュータ読み取り可能な記録媒体

Family Cites Families (42)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5471207A (en) * 1994-02-23 1995-11-28 Ricoh Company Ltd. Compression of palettized images and binarization for bitwise coding of M-ary alphabets therefor
US5832234A (en) * 1995-09-29 1998-11-03 Intel Corporation Encoding images using block-based macroblock-level statistics
DE69829783T2 (de) 1997-02-08 2005-09-01 Matsushita Electric Industrial Co., Ltd., Kadoma Quantisierungsmatrix für die Codierung von Stand- und Bewegtbildern
US6023296A (en) * 1997-07-10 2000-02-08 Sarnoff Corporation Apparatus and method for object based rate control in a coding system
JP4362795B2 (ja) 1999-07-13 2009-11-11 日本ビクター株式会社 動画像符号化装置及びその方法
US6804301B2 (en) * 2001-08-15 2004-10-12 General Instrument Corporation First pass encoding of I and P-frame complexity for compressed digital video
US6882685B2 (en) 2001-09-18 2005-04-19 Microsoft Corporation Block transform and quantization for image and video coding
US6763068B2 (en) 2001-12-28 2004-07-13 Nokia Corporation Method and apparatus for selecting macroblock quantization parameters in a video encoder
US7251275B2 (en) * 2002-06-25 2007-07-31 General Instrument Corporation Methods and apparatus for statistical multiplexing during dual pass encoding
EP1520431B1 (fr) * 2002-07-01 2018-12-26 E G Technology Inc. Compression et transport efficaces de video sur un reseau
JP4221655B2 (ja) 2003-03-06 2009-02-12 ソニー株式会社 符号化装置および符号化方法、プログラム、並びに記録媒体
SG140441A1 (en) * 2003-03-17 2008-03-28 St Microelectronics Asia Decoder and method of decoding using pseudo two pass decoding and one pass encoding
US20050047504A1 (en) * 2003-09-03 2005-03-03 Sung Chih-Ta Star Data stream encoding method and apparatus for digital video compression
TWI225363B (en) 2003-11-10 2004-12-11 Mediatek Inc Method and apparatus for controlling quantization level of a video encoding bit stream
JP4257655B2 (ja) 2004-11-04 2009-04-22 日本ビクター株式会社 動画像符号化装置
US7706442B2 (en) * 2005-02-15 2010-04-27 Industrial Technology Research Institute Method for coding mode selection of intra prediction in video compression
US8548055B2 (en) * 2005-03-10 2013-10-01 Qualcomm Incorporated Encoding of multimedia data
JP4146444B2 (ja) * 2005-03-16 2008-09-10 株式会社東芝 動画像符号化の方法及び装置
WO2006118383A1 (fr) * 2005-04-29 2006-11-09 Samsung Electronics Co., Ltd. Procede et dispositif de codage video permettant l'echelonnement rapide a granularite fine
US8634469B2 (en) * 2006-02-06 2014-01-21 Thomson Licensing Method and apparatus for reusing available motion information as a motion estimation predictor for video encoding
TWI324883B (en) 2006-06-26 2010-05-11 Himax Tech Inc Method of estimating a quantization parameter
KR100829169B1 (ko) * 2006-07-07 2008-05-13 주식회사 리버트론 Ｈ.264 코딩의 압축모드 예측 장치 및 방법
US8036270B2 (en) * 2006-07-27 2011-10-11 Sharp Laboratories Of America, Inc. Intra-frame flicker reduction in video coding
EP2080377A2 (fr) 2006-10-31 2009-07-22 THOMSON Licensing Procédé et appareil pour manipuler des flux binaires
US7653130B2 (en) * 2006-12-27 2010-01-26 General Instrument Corporation Method and apparatus for bit rate reduction in video telephony
US20080170793A1 (en) * 2007-01-12 2008-07-17 Mitsubishi Electric Corporation Image encoding device and image encoding method
ES2532375T3 (es) * 2007-03-14 2015-03-26 Nippon Telegraph And Telephone Corporation Método de control de velocidad binaria de codificación, dispositivo, programa, y medio de grabación que contiene el programa
US8331438B2 (en) * 2007-06-05 2012-12-11 Microsoft Corporation Adaptive selection of picture-level quantization parameters for predicted video pictures
KR101345295B1 (ko) * 2007-06-11 2013-12-27 삼성전자주식회사 인트라 픽처만을 이용한 동영상 시퀀스의 부호화시의비트율 제어 방법 및 장치
WO2009045683A1 (fr) * 2007-09-28 2009-04-09 Athanasios Leontaris Techniques de compression et de transmission vidéo
US8520730B2 (en) * 2007-11-28 2013-08-27 Panasonic Corporation Picture coding method and picture coding apparatus
KR101700966B1 (ko) * 2008-01-10 2017-01-31 톰슨 라이센싱 인트라 예측 비디오의 조명 보상 방법 및 장치
US8542730B2 (en) * 2008-02-22 2013-09-24 Qualcomm, Incorporated Fast macroblock delta QP decision
KR101460608B1 (ko) * 2008-03-04 2014-11-14 삼성전자주식회사 필터링된 예측 블록을 이용한 영상 부호화, 복호화 방법 및장치
US20090274211A1 (en) * 2008-04-30 2009-11-05 Omnivision Technologies, Inc. Apparatus and method for high quality intra mode prediction in a video coder
JP2010035025A (ja) * 2008-07-30 2010-02-12 Canon Inc 符号化装置およびその方法
JP5232114B2 (ja) * 2008-09-29 2013-07-10 パナソニック株式会社 画像符号化装置、及び画像符号化方法
CN102106147B (zh) * 2008-09-29 2013-06-19 松下电器产业株式会社 动态图像编码方法以及动态图像编码装置
JP5564061B2 (ja) * 2009-02-06 2014-07-30 トムソン ライセンシング ビデオ・エンコーダおよびデコーダにおける暗黙的および半暗黙的イントラ・モード信号伝達方法および装置
JP5353532B2 (ja) * 2009-07-29 2013-11-27 ソニー株式会社 画像処理装置及び画像処理方法
KR101733612B1 (ko) * 2009-10-22 2017-05-08 톰슨 라이센싱 비디오 엔코딩 및 디코딩을 위한 ｄｃ 인트라 예측 모드의 방법 및 장치
CN105791823B (zh) * 2009-12-08 2019-11-29 交互数字麦迪逊专利控股公司 视频编码和解码的自适应模版匹配预测的方法和装置

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0795564A (ja) * 1993-09-20 1995-04-07 Toshiba Corp 動画像符号化装置
JP2009055262A (ja) * 2007-08-27 2009-03-12 Nippon Telegr & Teleph Corp <Ntt> 映像符号化制御装置、映像符号化制御方法、映像符号化制御プログラムおよびそのプログラムを記録したコンピュータ読み取り可能な記録媒体

Also Published As

Publication number Publication date
KR101522391B1 (ko) 2015-05-21
TWI510050B (zh) 2015-11-21
ES2604202T3 (es) 2017-03-03
JPWO2012121211A1 (ja) 2014-07-17
EP2648409A4 (fr) 2014-05-21
KR20130095322A (ko) 2013-08-27
US9906803B2 (en) 2018-02-27
RU2573279C2 (ru) 2016-01-20
WO2012121211A1 (fr) 2012-09-13
CA2824164A1 (fr) 2012-09-13
JP5410638B2 (ja) 2014-02-05
CN103329532A (zh) 2013-09-25
CN103329532B (zh) 2016-10-26
TW201244488A (en) 2012-11-01
US20130343453A1 (en) 2013-12-26
RU2013135712A (ru) 2015-04-20
EP2648409A1 (fr) 2013-10-09
BR112013019415A2 (pt) 2019-09-24

Similar Documents

Publication Publication Date Title
AU2015213341B2 (en) Video decoder, video encoder, video decoding method, and video encoding method
US9621917B2 (en) Continuous block tracking for temporal prediction in video encoding
EP2129134B1 (fr) Procédé et dispositif de codage d&#39;image variant dans le temps, et dispositif de décodage d&#39;image variant dans le temps
EP3154266A1 (fr) Appareil et procédé de codage/décodage sélectif d&#39;une image par transformée de cosinus/sinus discrète
US20070199011A1 (en) System and method for high quality AVC encoding
CA2960617A1 (fr) Optimisation de la perception pour codage video a base de modeles
EP1936994A2 (fr) Codage de vidéo à prédiction pondérée
JP2005191706A (ja) 動画像符号化方法及びその装置
US10298945B2 (en) Video encoding method, video decoding method, video encoding apparatus, video decoding apparatus, and programs thereof
JP2007267414A (ja) フレーム内画像符号化方法及びその装置
US20170099491A1 (en) Apparatus, method and non-transitory medium storing program for encoding moving picture
US11134250B2 (en) System and method for controlling video coding within image frame
WO2019104635A1 (fr) Système et procédé de commande de codage vidéo à un niveau de trame
EP2648409B1 (fr) Dispositif et procédé de commande de quantification et programme de commande de quantification
CA2720698C (fr) Procede de codage video, equipement de codage video, programme de codage video et son support d&#39;enregistrement
JP6239838B2 (ja) 動画像符号化装置、その制御方法、及び撮像装置
EP3136725B1 (fr) Appareil de codage vidéo
Wang et al. Motion estimation and mode decision for low-complexity h. 264 decoder
KR20130057020A (ko) 인접한 블록의 움직임 벡터를 이용한 고속 움직임 추정 방법
KR20040084328A (ko) 중간 데이터에 기초한 적응적 인트라 매크로블록 갱신 방법

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130705

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602012021915

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04N0007260000

Ipc: H04N0019196000

A4 Supplementary search report drawn up and despatched

Effective date: 20140424

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/124 20140101ALI20140416BHEP

Ipc: H04N 19/593 20140101ALI20140416BHEP

Ipc: H04N 19/196 20140101AFI20140416BHEP

Ipc: H04N 19/14 20140101ALI20140416BHEP

DAX Request for extension of the european patent (deleted)
17Q First examination report despatched

Effective date: 20150206

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

INTG Intention to grant announced

Effective date: 20160307

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 822071

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160915

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602012021915

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20160817

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 822071

Country of ref document: AT

Kind code of ref document: T

Effective date: 20160817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161117

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161219

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161118

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

REG Reference to a national code

Ref country code: ES

Ref legal event code: FG2A

Ref document number: 2604202

Country of ref document: ES

Kind code of ref document: T3

Effective date: 20170303

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 6

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602012021915

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161117

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

26N No opposition filed

Effective date: 20170518

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170331

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 7

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20120305

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20160817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20160817

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161217

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20230324

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: ES

Payment date: 20230529

Year of fee payment: 12

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: DE

Payment date: 20240320

Year of fee payment: 13

Ref country code: GB

Payment date: 20240320

Year of fee payment: 13