EP2654298B1 - Detection of video feature based on variance metric - Google Patents

Detection of video feature based on variance metric Download PDF

Info

Publication number
EP2654298B1
EP2654298B1 EP13163848.8A EP13163848A EP2654298B1 EP 2654298 B1 EP2654298 B1 EP 2654298B1 EP 13163848 A EP13163848 A EP 13163848A EP 2654298 B1 EP2654298 B1 EP 2654298B1
Authority
EP
European Patent Office
Prior art keywords
image
region
variances
sum
metric
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Not-in-force
Application number
EP13163848.8A
Other languages
German (de)
English (en)
French (fr)
Other versions
EP2654298A3 (en
EP2654298A2 (en
Inventor
Ying Li
Xu Gang Zhao
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
ViXS Systems Inc
Original Assignee
ViXS Systems Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ViXS Systems Inc filed Critical ViXS Systems Inc
Publication of EP2654298A2 publication Critical patent/EP2654298A2/en
Publication of EP2654298A3 publication Critical patent/EP2654298A3/en
Application granted granted Critical
Publication of EP2654298B1 publication Critical patent/EP2654298B1/en
Not-in-force legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/40—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using video transcoding, i.e. partial or full decoding of a coded input stream followed by re-encoding of the decoded output stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/114—Adapting the group of pictures [GOP] structure, e.g. number of B-frames between two anchor frames
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/115—Selection of the code volume for a coding unit prior to coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142—Detection of scene cut or scene change
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object

Definitions

  • Video encoding or transcoding frequently is used to reduce the amount of video data to be stored or transmitted or to convert a video signal from one format to another.
  • Effective transcoding often relies on the accurate detection of features present in the video content, such as blank screens, scene changes, black borders, and the like.
  • Conventional techniques for identifying these features such as by detecting a change in sound level for identifying a scene change or by determining the number of pixels having a certain color for identifying a black border, often are inefficient or ineffective at identifying the corresponding feature.
  • the region may comprise a border region of the image; the at least one video feature may comprise presence of one of a black border region or a caption region in the image; and transcoding the image may comprise at least one of omitting image content of the region from a corresponding transcoded image for the transcoded video signal; and allocating a first number of bits to the region for rate control during transcoding and allocating a second number of bits to an active image region of the image for rate control during transcoding, the second number of bits different than the first number of bits.
  • the region may comprise the entire image; the at least one video feature may comprise a complexity of the image; and transcoding the image may comprise allocating a number of bits to the image for rate control during transcoding based on the complexity of the image.
  • the transcoding process can implement the transcoded version of the image as an Intra-coded frame (I-frame) in a new group of pictures (GOP).
  • I-frame Intra-coded frame
  • GOP group of pictures
  • the sum of variance metrics for one or more columns of pixel blocks or rows of pixel blocks may be used to detect the presence of a black border region or a caption region and the transcoding process adapted accordingly, such as by omitting the black border region/caption region from the corresponding image in the transcoded signal, by determining a true resolution of the active image region surrounded by the black border region or adjacent to the caption region (which can prove useful in setting the threshold for scene change detection), by detecting a scene change based on a presence or change in the black border region, or by assigning a different bit budget to the black border region/caption region while transcoding the image.
  • the video processing system 100 can comprise a video content server system, whereby the video source 102 comprises a hard drive storing original video content, the video destination 106 is a remote computer system connected to the video content server via a network, and the transcoding system 104 is used to transcode the video content responsive to current network conditions before the transcoded video content is transmitted to the remote computer system via the network.
  • the video source 102 comprises a hard drive storing original video content
  • the video destination 106 is a remote computer system connected to the video content server via a network
  • the transcoding system 104 is used to transcode the video content responsive to current network conditions before the transcoded video content is transmitted to the remote computer system via the network.
  • the decoder 116, encoder 118, and feature detection module 120 each may be implemented entirely in hardware, entirely as software stored in a memory 122 and executed by a processor 124, or a combination of hardware logic and software-executed functionality.
  • the transcoding system 104 is implemented as a SOC whereby portions of the decoder 118, the encoder 118, and the feature detection module 120 are implemented as hardware logic, and other portions are implemented via firmware stored at the SOC and executed by a processor of the SOC.
  • the hardware of the transcoding system 104 can be implemented using a single processing device or a plurality of processing devices.
  • Such processing devices can include a central processing unit (CPU), a graphics processing unit (GPU), a microcontroller, a digital signal processor, a field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, or any device that manipulates signals (analog and/or digital) based on operational instructions that are stored in a memory, such as memory 122.
  • Memory 122 may be a single memory device or a plurality of memory devices.
  • Such memory devices can include a hard disk drive or other disk drive, read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
  • FIG. 2 provides an example context to illustrate various terms used herein with respect to variance.
  • the illustrated image 200 (I-frame) is comprised of a matrix of pixel blocks 202 arranged in a plurality of rows and columns.
  • the image 200 can be a field of a video frame (e.g., an odd field or an even field in an interlaced implementation) or the complete video frame (e.g., both the odd field and even field combined in a progressive implementation).
  • a typical video image can comprise hundreds, or even thousands, of rows and columns of these pixel blocks 202.
  • the image 200 is a simplified image comprising sixty-four pixel blocks 202 arranged in eight horizontal rows (labeled 1-8) and eight columns (labeled A-H).
  • a sum of variances metric may be calculated for an expected active image region composed of those pixel blocks 202 that are, for example, both in rows 3-6 and in columns C-F, or a sum of variances metric may be calculated for an expected black border region composed of those pixel blocks 202 that are, for example, both in one of rows 1, 2, 7, or 8 and in one of columns A, B, G, or H. Further, a sum of variances metric may be calculated for all of the pixel blocks 202 of the image and denoted as SVAR I .
  • FIG. 3 illustrates an example method 300 of operation of the transcoding system 104 of FIG. 1 .
  • the transcoding system 104 receives the video signal 108 from the video sources 102, wherein the video signal 108 can be received from a remove source over a wired or wireless network, received from a local storage device via an I/O interface, and the like.
  • the transcoding system 104 transcodes the video signal 108 to generate the transcoded video signal 110.
  • This transcoding process can include one or more of an encoding format conversion, a resolution conversion, a bit rate conversion, a frame rate conversion, and the like.
  • the feature detection module 120 processes certain images represented in the video signal 108 (or the decoded version thereof) so as to identify certain characteristics in the images and modify the transcoding process in response to identifying the characteristics.
  • the images processed by the feature detection module 120 typically include, for example, the I-frames of the video signal 108 (or a subset of the I-frames), although predicted frames (P-frames) and bi-predicted frames (B-frames) also may be analyzed.
  • the processing of an image by the feature detection module 120 includes determining one or more sum of variances metric for an image, or one or more regions of the image, at block 308.
  • FIGs. 4 and 5 illustrate a method 400 ( FIG. 4 ) for detecting scene changes in a video signal 500 ( FIG. 5 ) based on SVAR metrics in accordance with at least one embodiment of the present disclosure.
  • the method 400 initiates at block 402 whereby the feature detection module 120 receives or selects for scene change analysis an image 502 from the video signal 500 being processed for encoding or transcoding by the transcoding system 104 ( FIG. 1 ).
  • the feature detection module 120 determines a current SVAR metric for the image 502 currently being analyzed for scene change detection.
  • the current SVAR metric comprises the current SVAR metric for the entire image 502 (SVAR I ).
  • the feature detection module 120 can detect a black border region 726 from these SVAR projections in a number of ways.
  • the feature detection module 120 can detect each black bar, or matte, of the black border individually.
  • the feature detection module 120 can sum or average the variance row projections in horizontal border region 716 to determine a region SVAR metric for detecting the presence of the top horizontal black bar 702.
  • the feature detection module 120 can sum or average the variance row projections in horizontal border region 718 to determine a region SVAR metric for detecting the presence of the bottom horizontal black bar 704.
  • the threshold for detecting the presence of the top black bar 702 may be set to K such that when the average of the variance row projections for the pixel blocks in region 716 of the image 700 falls below K, the feature detection module 120 identifies image 700 has having the top black bar 702 at horizontal region 716. While the threshold may be static in some implementations, in other implementations the threshold may dynamically changed based on feedback during the transcoding process.
  • an image having a higher SVAR I may be allocated a higher bit budget or lower quantization parameter than an image having a lower SVAR I , and thus a lower complexity.
  • an image having a higher SVAR I and thus a higher complexity may call for a higher quantization parameter than an image having a lower SVAR I , and thus a lower complexity under the same bit budget limitation.
  • FIGs. 3-8 illustrate example transcoding operations that may be implemented or modified based on determined SVAR metrics, any of a variety of transcoding operations may be implemented or modified in a similar manner without departing from the scope of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
EP13163848.8A 2012-04-19 2013-04-16 Detection of video feature based on variance metric Not-in-force EP2654298B1 (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US13/450,870 US9071842B2 (en) 2012-04-19 2012-04-19 Detection of video feature based on variance metric

Publications (3)

Publication Number Publication Date
EP2654298A2 EP2654298A2 (en) 2013-10-23
EP2654298A3 EP2654298A3 (en) 2014-10-01
EP2654298B1 true EP2654298B1 (en) 2016-10-05

Family

ID=48190088

Family Applications (1)

Application Number Title Priority Date Filing Date
EP13163848.8A Not-in-force EP2654298B1 (en) 2012-04-19 2013-04-16 Detection of video feature based on variance metric

Country Status (3)

Country Link
US (1) US9071842B2 (zh)
EP (1) EP2654298B1 (zh)
CN (1) CN103379335B (zh)

Families Citing this family (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9131097B2 (en) * 2011-09-16 2015-09-08 Dolby Laboratories Licensing Corporation Method and system for black bar identification
EP2670151A1 (en) * 2012-05-28 2013-12-04 Tektronix Inc. Heuristic method for drop frame detection in digital baseband video
KR20150039425A (ko) * 2013-10-02 2015-04-10 삼성전자주식회사 비디오/이미지 데이터 처리 방법 및 비디오/이미지 데이터 처리 시스템
US9813706B1 (en) 2013-12-02 2017-11-07 Google Inc. Video content analysis and/or processing using encoding logs
US20150189222A1 (en) * 2013-12-30 2015-07-02 Google Inc. Content-adaptive chunking for distributed transcoding
TWI538480B (zh) * 2014-05-27 2016-06-11 敦泰電子股份有限公司 藉由偵測鄰近像素複雜度來動態調整壓縮參數的影像壓縮系統
WO2016017695A1 (ja) * 2014-07-30 2016-02-04 オリンパス株式会社 画像処理装置
CN104244107B (zh) * 2014-08-26 2017-08-08 中译语通科技（北京）有限公司 一种基于字幕检测与识别的视频字幕还原方法
CN104394345B (zh) * 2014-12-10 2017-11-07 黄山中鼎信息技术有限公司 一种安防监控视频存储与回放方法
JP6537396B2 (ja) * 2015-08-03 2019-07-03 キヤノン株式会社 画像処理装置、撮像装置および画像処理方法
US11019349B2 (en) 2017-01-20 2021-05-25 Snap Inc. Content-based client side video transcoding
CN107027030B (zh) * 2017-03-07 2018-11-09 腾讯科技（深圳）有限公司 一种码率分配方法及其设备
US20180316953A1 (en) * 2017-04-28 2018-11-01 Sony Interactive Entertainment LLC Integrating media content for local channels and subscription channels
US10674111B2 (en) * 2017-12-11 2020-06-02 Disney Enterprises, Inc. Systems and methods for profile based media segment rendering
US10992938B2 (en) * 2018-09-28 2021-04-27 Ati Technologies Ulc Spatial block-level pixel activity extraction optimization leveraging motion vectors
US20190349585A1 (en) * 2019-07-23 2019-11-14 Intel Corporation Content and quantization adaptive coding structure decisions for video coding
CN110675425B (zh) * 2019-08-22 2020-12-15 腾讯科技（深圳）有限公司 一种视频边框识别方法、装置、设备及介质
CN111901606A (zh) * 2020-07-31 2020-11-06 杭州当虹科技股份有限公司 一种提升字幕编码质量的视频编码方法
US20220067213A1 (en) * 2020-08-31 2022-03-03 Lieb Lindsey S Subliminal Software Detection
CN112866572B (zh) * 2021-01-11 2022-08-23 浙江大华技术股份有限公司 画面黑边矫正的方法、装置、电子装置和存储介质
WO2023141917A1 (en) * 2022-01-28 2023-08-03 Qualcomm Incorporated Sequential flexible display shape resolution

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050025249A1 (en) * 2002-08-14 2005-02-03 Lifeng Zhao Systems and methods for selecting a macroblock mode in a video encoder

Family Cites Families (20)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5486863A (en) 1994-04-29 1996-01-23 Motorola, Inc. Method for determining whether to intra code a video block
JP3954656B2 (ja) * 1994-09-29 2007-08-08 ソニー株式会社 画像符号化装置及び方法
US5610659A (en) 1995-05-08 1997-03-11 Futuretel, Inc. MPEG encoder that concurrently determines video data encoding format and rate control
GB2306833B (en) 1995-10-30 2000-03-29 Sony Uk Ltd Video data compression
US6038256A (en) * 1996-12-31 2000-03-14 C-Cube Microsystems Inc. Statistical multiplexed video encoding using pre-encoding a priori statistics and a priori and a posteriori statistics
US5978029A (en) 1997-10-10 1999-11-02 International Business Machines Corporation Real-time encoding of video sequence employing two encoders and statistical analysis
EP0917362A1 (en) 1997-11-12 1999-05-19 STMicroelectronics S.r.l. Macroblock variance estimator for MPEG-2 video encoder
EP1374429A4 (en) 2001-03-05 2009-11-11 Intervideo Inc SYSTEMS AND METHOD FOR CODING AND DECODING REDUNDANT MOTION VECTORS IN COMPRESSED VIDEO BITSTRAMS
US7299370B2 (en) 2003-06-10 2007-11-20 Intel Corporation Method and apparatus for improved reliability and reduced power in a processor by automatic voltage control during processor idle states
US8879857B2 (en) 2005-09-27 2014-11-04 Qualcomm Incorporated Redundant data encoding methods and device
US20100027667A1 (en) * 2007-01-26 2010-02-04 Jonatan Samuelsson Motion estimation for uncovered frame regions
US9313504B2 (en) * 2007-07-02 2016-04-12 Vixs Systems, Inc. Pattern detection module with region detection, video encoding system and method for use therewith
US9628811B2 (en) * 2007-12-17 2017-04-18 Qualcomm Incorporated Adaptive group of pictures (AGOP) structure determination
JP5039976B2 (ja) * 2008-03-24 2012-10-03 株式会社メガチップス トランスコーダ
US8787447B2 (en) 2008-10-30 2014-07-22 Vixs Systems, Inc Video transcoding system with drastic scene change detection and method for use therewith
US9407925B2 (en) 2008-10-30 2016-08-02 Vixs Systems, Inc. Video transcoding system with quality readjustment based on high scene cost detection and method for use therewith
CN101742324A (zh) * 2008-11-14 2010-06-16 北京中星微电子有限公司 视频编解码方法、视频编解码系统及编解码器
JP2012517774A (ja) * 2009-02-10 2012-08-02 シリコン イメージ，インコーポレイテッド ブロックノイズの検出及びフィルタ処理
EP2422520A1 (en) * 2009-04-20 2012-02-29 Dolby Laboratories Licensing Corporation Adaptive interpolation filters for multi-layered video delivery
KR20110053160A (ko) * 2009-11-13 2011-05-19 삼성전자주식회사 비디오 부가 재생 정보의 3차원 깊이감 조절을 위한 멀티미디어 스트림 생성 방법과 그 장치, 및 수신 방법과 그 장치

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050025249A1 (en) * 2002-08-14 2005-02-03 Lifeng Zhao Systems and methods for selecting a macroblock mode in a video encoder

Also Published As

Publication number Publication date
EP2654298A3 (en) 2014-10-01
US20130279563A1 (en) 2013-10-24
US9071842B2 (en) 2015-06-30
CN103379335B (zh) 2018-02-13
CN103379335A (zh) 2013-10-30
EP2654298A2 (en) 2013-10-23

Similar Documents

Publication Publication Date Title
EP2654298B1 (en) Detection of video feature based on variance metric
JP6498679B2 (ja) 動きベクトル精度の選択
US9426475B2 (en) Scene change detection using sum of variance and estimated picture encoding cost
JP5508534B2 (ja) シーン切替検出
JP5002286B2 (ja) 画像符号化装置、画像符号化方法、プログラム及び記憶媒体
JP5555741B2 (ja) デジタル映像データを符号化するための方法およびデジタル映像符号化システム
CN105847793B (zh) 视频编解码方法及其装置
TWI451766B (zh) 用於視頻編碼之電力效率高的運動估計技術
US9560361B2 (en) Adaptive single-field/dual-field video encoding
US11012698B2 (en) Image encoding apparatus and method for controlling the same
US20150016518A1 (en) Video Decoding with Reduced Complexity Deblocking
EP3057319A1 (en) Adaptive resolution video decoder
US20150163486A1 (en) Variable bitrate encoding
US9654775B2 (en) Video encoder with weighted prediction and methods for use therewith
JP5748225B2 (ja) 動画像符号化方法，動画像符号化装置および動画像符号化プログラム
JP2009535881A (ja) エンコード／トランスコード及びデコードのための方法及び装置
US20150163484A1 (en) Variable bitrate encoding for multiple video streams
US20150208082A1 (en) Video encoder with reference picture prediction and methods for use therewith
US20080080618A1 (en) Video decoding apparatus and method of the same
US9542611B1 (en) Logo detection for macroblock-based video processing
JP5389205B2 (ja) 画像符号化装置、画像符号化方法、プログラム及び記憶媒体
KR101582674B1 (ko) 영상 감시 시스템에서의 활성영상 저장방법 및 장치
JP2008028707A (ja) 画質評価装置、符号化装置及び画質評価方法
JP6740533B2 (ja) 符号化装置、復号装置及びプログラム
JP2008109642A (ja) 映像復号化装置及び方法

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20130416

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

PUAL Search report despatched

Free format text: ORIGINAL CODE: 0009013

AK Designated contracting states

Kind code of ref document: A3

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/00 20140101AFI20140929BHEP

17Q First examination report despatched

Effective date: 20150414

REG Reference to a national code

Ref country code: DE

Ref legal event code: R079

Ref document number: 602013012331

Country of ref document: DE

Free format text: PREVIOUS MAIN CLASS: H04N0007260000

Ipc: H04N0019115000

GRAP Despatch of communication of intention to grant a patent

Free format text: ORIGINAL CODE: EPIDOSNIGR1

RIC1 Information provided on ipc code assigned before grant

Ipc: H04N 19/142 20140101ALI20160517BHEP

Ipc: H04N 19/14 20140101ALI20160517BHEP

Ipc: H04N 19/40 20140101ALI20160517BHEP

Ipc: H04N 19/115 20140101AFI20160517BHEP

Ipc: H04N 19/114 20140101ALI20160517BHEP

Ipc: H04N 19/132 20140101ALI20160517BHEP

Ipc: H04N 19/17 20140101ALI20160517BHEP

INTG Intention to grant announced

Effective date: 20160622

GRAS Grant fee paid

Free format text: ORIGINAL CODE: EPIDOSNIGR3

GRAA (expected) grant

Free format text: ORIGINAL CODE: 0009210

AK Designated contracting states

Kind code of ref document: B1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

REG Reference to a national code

Ref country code: GB

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: CH

Ref legal event code: EP

REG Reference to a national code

Ref country code: AT

Ref legal event code: REF

Ref document number: 835490

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161015

REG Reference to a national code

Ref country code: IE

Ref legal event code: FG4D

REG Reference to a national code

Ref country code: DE

Ref legal event code: R096

Ref document number: 602013012331

Country of ref document: DE

REG Reference to a national code

Ref country code: NL

Ref legal event code: MP

Effective date: 20161005

REG Reference to a national code

Ref country code: LT

Ref legal event code: MG4D

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: LV

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

REG Reference to a national code

Ref country code: FR

Ref legal event code: PLFP

Year of fee payment: 5

REG Reference to a national code

Ref country code: AT

Ref legal event code: MK05

Ref document number: 835490

Country of ref document: AT

Kind code of ref document: T

Effective date: 20161005

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: NO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170105

Ref country code: GR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170106

Ref country code: SE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: LT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: FR

Payment date: 20170313

Year of fee payment: 5

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: BE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: AT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: IS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170205

Ref country code: NL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: PL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: RS

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: ES

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: HR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: PT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170206

REG Reference to a national code

Ref country code: DE

Ref legal event code: R097

Ref document number: 602013012331

Country of ref document: DE

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: RO

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: DK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: CZ

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: EE

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: SK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

PGFP Annual fee paid to national office [announced via postgrant information from national office to epo]

Ref country code: GB

Payment date: 20170412

Year of fee payment: 5

Ref country code: DE

Payment date: 20170411

Year of fee payment: 5

PLBE No opposition filed within time limit

Free format text: ORIGINAL CODE: 0009261

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: NO OPPOSITION FILED WITHIN TIME LIMIT

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SM

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

Ref country code: BG

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20170105

Ref country code: IT

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

26N No opposition filed

Effective date: 20170706

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: SI

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

REG Reference to a national code

Ref country code: CH

Ref legal event code: PL

REG Reference to a national code

Ref country code: IE

Ref legal event code: MM4A

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MC

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CH

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170430

Ref country code: LI

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170430

Ref country code: LU

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: IE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MT

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20170416

REG Reference to a national code

Ref country code: DE

Ref legal event code: R119

Ref document number: 602013012331

Country of ref document: DE

GBPC Gb: european patent ceased through non-payment of renewal fee

Effective date: 20180416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: DE

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20181101

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: GB

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: FR

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20180430

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: HU

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT; INVALID AB INITIO

Effective date: 20130416

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: CY

Free format text: LAPSE BECAUSE OF NON-PAYMENT OF DUE FEES

Effective date: 20161005

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: MK

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: TR

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005

PG25 Lapsed in a contracting state [announced via postgrant information from national office to epo]

Ref country code: AL

Free format text: LAPSE BECAUSE OF FAILURE TO SUBMIT A TRANSLATION OF THE DESCRIPTION OR TO PAY THE FEE WITHIN THE PRESCRIBED TIME-LIMIT

Effective date: 20161005