WO1997039417A3 - Method and apparatus for training a neural network to use a fidelity metric - Google Patents

Method and apparatus for training a neural network to use a fidelity metric Download PDF

Info

Publication number
WO1997039417A3
WO1997039417A3 PCT/US1997/005224 US9705224W WO9739417A3 WO 1997039417 A3 WO1997039417 A3 WO 1997039417A3 US 9705224 W US9705224 W US 9705224W WO 9739417 A3 WO9739417 A3 WO 9739417A3
Authority
WO
WIPO (PCT)
Prior art keywords
neural network
fidelity
vdm
training
processing apparatus
Prior art date
Application number
PCT/US1997/005224
Other languages
French (fr)
Other versions
WO1997039417A2 (en
Inventor
Aalbert Devries
Jeffrey Lubin
Heidi A Peterson
Clay Spence
Original Assignee
Sarnoff Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sarnoff Corp filed Critical Sarnoff Corp
Publication of WO1997039417A2 publication Critical patent/WO1997039417A2/en
Publication of WO1997039417A3 publication Critical patent/WO1997039417A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H04N19/198—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters including smoothing of a sequence of encoding parameters, e.g. by averaging, by choice of the maximum, minimum or median value
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/142—Detection of scene cut or scene change
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/147—Data rate or code amount at the encoder output according to rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/179—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a scene or a shot
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/192—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding the adaptation method, adaptation tool or adaptation type being iterative or recursive
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/189—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding
    • H04N19/196—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the adaptation method, adaptation tool or adaptation type used for the adaptive coding being specially adapted for the computation of encoding parameters, e.g. by averaging previously computed encoding parameters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/87—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving scene cut or scene change detection in combination with video compression
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding

Abstract

A signal processing apparatus (100, 400) and concomitant method for learning and using fidelity metric as a control mechanism and to process large quantities of fidelity metrics from a visual discrimination measure (VDM) (111) to a manageable subjective image quality ratings. The signal processing apparatus incorporates a VDM (111) and a neural network (112). The VDM (111) receives input image sequences and generates fidelity metrics, which are received by a neural network (112). The neural network (112) is trained to learn and use the fidelity metrics as a control mechanism, e.g., to control a video encoder.
PCT/US1997/005224 1996-03-29 1997-03-31 Method and apparatus for training a neural network to use a fidelity metric WO1997039417A2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US1426696P 1996-03-29 1996-03-29
US60/014,266 1996-03-29
US1468796P 1996-04-02 1996-04-02
US60/014,687 1996-04-02

Publications (2)

Publication Number Publication Date
WO1997039417A2 WO1997039417A2 (en) 1997-10-23
WO1997039417A3 true WO1997039417A3 (en) 1998-05-14

Family

ID=26685872

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US1997/005224 WO1997039417A2 (en) 1996-03-29 1997-03-31 Method and apparatus for training a neural network to use a fidelity metric

Country Status (1)

Country Link
WO (1) WO1997039417A2 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB9803580D0 (en) * 1998-02-21 1998-04-15 Nds Ltd Determining visually noticeable differences between two images
JP2004527172A (en) * 2001-03-29 2004-09-02 コーニンクレッカ　フィリップス　エレクトロニクス　エヌ　ヴィ Scalable system and method for optimizing a random system of algorithms for image quality
CN101695141B (en) * 2009-10-20 2012-05-23 浙江大学 Method and device for evaluating video quality
GB2543958B (en) 2015-02-19 2017-09-27 Magic Pony Tech Ltd Enhancing visual data using and augmenting model libraries
WO2016156864A1 (en) 2015-03-31 2016-10-06 Magic Pony Technology Limited Training end-to-end video processes
GB201604672D0 (en) 2016-03-18 2016-05-04 Magic Pony Technology Ltd Generative methods of super resolution
GB201603144D0 (en) 2016-02-23 2016-04-06 Magic Pony Technology Ltd Training end-to-end video processes
US10827185B2 (en) * 2016-04-07 2020-11-03 Netflix, Inc. Techniques for robustly predicting perceptual video quality
EP3298579B1 (en) 2016-04-12 2021-07-21 Magic Pony Technology Limited Visual data processing using energy networks
GB201607994D0 (en) 2016-05-06 2016-06-22 Magic Pony Technology Ltd Encoder pre-analyser
US10929607B2 (en) * 2018-02-22 2021-02-23 Salesforce.Com, Inc. Dialogue state tracking using a global-local encoder

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5107347A (en) * 1989-07-24 1992-04-21 Sony Corporation Image data inputting system for image processing apparatus
US5309525A (en) * 1990-06-14 1994-05-03 Canon Kabushiki Kaisha Image processing apparatus using neural network
US5394483A (en) * 1992-06-30 1995-02-28 Eastman Kodak Co Method and apparatus for determining visually perceptible differences between images
US5491776A (en) * 1991-08-05 1996-02-13 Kawasaki Steel Corporation Signal processing apparatus and learning method therefor
US5495542A (en) * 1990-03-23 1996-02-27 Canon Kabushiki Kaisha Binary to multi-level image restoration using neural network

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5107347A (en) * 1989-07-24 1992-04-21 Sony Corporation Image data inputting system for image processing apparatus
US5495542A (en) * 1990-03-23 1996-02-27 Canon Kabushiki Kaisha Binary to multi-level image restoration using neural network
US5309525A (en) * 1990-06-14 1994-05-03 Canon Kabushiki Kaisha Image processing apparatus using neural network
US5491776A (en) * 1991-08-05 1996-02-13 Kawasaki Steel Corporation Signal processing apparatus and learning method therefor
US5394483A (en) * 1992-06-30 1995-02-28 Eastman Kodak Co Method and apparatus for determining visually perceptible differences between images

Also Published As

Publication number Publication date
WO1997039417A2 (en) 1997-10-23

Similar Documents

Publication Publication Date Title
WO1997039417A3 (en) Method and apparatus for training a neural network to use a fidelity metric
EP0396360A3 (en) Apparatus for inter-frame predictive encoding of video signal
CA2061527A1 (en) Method and apparatus for controlling coefficients of adaptive filter
EP0398268A3 (en) Video printer device
CA2066237A1 (en) Computer neural network regulatory process control system and method
EP1134696A3 (en) Image processing method and apparatus
CA2150281A1 (en) Control process for use in the production of printed products and means for performing the process
MY145597A (en) Method and apparatus for representing image granularity by one or more parameters
DE69127504D1 (en) Process for transmitting picture and / or sound signals
NO179890C (en) Adaptive motion compensation for digital television
JP2001119644A5 (en) Image processing device, television receiver, and image processing method
EP1202573A3 (en) Moving image packet decoding and reproducing apparatus, reproduction time control method thereof, computer program product for controlling reproduction time and multimedia information receiving apparatus
EP0441586A3 (en) Image processing apparatus
GB9414446D0 (en) Method and apparatus for controlling updates of extended data services (eds) data
EP1326449A3 (en) Data decoding method and decoding system
WO1999016234A3 (en) System, method and medium for increasing compression of an image while minimizing image degradation
NO178098C (en) Method and apparatus for removing liquid from a mixture of liquid and solid
FI940331A0 (en) Processing of speech coder parameters in a telecommunication system receiver
DE59205551D1 (en) Process for the transmission of digital HDTV signals
EP1111931A3 (en) Methods, apparatuses and program recording media for image coding and image decoding
EP1544789A4 (en) Image color correcting device, image color correcting method and image color correcting program
EP1978745A3 (en) Statistical adaptive video rate control
FR2669035B1 (en) PROCESS AND APPARATUS FOR THE PRODUCTION OF CARBON BLACK.
GB9604315D0 (en) Training process
DE3842411A1 (en) TELEVISION RECEIVER WITH DIGITAL AND ANALOGUE SOUND SIGNAL PROCESSING

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): CA JP KR MX

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE

121 Ep: the epo has been informed by wipo that ep was designated in this application
DFPE Request for preliminary examination filed prior to expiration of 19th month from priority date (pct application filed before 20040101)
AK Designated states

Kind code of ref document: A3

Designated state(s): CA JP KR MX

AL Designated countries for regional patents

Kind code of ref document: A3

Designated state(s): AT BE CH DE DK ES FI FR GB GR IE IT LU MC NL PT SE

NENP Non-entry into the national phase

Ref country code: JP

Ref document number: 97537111

Format of ref document f/p: F

122 Ep: pct application non-entry in european phase