WO2006110046A1 - Optimisation de fichiers video - Google Patents

Optimisation de fichiers video Download PDF

Info

Publication number
WO2006110046A1
WO2006110046A1 PCT/NO2006/000136 NO2006000136W WO2006110046A1 WO 2006110046 A1 WO2006110046 A1 WO 2006110046A1 NO 2006000136 W NO2006000136 W NO 2006000136W WO 2006110046 A1 WO2006110046 A1 WO 2006110046A1
Authority
WO
WIPO (PCT)
Prior art keywords
sections
video file
digital video
video
granularity
Prior art date
Application number
PCT/NO2006/000136
Other languages
English (en)
Inventor
Håvard LILLEBO
Original Assignee
Viewstart As
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Viewstart As filed Critical Viewstart As
Publication of WO2006110046A1 publication Critical patent/WO2006110046A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/154—Measured or subjectively estimated visual quality after decoding, e.g. measurement of distortion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162—User input
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction

Definitions

  • the present invention is related to optimization of video files, and especially to a method and system of providing a video file comprising low resolution video parts mixed with high resolution video parts in such a manner that the whole video file is perceived by a viewer as comprising only high resolution video clips.
  • Digital storage and distribution of video movies and video clips are at present in high demand because of the widespread use of broad band technology that is facilitating "video on demand" concepts, distribution of video clips over the Internet, such as video based commercials, video information and entertainment etc.
  • High Definition TV is also enhancing the video quality and demands on system resources used in making, storing and distribution of such HDTV video clips.
  • the broad band technology has still bandwidth limitations providing a need for technology that can reduce the necessary transfer capacity in networks when transferring huge files (up to lGbps for HDTV video).
  • the prior art has several methods for reducing the necessary bandwidth of files.
  • Video films that are digitized can be digitized with different resolution, for example 8 bits, 16 bits etc. as known to a person skilled in the art. The more details that is wanted to be rendered the higher the number of bits is required.
  • High resolution requires high storage capacity since the higher resolution generates longer files. This is a problem especially when dealing with distribution of video which may then be impractical to do over a network.
  • DCT discrete cosine transform
  • VQ vector quantization
  • DWT discrete wavelet transform
  • Discrete cosine transform is a compression algorithm that samples an image at regular intervals, analyzes frequency components present in the sample, and discards those frequencies which do not affect the image as the human eye perceives it.
  • DCT is the basis of standards such as JPEG, MPEG, H.261, and H.263.
  • Vector quantization is a compression that uses an array of data, instead of individual values. Redundant data is then compressed, while at the same time retaining the desired objective of the data stream's original intent.
  • Fractal compression is a form of vector quantization compression. Compression is performed by locating self-similar sections (fractals) of an image, then using a fractal algorithm to generate the sections.
  • Discrete wavelet transform compression mathematically transforms an image into frequency components. The process is performed on the entire image. The result is a hierarchical representation of an image comprising layers each representing a frequency band.
  • the background of a scene may easily be judged as being indiscernible to the viewer.
  • the video film stage director may think different about some parts of the background comprising details regarded as necessary for the story to be communicated correctly or conveying the correct mood of the film.
  • Several items may be judged to be on a same level of importance in a scene, for example wine bottles on a table.
  • a special bottle may have a special significance in the story. Therefore it is important that this special bottle has enough resolution in the video clip to be distinguished as being important.
  • the relationship between foregrounds, background, importance of objects etc. is an important factor when judging a video film as being of good quality when viewed by spectators.
  • the human vision system perceives images in color using receptors on the retina of the eye which respond to three relatively broad color bands in the regions of red, green and blue (RGB) in the color spectrum (red, orange, yellow, green, blue, indigo, violet). Colors in between these are perceived as different linear combinations of RGB. This is the reason why electronic representations of images and/or video clips are represented by pixels comprising three variable intensities of red, green and blue colors.
  • RGB red, orange, yellow, green, blue, indigo, violet
  • the quantization effect present in digital images and/or video clips is interpreted as granularity of scenes, and may be measured and compared with threshold levels indicating if the present quantization or resolution or granularity is adequate to fulfill the requirements of quality of the images or video clips. Based on this evaluation, parts of images or video clips comprising low resolution parts may be substituted with corresponding high resolution parts thereby providing an optimized video file according to the present invention.
  • the steps of optimizing a video file may be performed in an iterative fashion. The resulting file of a processing through the steps of the optimization, according to the present invention, may iteratively be processed several times through the optimizing steps.
  • the optimization process is started by loading a low resolution video file and a high resolution version of the same video file into a computer system. If necessary, the low and high resolution versions of the same file are decompressed by appropriate decompression software before being used.
  • the designer of the video may, according to an example of embodiment of the present invention, draw a contour (outline) or a rectangle etc. enclosing objects regarded as having high importance for the story or visual appearance in sections with higher granularity than desired.
  • a computer program will then identify the sections marked in the low resolution version of the file and identify the corresponding sections in the high resolution version of the file.
  • the sections with high resolution are then replacing the corresponding sections identified in the low resolution file.
  • the resulting file is then an optimized video file according to the present invention.
  • Figure 1 depicts an example of embodiment of the present invention.
  • Figure 2 depicts a video time line comprising high and low resolution version of a video file.
  • Figure 3 illustrates an example of embodiment of the present invention comprising replacing high resolution sections with corresponding sections in a low resolution file.
  • Figure 4 depicts an example of a high resolution image.
  • Figure 5 depicts the same image as in figure 4 with lower resolution.
  • Figure 6 illustrates an example of embodiment of the present invention.
  • Figure 1 illustrates the principle scheme for providing an optimized video file according to the present invention.
  • At least one high quality version of the video file together with at least one low quality version of the same file is used as basis for providing the resulting video file.
  • frames 100 of the high quality video file comprise section 102 that is marked by a user, for example by drawing a frame surrounding the section that is selected. The particular selection is done, for example, because the section is important for the story depicted in the video file.
  • the selected section of the high quality file is then combined with the low quality video file to provide the optimized resulting file.
  • the video file may be represented with a plurality of resolutions in different files thereby enabling the user to select a version of a section that display sufficient resolution to be used in the resulting optimized file. In this manner, further tradeoff between file length and resolution may be achieved.
  • this scheme may be provided by a computer program comprising a video editing function as known to a person skilled in the art.
  • a video editing program may also comprise a time line as indicated in figure 2 and 3.
  • hi figure 2 selections are indicated by the outlines 102 in a high resolution version of the file in consecutive frames along the time line indicated in the bottom of the figure.
  • the time line will also indicate a starting time and an ending time for the substitutions done in the low resolution version of the file thereby enabling an identification or relation to insertion points in the low resolution file as known to a person skilled in the art.
  • Figure 3 illustrates the resulting optimized video file.
  • Figure 4 illustrates an image of a landscape with sufficient resolution to reveal details in such a manner that spectators will appreciate this image as a true rendering of the real landscape the image portraits.
  • Figure 5 illustrates the same landscape with a much lower resolution. Although the resolution is much lower the impression of the image is still such that a spectator can identify and recognize the landscape in the image. However, some parts of the image experience quantization errors that may be visible blocking the real structure of the landscape. For example, the lower resolution triggers the human eye to interpolate the grey scale in a different manner than when the image has higher resolution. Wider sections of the image have the same grey scale level when in low resolution. This affects the perspective perception of the image.
  • the hill surrounded by the rectangle in figure 5 seems to be closer to the foreground than in figure 4 due to this effect.
  • the section inside the rectangle should have at least the same resolution as in figure 5.
  • the rest of the image in figure 4 may be viewed as having sufficient resolution or granularity.
  • the granularity may be measured.
  • the principle of the measurement is to define a threshold granule, for example 5x5 pixels. This defines the lowest permissible resolution of an image or video clip. Further, it is necessary to evaluate how many details an image or video frame comprises. If there is many details in an image or video frame, colors, gray scale, intensity etc. of one granule (5x5 pixels for example) will change when moving from one granule to adjacent granules.
  • a measure of detail of an image or video frame may be established by counting the change or frequency of a parameter such as color etc. or a collection of parameters such as color and/or intensity etc when moving from one granule to the next.
  • the frequency or count of a parameter or a collection of parameters may then be compared with a threshold representative for the actual images or video clips under investigation. If the count or frequency is below the threshold definition, the image or video frame is regarded as having enough resolution since the rate of change, for example of color, is low indicating that the color is rather uniformly distributed, which only requires low resolution to be seen with enough perception by a spectator.
  • Figure 6 depicts an example of embodiment of the above scheme for evaluating granularity of images and/or video clips.
  • Figure 6 depicts the section selected in figure 5.
  • a measuring line is drawn in the Y direction.
  • the measuring line is divided into sections corresponding to the granularity definition. For example, if a granule is defined as being 5x5 pixels, the measuring line is divided into sections that each are five pixels long.
  • the measuring line is then moved to the left in the X direction, as indicated in figure 6, thereby traversing the image in the X direction.
  • the movement of the measuring line is stepped with a number of pixels corresponding to the granule definition. In this example of embodiment, this means a step length of five pixels.
  • the color, intensity etc. of a granule may be read by a program by reading the content of a memory comprising the image or video frame currently being displayed on a computer display as known to a person skilled in the art.
  • the pixels constituting a granule may each have different parameter values such as colors.
  • a mean value of a selected parameter for all pixels in a granule may be used as a measure of the parameter for that particular granule.
  • each granule traversed by each of the sections on the measuring line is measured, and the counting of a selected parameter such as color is updated for each section on the measuring line when the parameter or at least one of a set of parameters change when the measuring line is moved.
  • these measures may be combined, for example as a mean value to form a measure of detail for the images or video clips under investigation.
  • a defined threshold value specific for the video file at present being used By comparing this measure with a defined threshold value specific for the video file at present being used, a decision of replacing the analyzed section with a corresponding section from a video file with higher resolution is made. Based on this evaluation it is possible to produce an optimized video file according to the present invention.
  • the measuring of detail as described above is performed on all images or all frames constituting a video file thereby producing an optimized video file comprising high definition sections of all items requiring a high resolution, while all section requiring a lower resolution is rendered with the resolution provided by the low resolution version of the file.
  • the granule definition is by default the same as the resolution provided in the video file.
  • the threshold level defining the measure of detail is by default proportional to twice the resolution provided by the video file.
  • the default values may be adjusted after a trial run with these settings. In this manner a user may trim these values to reflect a best choice for a particular video file just by visually inspecting the resulting optimized video file.
  • the process of generating an optimized video file is performed in an iterative manner. After an initial run with for example the video editor illustrated in figure 2 and 3, the resulting video file is entered in to the editor again as the low resolution version of the video file under production. By marking sections to be replaced by corresponding high resolution sections, the quality of the video may further be enhanced. This process may be repeated until the quality may no longer be enhanced.
  • the measure of detail is implemented as a function in the video editor and may be used to automatically to evaluate the decision of replacement of low resolution sections with high resolution sections.
  • At least one low resolution version of a video file and a high resolution version of the same file are entered into an editor. If the files are initially compressed, the files are decompressed accordingly. A user is marking sections that are to be preferably rendered with high resolution in the resulting optimized video file. The resulting video file is then measured to provide a measure of granularity of the resulting optimized file. If the measure is below a user selected level, the resulting optimized video file is used as the final result. If the measure of granularity is above the user selected level, the resulting optimized file is automatically entered into the editor again as the low resolution version of the video file, and the user is instructed by a message in the editor to perform his selections once more.
  • the problematic frames identified through the calculation of granularity, comprising too high granularity are displayed on the time line, as known to a person skilled in the art. hi this manner the system may automatically adjust problematic frames providing a higher resolution for such frames by automatically replacing these frames with higher resolution versions of the frames.
  • the replacement when the editor receives a plurality of the same video file comprising different resolutions, the replacement may be performed iteratively. When a frame is identified as being problematic the frame is replaced with a corresponding frame from the next level of higher resolution. The editor is then used again to produce a resulting optimized video file. If the calculation of granularity again reveals frames being problematic and displayed again on the time line, the replacement frames are taken from the next level of resolution, and so forth. The process terminates when the calculation of granularity no longer reveals problematic frames.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

La présente invention se rapporte à un procédé et à un système permettant l'optimisation de fichiers vidéo numériques s'agissant de la longueur des fichiers vidéo et de leur résolution ou de la qualité vidéo. Conformément à la présente invention, une mesure de la qualité vidéo est effectuée sur la base d'une erreur de quantification introduite dans la représentation numérique de la vidéo. Cette mesure ou granularité est utilisée pour décider si une section d'une version à faible résolution dudit fichier vidéo numérique doit être remplacée par une version haute résolution de la même section.
PCT/NO2006/000136 2005-04-12 2006-04-11 Optimisation de fichiers video WO2006110046A1 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US67036805P 2005-04-12 2005-04-12
US60/670,368 2005-04-12
US20051807 2005-04-13

Publications (1)

Publication Number Publication Date
WO2006110046A1 true WO2006110046A1 (fr) 2006-10-19

Family

ID=37087794

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/NO2006/000136 WO2006110046A1 (fr) 2005-04-12 2006-04-11 Optimisation de fichiers video

Country Status (1)

Country Link
WO (1) WO2006110046A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016154366A1 (fr) * 2015-03-23 2016-09-29 Arris Enterprises, Inc. Système et procédé permettant de compresser sélectivement des images

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0613297A2 (fr) * 1993-02-23 1994-08-31 Matsushita Electric Industrial Co., Ltd. Magnétoscope numérique pour télévision haute définition avec des modes de reproduction spéciaux
US5576902A (en) * 1993-01-13 1996-11-19 Hitachi America, Ltd. Method and apparatus directed to processing trick play video data to compensate for intentionally omitted data
US6147709A (en) * 1997-04-07 2000-11-14 Interactive Pictures Corporation Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience
US20020110197A1 (en) * 1997-11-20 2002-08-15 Hitachi America, Ltd. Methods and apparatus for representing different portions of an image at different resolutions
WO2002093916A2 (fr) * 2001-05-14 2002-11-21 Elder James H Capteur visuel panoramique
US20050193311A1 (en) * 1996-10-25 2005-09-01 Arnab Das Content-based video compression

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5576902A (en) * 1993-01-13 1996-11-19 Hitachi America, Ltd. Method and apparatus directed to processing trick play video data to compensate for intentionally omitted data
EP0613297A2 (fr) * 1993-02-23 1994-08-31 Matsushita Electric Industrial Co., Ltd. Magnétoscope numérique pour télévision haute définition avec des modes de reproduction spéciaux
US20050193311A1 (en) * 1996-10-25 2005-09-01 Arnab Das Content-based video compression
US6147709A (en) * 1997-04-07 2000-11-14 Interactive Pictures Corporation Method and apparatus for inserting a high resolution image into a low resolution interactive image to produce a realistic immersive experience
US20020110197A1 (en) * 1997-11-20 2002-08-15 Hitachi America, Ltd. Methods and apparatus for representing different portions of an image at different resolutions
WO2002093916A2 (fr) * 2001-05-14 2002-11-21 Elder James H Capteur visuel panoramique

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2016154366A1 (fr) * 2015-03-23 2016-09-29 Arris Enterprises, Inc. Système et procédé permettant de compresser sélectivement des images
US10277906B2 (en) 2015-03-23 2019-04-30 Arris Enterprises Llc System and method for selectively compressing images

Similar Documents

Publication Publication Date Title
Winkler Perceptual video quality metrics—A review
CN109416832B (zh) 高效的基于直方图的亮度外观匹配
Nuutinen et al. CVD2014—A database for evaluating no-reference video quality assessment algorithms
Miyahara et al. Objective picture quality scale (PQS) for image coding
US6989868B2 (en) Method of converting format of encoded video data and apparatus therefor
RU2607981C2 (ru) Устройства и способы для анализа градуировок изображения
CN104255029B (zh) 产生用于不同的颜色取样格式的子像素值
EP1851683B1 (fr) Traitement intermediaire numerique (di) et distribution avec compression echelonnable dans le domaine de la post-production de films
US10701359B2 (en) Real-time content-adaptive perceptual quantizer for high dynamic range images
CN112913237A (zh) 使用深度神经网络的人工智能编码和人工智能解码方法和设备
US20110090950A1 (en) System and method for enhancing data compression using dynamic learning and control
CN108885783A (zh) 编码和解码可逆制作质量单层视频信号
US20170140513A1 (en) Efficient Image Processing on Content-Adaptive PQ Signal Domain
CN112889282B (zh) 用于对图像执行人工智能编码和人工智能解码的方法和设备
TWI733986B (zh) 用以編碼和解碼視頻資料之方法、設備及系統
KR20150010903A (ko) 모바일 단말 화면을 위한 3ｋ해상도를 갖는 디스플레이 영상 생성 방법 및 장치
US6804403B1 (en) Region-based scalable image coding
TWI390984B (zh) 用以於轉換域中次取樣影像之裝置及方法
CN1498501A (zh) 视频数据压缩和解压缩的方法
TWI812874B (zh) 張量乘積之b平滑曲線預測子
US20150326878A1 (en) Selective perceptual masking via scale separation in the spatial and temporal domains using intrinsic images for use in data compression
JP5841944B2 (ja) スケーラブルな画像符号化および復号化
US20120170663A1 (en) Video processing
JP2006262294A (ja) 画像処理装置、画像処理方法、プログラムおよび記録媒体
CA2337430C (fr) Codage d'image agrandissable base sur la region

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application
NENP Non-entry into the national phase

Ref country code: DE

NENP Non-entry into the national phase

Ref country code: RU

122 Ep: pct application non-entry in european phase

Ref document number: 06733109

Country of ref document: EP

Kind code of ref document: A1