JP2005535200A - Personal TV optimization - Google Patents

Abstract

Methods, systems, and computer readable media are provided for modeling personal television systems. A first functional portion corresponding to encoding of a video signal before storage in a personal television system, wherein a computer-implemented model of a video processing chain is formed, including processing steps performed on a received video signal by a personal television system; One or more additional functions are inserted into the model, including a second functional part corresponding to the decoding of the stored encoded video data in the personal television system.

Description

  The present invention relates to modeling and optimization techniques for televisions.

Regardless of whether it is being transmitted wirelessly or distributed via a cable television (CATV) system, analog television broadcast signals can be received and recorded for later playback by the viewer using digital compression and A number of “personal television” (PTV) systems that can use storage technology are on the market. Three such systems include “TiVo ^™ ” sold by Philips and Sony, “Replay-TV ^™ ” sold by Matsushita (Panasonic), and “Ultimate-TV ^™ ” sold by Microsoft. Is mentioned. These systems perform the same main function that viewers have traditionally used video cassette recorders (VCRs), ie time-shifting broadcast programs, but viewers record a second program. While providing the additional advantage of allowing the first program to be viewed, and the position of the recorded program within a linear videotape, and the position and length of the tape available to record a new program It's freed up by remembering that. As such, these systems are becoming increasingly popular and this trend will likely continue as long as analog form broadcast video transmission and distribution continues to be utilized.

  A PTV system typically receives analog radio frequency (RF) broadcast signals (although the analog output of a satellite television decoder may be used), and those signals are demodulated and analog video decoded (NTSC, PAL). Etc.) and then convert to digital format. These signals are then compressed using a lossy digital video compression scheme, such as MPEG 1 or 2, and recorded in a mass storage device such as a high density hard disk drive (HDD). By using compression technology along with the high data capacity of recent storage devices (which continues to increase very rapidly with time), it becomes possible to record hours of broadcast video on the HDD, thereby enabling analog video recording. The need for tape for use has been eliminated and the advantages of digital storage described above have been provided.

  In general, video processing systems are required to improve the image quality of sources that are transmitted as either analog broadcast or digital video, but not both. In the case of analog broadcast video, image quality is typically degraded by channel noise, which can include, for example, Gaussian noise whose spectral shape is determined by the transmission channel and the demodulation and analog decoding of the video signal. Other features such as color accuracy and image sharpness can also be affected by this process. In the case of digital video, there is no noise (except for a small amount of noise captured before digitization of the video signal), and color accuracy is not affected. However, lossy compression and subsequent decoding can introduce many video artifacts, such as block impairments, and can also affect image sharpness. Conversely, such artifacts cannot exist in an analog broadcast video source (except when the source is originally in digital form. In such cases, the source is typically encoded using a low compression ratio. Therefore, the image quality is high and there is almost no deterioration of the image quality). Therefore, conventionally, only the image quality degradation of one of the categories has appeared in the received video broadcast, but not both.

  However, in a PTV system, analog broadcast signals are demodulated and decoded in a conventional manner, and thus can have any or all of the image quality degradation of conventional analog formats. The signal is then compressed at the highest acceptable compression ratio to use as little HDD space as possible. As a result, when the video is later read from the HDD and decoded, any or all of the image quality degradations in the digital format described above are captured in the video. Such image degradation can be significant at the high compression ratios commonly used in these systems. As a result, when viewed by the user, the same video sequence will have both analog broadcast image quality degradation and digital encoding image quality degradation, and either or both of these image quality degradations will be extremely prominent. It may become. This is a situation that is clearly different from the conventional situation.

  Many methods have been proposed to optimize image quality in video processing systems. General methods for automatic video chain optimization are “Automated Video Chain Optimization” by Van Zon, Kees and Ali, Walid, International Conference on Consumer Electronics; ICCE ), 2001, and Ali, Walid and Van Zon, Kees, “Optimizing a Random System of Cascaded Video Processing Modules by Serially Connected Video Processing Modules.” Parallel Evoluti on Modeling ”, International Conference on Image Processing, 2001. All of these documents are hereby incorporated by reference in their entirety. The reference video sequence is processed using the initial video processing chain of processing operations and the image quality is measured. Usually, an optimization technique such as a genetic algorithm that makes it possible to perform overall optimization over a large parameter search space is a method for specifying an objective image quality (OIQ) of a resultant image. Applied with. Such methods have been applied or proposed for analog video processing systems designed to improve the quality of analog broadcast video or digital video systems designed for digitally encoded video.

  For analog video, techniques such as channel noise reduction (temporal and / or spatial) and peaking for sharpness enhancement (horizontal and sometimes vertical) are often used to improve image quality. Specifically, a “chain” of video processing performs these and other video functions. Such chain optimization requires the setting of parameters for all these functions so as to simultaneously provide the best possible image quality for a given function.

  For digital video, there is a well-known phenomenon that small differentials appear as a result of noise between corresponding regions of different images, which are still subject to encoding and thereby limited data bandwidth In some cases, some form of noise reduction is used prior to compression coding. This result in addition to noise retention is equivalent to reducing the effective encoded data rate and is equivalent to increasing the compression ratio. The visual degradation of image quality, which is usually a direct result of such low noise levels, is a secondary matter to consider in these systems. In some systems, a method of reducing the effect of block deterioration that appears as a result of encoding at a high compression ratio after decoding is used. By discarding the high frequency components that can also occur at these high ratios, the sharpness of the image is also significantly reduced, and some form of sharpness enhancement may be desirable.

  Therefore, an optimization method for PTV is desired.

  Methods, systems, and computer readable media are provided for modeling personal television systems. A first functional portion corresponding to encoding of a video signal before storage in a personal television system, wherein a computer-implemented model of a video processing chain is formed, including processing steps performed on a received video signal by a personal television system; One or more additional functions are inserted into the model, including a second functional part corresponding to the decoding of the stored encoded video data in the personal television system.

  A complete PTV system having the functions of encoding, storing, reading, decoding and processing of digital video has the above-mentioned first to fourth elements (encoding, storing, reading and decoding in PTV) as a video processing chain. , The analog video system can be optimized using techniques based on what has been described above. In particular, encoding is typically performed using the desired data rate (or compression ratio) as the main (and optionally only) system parameter. On the other hand, decoding is a function of the extracted video data stream, and therefore has no adjustable parameters.

  Finally, as the data flow representation of video data is used in the optimization method, as recently more and more common, storage media writing and reading are simply time delays in the video data stream. Therefore, it can be completely ignored. MPEG format encoding and decoding is a digital processing algorithm that can be modeled as such in the context of this optimization, but in addition, since this method is a data driven method, the data can be stored on a hard disk. When reading data thereafter, there is no concept of time. Therefore, writing data that is followed by reading is only a time delay. Since the encoding, storing, reading and decoding are always performed as directly continuous, these four elements of the digital processing system can be treated as a single function.

  Furthermore, a relatively simple model of PTV encoding, storing, reading and decoding processing operations can change a single parameter during the optimization process to provide effective optimization. An example of such a single parameter is the compression ratio. Since there is a one-to-one relationship between the compression ratio and the data rate, the data rate may be used instead of the compression ratio. Alternatively, a more complex model of PTV encoding, storage, reading and decoding processing operations that changes two or more parameters during the optimization process may be used. FIG. 1 is a block diagram illustrating a system for optimizing a PTV system.

  A PTV system can be modeled using a combination of the respective functions described above for each type of system (ie, analog and digital). However, in PTV systems, noise reduction serves two functions: reducing analog channel noise and improving digital coding efficiency. In addition, the sharpness enhancement needs to cancel both the effects of the distortion of the analog frequency due to channel, demodulation and decoding, and the discarding of high frequency information due to digital compression.

  Proper optimization of such a system entails more demand than simply combining individual optimizations of analog and digital systems. Since the functions executed in the video processing chain are non-linear, the order in which multiple functions are executed in the model reflects the order in which those functions are executed in the actual video processing chain. Should be done and the order of processing operations (in addition to their algorithm parameters and data bit precision) can be optimized. The PTV system can be modeled by adding only one additional element to the elements of the conventional video processing chain, ie the elements of encoding, storing, reading and decoding functions. This element also has only one parameter, specifically the compression ratio. Thus, the PTV system can be modeled by modifying the modeling methods that have been proposed and used in conventional video processing chains without significantly increasing the optimization complexity. For this reason, such methods can be used to optimize the overall image quality of a type of video processing system that is expected to be of increasing commercial significance.

  Referring to FIG. 1, an exemplary optimization system is shown. This system includes a video processing chain simulator 100. The simulator 100 is data driven as opposed to event driven simulation. The simulator 100 performs a digital signal processing operation substantially the same as the processing operation performed by the actual video processing system in the PTV on the input reference video. However, the simulator 100 differs from an actual video processing system only in that these processing operations do not have to be executed in real time.

  Data input to the video processing chain simulator 100 is data in a digital format. When the PTV receives an analog transmission signal, the video input data represents the analog transmission signal after being demodulated and digitized into a baseband video signal. During optimization, the same video sequence is used as input throughout the iterative modeling and optimization process.

  The video processing chain simulator 100 includes a plurality of video processing algorithms 110a to 110n. For example, algorithm 1 (block 110a) may be noise reduction and algorithm 2 (block 110b) may be sharpness enhancement, thus continuing to algorithm N (block 110n) and algorithm N (block 110n) may be removal of the MPEG blocking effect. The choice of algorithm is driven by the algorithm used in the PTV system to be modeled.

  At least one additional function 120 is added to represent the processing performed during the encoding, storing, reading and decoding functions. The coding portion of this function 120 should be modeled on the actual coding hardware used in the optimized PTV system. In practice, an MPEG format encoder chip may be used. Since such a chip uses a certain encoding algorithm, a software model of an algorithm implemented by the chip is used. Since one purpose of MPEG is to obtain as good compression as possible, i.e. to obtain as good image fidelity as possible, there are a variety of different models using MPEG-style encoders. This requires that as many pixels as possible be processed efficiently, that is, as much pixel information as possible using a minimum number of encoded bits. There are many possible and proper encodings for any video sequence. Therefore, the encoder model should be modeled on the actual hardware used in the system.

  The decoder part of function 120 was implemented directly from the MPEG1 standard (ISO / IEC 11172-1 (from -5): 1993) or MPEGII standard (ISO / IEC 13818-4: 1998 / Amd2: 2000). May be. The contents of these standards are hereby incorporated by reference. Since the various parts of the MPEG standard are defined as C code segments or blocks, one skilled in the art could easily employ a decoder C code to model an MPEG decoder.

  As described above, the storage and retrieval processing operations only correspond to delays and are not modeled during simulation. Neither processing operation converts data or affects image quality. The decoded output data from the video processing chain simulator 100 is supplied to the objective image quality measurement block 130. An objective image quality metric typically consists of an image quality measure given by the objective metric for an image or video sequence using a predetermined set of images or video sequences, and a subjective Selected and adjusted by giving a good correlation with the typical image evaluation. An objective image quality metric may provide a measure of image quality that takes into account, for example, analog noise, blocking, ringing, mosquito artifacts and other types of artifacts. This metric can also be applied to different weightings for the effects of each of the above factors on image quality. In that case, adjusting the weights can improve the correlation between the measures given by the objective metric and the subjective assessment.

  Algorithm parameters, processing ordering, data bit accuracy, and resulting objective image quality measurements are parameters that are relevant to the purpose of the optimization. Actual video output data may not be stored for optimization purposes.

  Block 140 is a video processing system optimization block. Desirable attributes of this optimization block 140 include accuracy and speed. Since there are so many different combinations of parameter values and processing ordering, it is not practical to try every single combination to find the overall optimum. On the other hand, even if there is a larger overall maximum or a smaller overall minimum, algorithms that tend to converge to local maxima or minima (eg, Newton's method) should not be used.

  A genetic algorithm is one preferred approach for the video processing chain optimization block 140. Genetic algorithms use an iterative and non-restrictive approach based on evolution that can evolve towards the overall optimum without requiring prior information about the search space. The genetic algorithm generates a set of solution candidates called “chromosomes” each consisting of a plurality of “genes”. Each gene corresponds to a particular algorithm parameter value (or a subset of algorithm parameter values), processing ordering, or data bit precision.

  A video chain is formed for each chromosome within a generation. This is because all blocks in a given desired order, a given set of data bit precision values, and a given set of algorithm parameter values are selected, and a simulation 100 is performed to provide objective image quality. It means that the number of the measurement 130 (“fitness value”) is given to the genetic algorithm. Based on the values of the objective image quality measurement 130, the subset of chromosomes with the optimal image quality measurement is selected and combined by “crossover” to form the next generation. In crossover, a subset of algorithm parameter values, parameter ordering, and data bit precision (ie, a subset of genes) are exchanged between chromosomes. A “mutation” is then introduced by perturbing some of the genes with some (usually low) probability defined by the user. Mutation ensures that no part of the search space is likely to be searched, thereby converging to a local minimum or local maximum when a better overall optimal solution exists. To reduce the possibility of losing. The entire set of next generation chromosomes is ready for immediate processing and evaluation.

  Once the value to be used for the next generation is selected, the video processing chain control block 150 further encodes, stores, reads and stores each of the video processing algorithms 110a-110n in addition to processing ordering and data bit precision. A parameter value is set for the decoding function 120. Thus, the video processing chain simulator 100 can be re-executed using the same set of input video data for each of the next generation chromosomes.

  The criterion for termination of the iterative process may be a criterion based on the arrival of an acceptable approximate solution or a stable solution, or a predetermined number of mutation iterations, a predetermined number of generation iterations, etc. It may be a criterion based on.

  Genetic algorithms are attractive because they give accurate results despite requiring only a relatively small portion of the entire search space to be evaluated (resulting in fast convergence). However, those skilled in the art could use other optimization algorithms for the video processing system optimization block 140.

  FIG. 2 illustrates one method for modeling a personal television system.

  In step 200, a computer-implemented model 100 of a video processing chain is formed that includes processing steps 110a-110n performed on a received video signal by a personal television system. This model allows for adjustment of algorithm parameters, processing ordering and data bit accuracy.

  In step 202, an encoder-specific functional part corresponding to the encoding of the video signal before storage in the personal television system is inserted into the video processing chain model 100.

  In step 204, the simulation can ignore storage and retrieval. This is because none of these processing operations convert image data or affect image quality.

  In step 206, a functional portion corresponding to the decoding of the MPEG1 format or the MPEG2 format of the data stored in the PTV is inserted into the video processing chain model 100. In some embodiments, steps 202 and 206 are integrated to form a single function corresponding to encoding and decoding, and storage and reading are ignored.

  Steps 208-214 provide iterative optimization. In step 208, the video processing chain model 100 is executed.

  In step 210, the simulation results are evaluated against an objective image quality metric.

  In step 212, the model is adjusted according to an optimization algorithm, which may be, for example, a genetic algorithm. This adjustment is an adjustment of the encoding / decoding function using the compression ratio (or data rate) as an adjustment parameter in step 214, and this compression ratio (or data rate) may be the only adjustment parameter. Adjustments may be included.

  Steps 208-214 are repeated until the termination criteria are met.

The present invention may be implemented in the form of computer-implemented processes and apparatuses for performing the processes. The present invention also includes random access memory (RAM), floppy disk, read only memory (ROM), CD-ROM, DVD-ROM, hard drive, high density (eg, “ZIP ^™ ” or “JAZZ ^™ ”). Tangible, such as a removable disk or any other computer-readable storage medium that, when the computer loads and executes the computer program code from the medium, makes the computer an apparatus embodying the invention It may be implemented in the form of computer program code embedded in the medium. The invention also includes any transmission medium, such as stored in a storage medium, loaded and / or executed by a computer, electrical conductors or cables, optical fiber, or electromagnetic radiation. Implemented in the form of computer program code such that when the computer loads and executes the computer program code, whether or not it is transmitted via the computer, the computer becomes an apparatus embodying the present invention. May be. When implemented on a general-purpose processor, the computer program code segments configure the processor so that a specific logic circuit is formed. Although the invention has been described in the form of exemplary embodiments, it is not limited to those forms. Rather, the claims should be construed broadly to include other variations and embodiments of the invention that would occur to those skilled in the art without departing from the scope of the invention and its equivalents. .

Block diagram of a system that optimizes the video processing chain of a personal television Flow chart of processing to optimize the video processing chain of personal TV

Claims (27)

A method of modeling a personal television system,
(A) forming a computer-implemented model of a video processing chain including processing steps performed on the received video signal by the personal television system;
(B) a first functional part corresponding to the encoding of the video signal before storage in the personal television system, and a second functional part corresponding to decoding of the encoded video data stored in the personal television system. Inserting one or more additional functions, including functional parts, into the model.   The method of claim 1, wherein the first functional portion includes an encoder specific model corresponding to a given encoder included in the personal television system.   3. The method according to claim 2, wherein the second functional part includes an MPEG1 format decoder model or an MPEG2 format decoder model.   The method of claim 1, wherein the first functional part and the second functional part are contained in a single function.   5. The method of claim 4, wherein the model ignores storage and retrieval of the encoded video signal within the personal television system.   5. A method according to claim 4, characterized in that the compression ratio of the stored encoded video signal is used as a parameter for adjusting the single function. (C) after the step (b), executing the computer-implemented model;
The method of claim 1, further comprising: (d) adjusting the model of the video processing chain based on the output generated by step (c).   The method of claim 7, wherein step (d) comprises selecting the adjustment using a genetic algorithm.   8. The method of claim 7, further comprising evaluating the output generated by step (c) using an objective image quality metric. A computer on which the computer program code is encoded and stored such that when the computer program code is executed by a processor, the processor executes a method for modeling a video processing chain in a personal television system. A readable medium, the method comprising:
(A) forming a computer-implemented model of a video processing chain including processing steps performed on the received video signal by the personal television system;
(B) a first functional part corresponding to the encoding of the video signal before storage in the personal television system, and a second functional part corresponding to decoding of the encoded video data stored in the personal television system. A computer-readable medium comprising: inserting one or more additional functions, including functional parts, into the model.   The computer-readable medium of claim 10, wherein the first functional portion includes an encoder specific model corresponding to a given encoder included in the personal television system.   12. The computer readable medium of claim 11, wherein the second functional portion includes an MPEG1 format decoder model or an MPEG2 format decoder model.   The computer-readable medium of claim 10, wherein the first functional portion and the second functional portion are contained in a single function.   The computer-readable medium of claim 13, wherein the model ignores storage and retrieval of the encoded video signal in the personal television system.   14. The computer readable medium of claim 13, wherein a compression ratio of the stored encoded video signal is used as a parameter for adjusting the single function. Said method comprises
(C) after the step (b), executing the computer-implemented model;
11. The computer readable medium of claim 10, further comprising: (d) adjusting the model of the video processing chain based on the output generated by the step (c).   The computer-readable medium of claim 16, wherein step (d) comprises selecting the adjustment using a genetic algorithm.   The computer-readable medium of claim 16, wherein the method further comprises evaluating the output generated by step (c) using an objective image quality metric. A system for modeling a personal television system,
Including a computer programmed with a model of a video processing chain, including processing steps performed on the received video signal by the personal television system;
The computer includes, in the model, a first functional part corresponding to encoding of the video signal before storage in the personal television system, and decoding of encoded video data stored in the personal television system. A system comprising one or more additional functions including a second functional part corresponding to.   20. The system of claim 19, wherein the first functional part includes an encoder specific model corresponding to a given encoder included in the personal television system.   21. The system of claim 20, wherein the second functional part includes an MPEG1 format decoder model or an MPEG2 format decoder model.   20. The system of claim 19, wherein the first functional part and the second functional part are included in a single function.   23. The system of claim 22, wherein the model ignores storage and retrieval of the encoded video signal within the personal television system.   23. The system of claim 22, wherein a compression ratio of the stored encoded video signal is used as a parameter for adjusting the single function.   20. The system of claim 19, further comprising automatic means for adjusting the model of the video processing chain based on output generated by executing the model.   26. The system of claim 25, wherein the automatic means selects the adjustment using a genetic algorithm.   The system of claim 25, further comprising means for evaluating an output generated by executing the model using an objective image quality metric.

Images