AU2005207982A1 - Systems and methods for encoding and decoding video information - Google Patents

Systems and methods for encoding and decoding video information Download PDF

Info

Publication number
AU2005207982A1
AU2005207982A1 AU2005207982A AU2005207982A AU2005207982A1 AU 2005207982 A1 AU2005207982 A1 AU 2005207982A1 AU 2005207982 A AU2005207982 A AU 2005207982A AU 2005207982 A AU2005207982 A AU 2005207982A AU 2005207982 A1 AU2005207982 A1 AU 2005207982A1
Authority
AU
Australia
Prior art keywords
pixel
values
encoded
segment
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
AU2005207982A
Inventor
Adam Leslie Clark
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Adams Platform Pty Ltd
Original Assignee
Adams Platform Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US10/771,096 external-priority patent/US20050180641A1/en
Priority claimed from US10/770,432 external-priority patent/US20050169365A1/en
Priority claimed from US10/770,952 external-priority patent/US20050196049A1/en
Priority claimed from US10/771,097 external-priority patent/US7505045B2/en
Application filed by Adams Platform Pty Ltd filed Critical Adams Platform Pty Ltd
Publication of AU2005207982A1 publication Critical patent/AU2005207982A1/en
Abandoned legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/98—Adaptive-dynamic-range coding [ADRC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component

Description

WO 2005/073920 PCT/AU2005/000108 SYSTEMS AND METHODS FOR ENCODING AND DECODING VIDEO INFORMATION RELATED APPLICATIONS [0001] This application is related to and hereby claims the priority benefit of the following 5 United States patent applications, each of which is incorporated herein by reference: 1. U.S. Patent Application 10/770,952, filed 2 February 2004; 2. U.S. Patent Application 10/771,097, filed 2 February 2004; 3. U.S. Patent Application 10/771,096, filed 2 February 2004; and 4. U.S. Patent Application 10/770,432, filed 2 February 2004. 10 FIELD OF THE INVENTION [0002] The present invention relates generally to communication systems and, in particular, to a system. and method for compressing video data for transmission over low bandwidth communication links and subsequently decompressing the compressed data at a receiver. 15 BACKGROUND OF THE INVENTION [0003] Data compression is widely used in a variety of contexts for removing redundancies from data. Redundancies can come in a variety of forms, for example repeated bit or byte sequences, or, often, on larger scales. All data compression methodologies seek to transform one 20 data representation into another, more compact representation for specific data sets. Video signals, for example, can be digitized, encoded, and subsequently decoded in a manner which significantly decreases the number of bits necessary to represent a decoded reconstructed video without noticeable, or with acceptable, degradation in the reconstructed video. Video coding is an important part of many applications such as digital television transmission, video conferencing, video 25 databases, etc. [0004] Data compression techniques can be lossy or lossless. Lossless compression involves a transformation of the representation of a data set such that it is possible to reproduce exactly the original data set by performing a decompression transformation. Lossy compression is a representation that allows one to reproduce something that only approximates the original data set. 30 Lossy compression techniques can frequently produce more compact data representations than lossless compression techniques can but are suitable only in instances where the literal bit-pattern of a digital file is not required. [0005] Many examples of both lossy and lossless compression methods exist in the art. For example, run-length encoding (RLE) is a widely used lossless compression technique. RLE makes WO 2005/073920 PCT/AU2005/000108 -2 use of the fact that many data representations consist largely of strings of repeated bytes. Thus, rather than represent each character with its own byte, RLE simply replaces the original data set with an iteration count followed by the character to be repeated. [0006] When it comes to very large data files, for example digital audio/video files, very 5 sophisticated compression techniques have been developed. For example, the Motion Picture Experts Group (MPEG) has promulgated a number of interoperability standards that describe schemes for compressing and decompressing digital audio/video information for many applications. Likewise, the International Telecommunications Union (ITU) has published its own set of interoperability standards for audio/video compression. Such encoding processes are 10 typically implemented using a digital video coder/decoder (codec), which divides the images into blocks and compresses the blocks according to a compression standard, such as the ITU-T H.263 and H.261 standards. In compression schemes of this type, a block may be compressed independent of the previous image or as a difference between the block and part of the previous image. ITU-T Recommendation H.263 in particular adopts a hybrid scheme of motion-compensated prediction to 15 exploit temporal redundancy and transform coding using the discrete cosine transform (DCT) of the remaining signal to reduce spatial redundancy. Half pixel precision is used for the motion compensation, and variable length coding is used for the symbol representation. [0007] Despite the variety of existing compression/decompression methodologies, however, there remains a problem in that these methodologies still do not provide sufficiently small files to 20 be transported using low-bandwidth communication links in acceptable times. For example, data streams are often transmitted over a variety of communication channels, including twisted pair telephone lines, coaxial cable, and wireless channels via local transceivers or orbiting satellites. Very often one or more relatively low-bandwidth communication channels, mainly twisted pair telephone lines, exist in these communication paths, constraining the type of information that may 25 be presented to users. As a result, users that are limited to accessing resources such as Internet web servers via these low-bandwidth communication links often are unable to achieve satisfactory user experiences because download times for audio/video files are too long. Thus, there is a need for new data compression, and, in particular, video data compression, methodologies. 30 SUMMARY OF THE INVENTION [0008] In one embodiment, the present invention provides a scheme for encoding data values by mapping multi-dimensional parameters of the data values to respective one-dimensional parameters, and creating a table of encoded data values in which the data values are represented by WO 2005/073920 PCT/AU2005/000108 -3 their respective encoded counterparts utilizing the one-dimensional parameters and in which redundant ones of the encoded data values share common table entries. [0009] As applied in the context of video data in one embodiment of the present invention, frames of video data are encoded, segment-by-segment, wherein each frame includes a number of 5 pixels and each pixel having a plurality of pixel color components. The encoding process includes creating a frame group table of encoded pixel values in which each pixel entry includes a dominant pixel color component of the plurality of pixel color components. For each encoded segment of the frame, the encoder determines a set of segment reference pixels, wherein each segment reference pixel is a pixel within the segment having a most intense dominant pixel color. 10 [00010] The encoded data may be decoded, on a pixel-by-pixel basis, using a table of encoded pixel parameter values, wherein each pixel is represented by an entry including a dominant pixel color component and by scaling a set of segment reference pixels comprised of segment reference pixel values according to each entry in the table of encoded pixel parameter values. Each set of the segment reference pixels may correspond to an encoded segment of a frame and the set of the 15 segment reference pixel values comprises a unique set of color pixels for each encoded segment. That is, each segment reference pixel may represent a pixel with a most intense dominant pixel color component for each encoded segment (e.g., a red pixel, green pixel, blue pixel, and black pixel). [00011] The encoding/decoding system may include an encoder, a server, and a decoder, with 20 the encoder configured to perform the segment-by-segment encoding, the server configured to communicate the frame group table and the segment reference pixels over a network to a receiver, and the decoder coupled to the receiver and configured to decode the frame group table on a pixel by-pixel basis by scaling the segment reference pixel parameter values according to each entry in the frame group table of encoded pixel parameter values to produce decoded pixels. 25 [000121 In one embodiment of the present invention, prior to encoding the pixel data, the encoder creates a frame group file to store a header, a frame table and the segment reference pixels. In various embodiments, the encoder may also store any associated audio/video data, synchronization information, or tags in the frame group file or may create one or more separate files for such information. The plurality of pixel color components stored within the table may 30 include any or all of luminance, chrominance, and color depth information. Prior to entry within the frame group table, the encoder scales down each entry to reduce the amount of stored data. In another embodiment, the non-dominant color values are also scaled down and entered into the table.
WO 2005/073920 PCT/AU2005/000108 -4 [00013] In a further embodiment of the present invention, the set of segment reference pixels may be determined by comparing, on a pixel by pixel basis for each segment (e.g., a frame, a line of a frame, a half of a frame, or other fraction of a frame), a current pixel color value with a previously stored dominant pixel color value and storing the plurality of pixel color components 5 and pixel parameters of the pixel with the most intense dominant pixel color component. The plurality of pixel color components may include at least one of the sets of primary color components: red, green, and blue; or cyan, magenta, and yellow, and may also include the primary color components and black. Black may be determined by comparing an average or aggregation of the pixel primary color component values to a black threshold value. 10 [00014] During encoding the encoder may write a pointer to a next frame group within the frame group file to ensure the decoder decodes the frame groups in the correct sequence. Moreover, redundant encoded pixel values of the frame group table may be encoded so as to share common table entries and therefore share identical dominant pixel color components and identical pixel parameter values. In another embodiment, the redundant encoded pixel values share 15 dominant pixel color components and pixel parameters values that are similar to one another within a tolerance range. Each one of the redundant entries may be decoded by recalling the previously decoded pixel parameter values associated with each one of the redundant entries. [00015] The decoding process may include processing a file comprised of a header, the table of encoded pixel parameters, and the segment reference pixels by using the header to determine data 20 locations within the file, including the beginning and end of the table of encoded pixel parameter values and the corresponding segment reference pixel values. BRIEF DESCRIPTION OF THE DRAWINGS [00016] The present invention will be understood and appreciated more fully from the 25 following detailed description taken in conjunction with the drawings in which: [00017] Figure 1 is a block diagram of an exemplary system for compressing streamed or live video, according to one embodiment of the present invention; [00018] Figure 2 illustrates a sequence of video frames with its corresponding raw video data, according to one embodiment of the invention; 30 [00019] Figure 3A illustrates the encoding of a raw video table, according to one embodiment of the present invention; [00020] Figure 3B illustrates a segment reference pixel table, according to one embodiment of the present invention; WO 2005/073920 PCT/AU2005/000108 -5 [00021] Figure 4 illustrates the decoding of a compressed video file, according to one embodiment of the present invention; [00022] Figure 5 is a flow diagram showing an example of an encoding process, according to one embodiment of the present invention; 5 [00023] Figure 6 is a flow diagram illustrating an example of a decoding process, according to one embodiment of the present invention; [00024] Figure 7 illustrates an exemplary network architecture for use according to one embodiment of the present invention; [00025] Figure 8 illustrates an exemplary computer architecture for use according to one 10 embodiment of the present invention; [00026] Figure 9 illustrates an example mapping of a table raw data values having one or more multi-dimensional parameters into a table of encoded data values having only one-dimensional parameters in accordance with an embodiment of the present invention; and [00027] Figure 10 illustrates in further detail the mapping of a multi-dimensional parameter 15 into a one-dimensional parameter in accordance with an embodiment of the present invention. DETAILED DESCRIPTION [00028] A system and method for encoding/decoding video are described. The present system and method overcome prior deficiencies in streaming live video content by encoding and decoding 20 video data such that high-quality video transmission over low bandwidth communication links is possible. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be evident, however, to one skilled in the art that the present invention may be practiced without these specific details. In some instances, well-known structures and devices are shown in block diagram form, 25 rather than in detail, in order to avoid obscuring the present invention. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention, and it is to be understood that other embodiments may be utilized and that logical, mechanical, electrical, and other changes may be made without departing from the scope of the present invention. [00029] Some portions of the detailed descriptions that follow are presented in terms of 30 algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of acts leading to a desired result. The acts are those requiring physical manipulations of physical quantities. Usually, WO 2005/073920 PCT/AU2005/000108 -6 though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, signals, datum, elements, symbols, characters, terms, numbers, or the like. 5 [00030] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout the description, discussions utilizing terms such as "processing" or "computing" or "calculating" or "determining" or "displaying" or the like, refer to the action and 10 processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices. 15 [00031] The present invention can be implemented by an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer, selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD 20 ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, and each coupled to a computer system bus. [00032] The algorithms and processes presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with 25 programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method. For example, any of the methods according to the present invention can be implemented in hard-wired circuitry, by programming a general purpose processor or by any combination of hardware and software. One of skill in the art will immediately appreciate that the invention can be practiced with computer system configurations 30 other than those described below, including hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, DSP devices, network PCs, minicomputers, mainframe computers, and the like. The invention can also be practiced in distributed computing environments where tasks are performed by remote processing devices that WO 2005/073920 PCT/AU2005/000108 -7 are linked through a communications network. The required structure for a variety of these systems will appear from the description below. [00033] The methods of the invention may be implemented using computer software. If written in a programming language conforming to a recognized standard, sequences of instructions 5 designed to implement the methods can be compiled for execution on a variety of hardware platforms and for interface to a variety of operating systems. In addition, the present invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein. Furthermore, it is common in the art to speak of software, in one form or another 10 (e.g., program, procedure, application, etc.), as taking an action or causing a result. Such expressions are merely a shorthand way of saying that execution of the software by a computer causes the processor of the computer to perform an action or produce a result. [00034] Figure 1 is a block diagram of an exemplary system 100 for live video compression, according to one embodiment of the present invention. Video compression system 100 is designed 15 to encode and deliver high quantity live video over low bandwidth transmission links (e.g., 9.6-56 kBps). In one embodiment of the present invention, video compression system 100 obtains video from a live feed source 104, such as a camera recording a live sporting event. Among varying embodiments, the source contributing video data to be streamed "live" may also be any device capable of delivering video content, such as a digital versatile disc (DVD), a computer storage 20 device, or digital video tape. It should be noted, analog video storage devices may also be used so long as the video stored thereon is first converted to a digital format prior to "live" encoding. [00035] A live feed source 104 produces digital output video signals in a raw data file format. Generally, audio signals accompany the video signals from source devices such as live feed source 104. The audio signals may be digitized and/or compressed and provided along with the raw video 25 data, either in a separate file or appended to the video file. In one embodiment of the present invention, the audio data may be processed independent of the raw video data 106 according to any audio compression method including the MPEG's (Moving Picture Experts Group) "MP3" or Microsoft's "way" format. Such audio may be synchronized with the video data file 106 at any point within the compression system 100. 30 [00036] The raw video data 106, including a start stream header, is provided to an encoder 112. The start stream header is included at the start of the stream of raw video data 106 and may include information regarding the audio data and raw video data 106, such as video frame starting and ending points, video tag information, the number of video frames per second, frame resolution (i.e., WO 2005/073920 PCT/AU2005/000108 -8 the number of pixels per frame), color depth information, audio synch information, and similar data regarding the video data stream that may be used by the encoder 112. [00037] Compression system 100 uses the encoder 112 to compress raw video data 106 for streaming video to a decoder at or near real-time. The details of the encoding process performed 5 by encoder 112 will be discussed below. The encoder 112 produces a compressed or encoded video file 114, which may include for each frame group, segment reference pixel values, encoded pixel data for all frames within the frame group, and header information for an encoded frame group, such as resolution settings for the decoder and audio/video synch information. In another embodiment, a trailer within the compressed video file may be generated that may include other 10 audio/video information such as a pointer identifying the next frame group to be decoded. The majority of the compressed video file 114 is a frame group table of pixel parameter values for each pixel of each video frame comprising the acquired video. Encoder 112 may also produce an audio output file that may or may not be compressed, as discussed above. For purposes of this specification, reference to the compressed video file 114 includes any audio/visual data, optional 15 data and/or header and trailer information. It should be appreciated, however, that in other embodiments, the header, the trailer, the compressed video data, and audio data may be written to separate files or appended to one or more files in any combination thereof. [00038] The compressed video file 114 may be transmitted over a network 116 to the decoder 118. The decoder 118 decodes the compressed video file 114 to include a decompressed video file 20 120 and synchronizes the audio data (if any) for audio/visual viewing via playback device 122. Playback device 122 may be any device accepting video data such as a television, cellular phone display, personal computer, personal digital assistant (PDA), automobile navigation system, or other similar device capable of displaying video data. The process performed by the decoder 118 will be described in detail below. 25 [000391 Figure 2 illustrates a sequence of video frames with its corresponding table of raw video data, according to one embodiment of the present invention. Video sequence 200 is composed of a number of video frames 2101-210,. Each video frame 210 is composed of thousands of pixels. The exact number of pixels in a frame depends upon the digital video format and more specifically, the frame resolution used. The present method and system support High 30 Definition Digital TV (HDTV), National TV Standards Committee (NTSC) having 30 interlaced frames per second at 525 lines of resolution with an audio FM frequency and a MTS signal for stereo, Phase Alternating Line (PAL) standards having 25 interlaced frames per second at 625 lines of resolution, System en coleur avec memoire (SECAM) and similar protocols. It should be noted, WO 2005/073920 PCT/AU2005/000108 -9 however, any analog audio/video format is to be converted to a digital audio/video format prior to encoding by encoder 112. [00040] Live feed source 104 generates frames 210 and provides the raw video data file 106 that describes video frames 2101., and their corresponding pixels. The raw video data file 106 5 contains the raw video frame data tables 2201-n, where n represents the number of frames, and each row 231-236 corresponds to pixels in each video frame 210, where the pixels between 235 and the last pixel 236 have been omitted for clarity. The columns of each raw video frame data table 220 describe the pixel numbers 222, red color component values 223, green color component values 224, blue color component values 225 (RGB values), luminance values 226, and chrominance 10 values 227 for each pixel in the respective frame 210. In alternate embodiments, other color spaces may be used such as, cyan, magenta, and yellow (CyMgY1). [00041] As illustrated with reference to Figure 2, each pixel parameter 223-227 of each pixel 222 in each frame table 220 requires multiple bytes of information to be stored per pixel, thus creating large file sizes for multiple frames of video data. Considering high quality video requires 15 at least a frame rate of 25 frames per second or 1,500 frames/minute, it should be apparent that the amount of storage and/or bandwidth required to stream and play an uncompressed or slightly compressed video file is quite large. [00042] Figure 3A illustrates a table of raw video data 2201 encoded into a frame group table 3201, according to one embodiment of the present invention. The pixel numbers in column 2221 20 are mapped to pixel numbers in column 3221. For each row, 231-236, of raw pixel data in table 2201 there is a corresponding row, 331-336, of compressed pixel data in table 3201, where the pixels between 235 (335) and the last pixel 236 (336) have been omitted for clarity. The pixel value sets (i.e., RGB values 2231-225 1) are processed and mapped to dominant color value 3231, as illustrated by pixel 1 value R10 in table 3201. The calculation of the dominant color value 3231 will 25 be discussed below. The luminance value 2261 is mapped to a scaled luminance value 326, and the chrominance value 2271 is mapped to a scaled chrominance value 3271. The calculation of scaled chrominance and luminance values 3271, 326, will also be discussed below. Each compressed video table 320 may further include color depth values 328. In one embodiment, the color depth value 328 is the average of the scaled chrominance and luminance values 327, 326. According to 30 one embodiment, as encoder 120 populates compressed video table 320, if a row of compressed pixel data is determined to be identical or at least sufficiently similar to a previously populated row, encoder 112 places a pointer to the appropriate previously entered row, as illustrated with reference to 334.
WO 2005/073920 PCT/AU2005/000108 - 10 [00043] Figure 3B illustrates pixel reference value sets 3501-n generated by encoder 112, where n is the number of encoded segments for each frame group. The segment reference pixel value sets 350, according to one embodiment of the present invention, may have up to four (4) reference pixel values corresponding to red 361, green 362, blue 363 and black 364 for each segment of a video 5 frame 210. The segment reference pixels are selected based upon the video frame 210's most intense dominant pixel color values for each encoded segment, as illustrated by red 361, green 362, blue 363, and black 364. The most intense dominant pixel color value is based on the highest raw pixel color values. The black segment reference pixel 356, according to one embodiment of the present invention, may be determined by comparing the color component values (e.g., RGB) in 10 aggregate. The segment reference pixel values may also include pixel parameter values, such as luminance value 356, and chrominance value 357, for each of the segment reference pixel colors 361-364. In other embodiments, the segment reference pixel values may also be scaled or alternatively, the reference pixel values may be full-scale values corresponding to the raw data format. In alternate embodiments, additional reference values may be used for color depth or other 15 similar graphics data or pixel parameters. Calculation of the pixel reference value sets 350 will be discussed in greater detail below. [00044] Figure 4 illustrates an exemplary decoding process 400 for a compressed video file 114, according to one embodiment of the present invention. Compressed video file 114 may include a frame group header, segment reference pixel values 350, and encoded video tables 320 20 for each video frame 210. Decoder 118 processes compressed video file 114 to provide a decoded video file 120. Decoded video file 120 includes a decoded video table 420 including decoded pixel parameter values 422-427 for each pixel 431-436. Decoding process 400 includes the mapping of a compressed video table 320 to a decoded video table 420 using segment reference pixel values 350. The pixel data 331-336 is decoded using table 350 and is respectively mapped to pixel data 431 25 436. The process performed by decoder 118 to populate decoded video table 420 will be described in detail below. [00045] The decoded video file 120 can be formatted for playback devices supporting different input protocols. Such protocols include NTSC, SECAM, PAL and HDTV as described above. Additionally, support for computer displays is provided. If a low-bandwidth communication link 30 exists between display 122 and decoder 118, decoder 118 may be configured, in one embodiment, to transmit a fraction of the lines per frame. In another embodiment, in order to minimize bandwidth consumption, the encoder 112 may encode only a fraction of the lines per frame, such as one of two fields of video, resulting in a smaller compressed video file 114 for transmission over network 116. In other embodiments, the video frames may be encoded in their entirety but a field WO 2005/073920 PCT/AU2005/000108 - 11 is removed and/or the screen resolution is reduced prior to transmission over network 116. In yet another embodiment, frames may be dropped prior to transmission. For example, a file encoded at 24 frames per second may be reduced to 12 frames per second by dropping ever other frame prior to transmission. These embodiments may be particularly useful when the playback device 122 is a 5 cellular telephone or other wireless device, requiring high quality video over low bandwidth networks, such as GSM, CDMA, and TDMA. [00046] Figure 5 illustrates a flowchart of encoding process 500 for encoding live or streaming video content, according to one embodiment of the present invention. As discussed with reference to Figure 1, encoder 112 receives raw video data 106 for encoding. The encoder 112 then provides 10 a compressed video file 114, including frame group header, segment reference pixel values 350, frame group table 320, and any additional parameters and optional information described above, to a decoder 118 via network 116. [00047] In one embodiment of the present invention, the encoder 112 receives the digitized video data as raw video data 106. At block 502, the encoder 112 determines from the raw video 15 data 106 the video format and frame information, and creates a frame group file in which a frame group header, a table of scaled pixel parameter values, and reference pixels will be stored. In another embodiment, the audio is also stored in the frame group file. The raw video data 106 may be of any format known in the art such as MPEG (Moving Picture Experts Group), MJPEG (moving JPEG (Joint Photographic Experts Group)), QuickTime's AVI (audio video interleaved), 20 among others. [00048] For example, with reference to Figure 2, the encoder 112 receives raw pixel data for frames 210 as further illustrated in the raw pixel data tables 220. At block 504, the encoder 112 determines, pixel by pixel and per segment, the dominant color of each pixel by examining each pixels color component values. For example, pixel one data 231 includes a red color component 25 value 2231 of 10,000, a green component value 224, of 2,000, and a blue component value 2251 of 500. Therefore, in one embodiment pixel l's dominant color value would correspond to the highest numerical value among the three color values (RGB) which is the red value of 10,000. In other embodiments other techniques for calculating the dominant color may be used, such as weighted color component value comparisons. At block 506, the current pixel's color component 30 values are compared to the highest previously stored values for that color component in order to determine the segment reference pixels corresponding to the most intense pixels for each color for each segment. In the case of a black pixel or segment reference pixel, according to one embodiment of the present invention, the color component values would all have to be above a threshold value. For example, black segment reference pixel 364 has red, green and blue values of WO 2005/073920 PCT/AU2005/000108 -12 9000, 8000,and 8500, respectively. Although the values may not be the highest value for each color (e.g., red segment reference pixel 361 red value of 10000), the black segment reference pixel corresponding to the most intense black pixel of the segment, is the pixel with the highest of all three color component values, red, green and blue. If any one of the values is below a threshold 5 value, the higher of the remaining two values determines the color pixel. An exemplary threshold value may be eighty percent of the maximum color component value for each color (e.g., 80% of 10000 = 8000). In another embodiment, a white segment reference pixel and white dominant pixel table values are based upon the color component values being below a threshold value. Continuing at block 508, if the current color component value(s) is (are) not greater than the stored segment 10 reference pixel value(s), the stored values remain unchanged. However, if the current color component value(s) is (are) greater than the stored value(s) then the encoder 112 overwrites the current segment reference pixel values corresponding to that color component with the new values. [00049] A segment may be defined as any portion of a frame or frames. For example, with reference to Figure 2, a segment may be defined as the first line of a frame as shown in 2101 pixels 15 1 to 5. Among various embodiments, a segment may be multiple frames, two lines of a frame or half of a frame. It should be appreciated that the segment size definition may be optimized to accommodate a variety of systems to minimize encoder processor overhead and maximize the frame transfer rate for streaming video content over a low-bandwidth connections. [00050] An example illustrating the concept of storing the segment reference pixel values is 20 shown with reference to Figures 3A and 3B, respectively. As shown in table 220, of Figure 3A, pixel parameters 231 and 232 each indicate that the dominant color for each pixel is red based upon a comparison of their respective RGB values. However, the red value for pixel one of 10,000 is greater than that of pixel two 9,000 and therefore would remain as the red segment reference pixel as shown in table 350 of Figure 3B. The segment reference pixel also retains its other pixel 25 parameters such as green color component value 3541, blue color component value 3551, luminance value 3561, and chrominance value 3571. In other embodiments, all or some of these values may be scaled or otherwise manipulated to decrease table size or alter display characteristics. [00051] After the dominant color of each pixel is determined and the color component values are compared to the stored segment reference pixel values, the pixel parameters, at block 512, are 30 scaled down and stored in the table. In one embodiment, as illustrated with reference to Figure 3A table 3201, the scaled pixel values include scaled predominant color value 3231, scaled luminance value 3261, scaled chrominance value 3271, and a calculated color depth value 3281. In one embodiment of the present invention, only the dominant color value, luminance value, and chrominance value are scaled down and stored in the table. In another embodiment, all of the raw WO 2005/073920 PCT/AU2005/000108 -13 pixel parameter values are scaled down and stored within the table, including the non-dominant color values. [00052] In one embodiment of the present invention, as shown in Figure 3A, the pixel parameters 231-235 are scaled down into a one through ten (1-10) scale as shown with scaled pixel 5 parameters 331-335 of table 3201. For example, pixel parameter row 233 of table 220, indicates the dominant pixel color is green with a green color component value of 8,000 and a luminance and chrominance value of 490 and 510, respectively. If full-scale raw color values were 10,000, then the dominant color value may be rounded to the nearest thousand and divided by the full scale to produce a 1-10 value. For example: 10 Green dominant raw (Gd) value of 8000 (note, a value 8200 would round to 8000); ScaledValue(Gd) _ 8000 10 Scaled Gd= G8 10 10000 As shown in scaled pixel parameter row 333 of table 3201, wherein a dominant green color value of 15 8,000 becomes G8. Similarly, if the luminance and chrominance have full-scale values of 1,000, those values for pixel parameter row 233 would each become 5, respectively. For example: Luminance (Lm) value of 490 rounds up to 500; Scaled Lm 500 10_ - 1,00 ; Scaled Lm= 5 (Similar calculation for chrominance) 10 1Q00 20 In one embodiment, the color depth is calculated based upon the average of the scaled down luminance and chrominance values and as illustrated in table 3201. In another embodiment, the calculation is performed at the decoder. In yet other embodiments, the raw values may be scaled into any number of ranges, such as a 1-25 scale. 25 [00053] Once the pixel parameters are scaled down and prior to storing the parameters in the table, the encoder 112, checks the current pixel parameter values with previously stored values in the table. If the scaled down parameter values are unique, at block 516, encoder 112 writes the parameter values for the pixel into the table. However, if an identical or sufficiently similar (e.g., within a tolerance amount) table entry already exists, the encoder 112, at block 518, creates an 30 entry for the current pixel that refers to the previously encoded pixel in the table. For example, with reference to Figure 3A, pixel parameter row 234, if scaled according to the process described above, would have identical scaled dominant pixel color, luminance, and chrominance to that of WO 2005/073920 PCT/AU2005/000108 -14 pixel parameter row 233. Therefore, encoder 112 inserts a reference to the previously encoded pixel as shown with reference to table 320, row 334. It should be appreciated that in dealing with tens of thousands of pixels, the combination of scaling down the dominant color, luminance, and chrominance values in addition to inserting pointers for redundant pixel values will result in a 5 significant reduction in the size of the encoded pixel table over that of the raw pixel table, and thus the amount of bandwidth required to transmit this information to a decoder is reduced. [00054] The encoder 112, at block 520, checks whether or not the encoding has reached the end of the segment. If the end of the segment has not been reached, then encoder 112 indexes to the next pixel corresponding to the next pixel parameter row and repeats blocks 506 through 518. 10 [00055] At the end of each segment, at block 522, the encoder 112 retrieves the segment reference pixel values corresponding to the most intense dominant pixel colors for the segment and writes those segment reference pixels to the frame group file. In one embodiment of the present invention, the coordinates assigned to a segment reference pixel are the coordinates of the pixel prior to a pixel color change within the segment, or if there is not a color change leading up to the 15 end of the segment, the segment reference pixel coordinates for that color are the coordinates of the last pixel of the segment. In other embodiments, the segment reference pixels may be stored, by coordinate references or otherwise, according to any programming method that would allow for the values to be scaled according to the encoding method described above. [00056] If a segment only has fewer than four reference pixel colors represented therein, then 20 there may be fewer than few segment reference pixels associated with that segment. For example, if a segment includes a row of five pixels, as illustrated with reference to Figures 2 and 3, table 320 illustrates that the segment only includes dominant color values of red and green and therefore will only have a red and green segment reference pixels as further illustrated in Figure 3B, segment reference pixel data 361 and 362. Therefore, in this example, the encoder 112 would only 25 write segment reference pixel data corresponding to the most intense red and green pixel colors of the segment to the frame group file. [00057] Once the encoder 112 writes the segment reference pixel data to the frame group file, the encoder 112, at block 524, determines if it has reached the end of the frame in the encoding process. If the process has not reached the end of the frame, the encoder 112 indexes to the next 30 segment and repeats blocks 504 through 520. If another frame has been reached, at block 526, the encoder 112 determines whether it has encoded the entire frame group. If the entire frame group has not been encoded, the encoder 112 indexes to the next frame and repeats blocks 504 through 524. However, if the end of the frame group has been reached, the encoder 112, at block 528, inserts a pointer used by the decoder 118 to identify the next frame group for decoding. Thereafter, WO 2005/073920 PCT/AU2005/000108 -15 the encoder 112 communicates the frame group in the compressed video file 114 through the network 116 to the decoder 118. At block 530, the encoder 112 begins encoding the next frame group and repeats blocks 404 through 528. In one embodiment, the frame group file includes multiple tables comprised of multiple frames. For example, a table may include pixel information 5 for 25 frames and a frame group may include five tables thus equaling 125 frames per frame group. [00058] Figure 6 illustrates a flowchart of a decoding process 600, according to one embodiment of the present invention. As discussed with reference to Figure 1, decoder 118 receives the compressed video file 114 through network 116. After decoding, the decoder 118 supplies decoded video data 120 to playback device 122. 10 [00059] The decoding process 600 begins at block 602 by receiving and caching (temporarily storing) the compressed video file 114 from network 116. At block 604, the decoder 118 begins decoding the scaled table of pixel parameter values beginning at the first pixel of the first frame. The decoder 118 reads from the table, the pixel location, reference pixel color value, luminance and chrominance. At block 608, the decoder 118 scales the corresponding segment reference pixel 15 values according to the table of pixel parameter values. [00060] For example, with reference to Figure 4, the decoder 118 uses the encoded pixel parameter values of table 3201 and the segment reference pixels of table 350, to generate decoded pixel parameter values as illustrated in table 4201. For example, using the scaled dominant color value G8 of pixel three, the scaled luminance and chrominance of 5, and the green segment 20 reference pixel 362 results in decoded pixel three values of table 4201. For example: Pixel parameter values of G8 (G, use green segment reference pixel), luminance (Lm) 5, and chrominance (Cm) 5, from table 3501: 25 Segment Reference Pixel G-R600, G10000, B740, Lm600, Cm400. Non-dominant R and B remain the same, scale dominant G Lm and Cm: 8 Scaled G (Gs) 8 Scaled (Gs) ; Gs = 8000; Rs = 480; Bs = 592 10 10000 30 5 Scaled Lm 5 Scaled C Cm = 200 3010 600 Lm= 300; Cm= 200400 10 600 10 400 WO 2005/073920 PCT/AU2005/000108 -16 Therefore, the decoded table entry would appear as illustrated in 433 of table 420 and is duplicated below: Decoded Pixel 3-R600, G8000, B740, Lm300, Cm200; In another embodiment, R and B are scaled by the same factor of 0.8, similar to the 5 calculation for Gs, above. In another embodiment, however, the scaled values of the table may include the non-dominant colors that may also be decoded with reference to the segment reference pixels. In other embodiments, the segment reference pixel values are the original raw full-scale values, and are either communicated with the table of scaled values or are resident within the decoder system. 10 [00061] In the case where an entry in the table of scaled pixel values is a reference pointer to a previous pixel entry, the decoder 118 duplicates the decoded results of previous pixel entry. [00062] The decoder 118 indexes to the next pixel to decode in the segment if it is determined, at block 610, that the segment has not been fully decoded. If the end of the segment has been reached, the decoder 118 determines at block 612 if the end of the frame has been reached. If not, 15 then the decoder 118 begins decoding the next segment of the frame using the process described above. If the entire frame has been decoded, the decoder determines, at block 614, if the entire frame group has been decoded. If not, the decoder 118 begins decoding the next frame in the table. If the entire frame group has been decoded, the encoder 118, at block 616, receives and decompresses (if necessary) any audio data associated with the previously decoded frame group. 20 At block 618, the decoder determines if the frame data requires reformatting for display. In one embodiment of the present invention, the user of the display device configures the decoder to format the decompressed video data 120 to accommodate various playback devices, such as Microsoft's Windows Media Player. If reformatting is required, reformatting is executed at block 620 and decoder, at block 620 synchronizes the audio and writes the decoded frame group to the 25 playback device 122. [00063] After decoding and displaying the decoded frame group, according to one embodiment of the present invention, the decoder 118 at block 624, reads from the frame group the pointer to the next frame group for decoding, and clears the previously decoded frame group from the cache. In one embodiment, the decoder may read a trailer appended to the communicated file. The trailer 30 may provide the decoder with audio/video information, such as the logical location or name of the next frame group to decode, the number of frames and or files remaining in the encoded video, index information to the next file, or other audio/video information related to playback.
WO 2005/073920 PCT/AU2005/000108 -17 [00064] Having discussed numerous illustrations of encoding and decoding functions according to the present method and system, a brief description of the communication network and computer architecture encompassing the present system is provided. 5 An Exemplary Network Architecture [00065] Elements of the present invention may be included within a client-server based system 500 such as that illustrated in Figure 7. One or more servers 710 communicate with a plurality of clients 730-735. The clients 730-735 may transmit and receive data from servers 710 over a variety of communication media including (but not limited to) a local area network 740 and/or a 10 larger network 725 (e.g., the Internet). Alternative communication channels such as wireless communication via GSM, TDMA, CDMA, Bluetooth, IEEE 802.11, or satellite broadcast (not shown) are also contemplated within the scope of the present invention. [00066] Servers 710 may include a database for storing various types of data. This may include, for example, specific client data (e.g., client account information and client preferences) 15 and/or more general data. The database on servers 710 in one embodiment runs an instance of a Relational Database Management System (RDBMS), such as Microsoft T M SQL-Server, Oracle T M or the like. A user/client may interact with and receive feedback from servers 710 using various different communication devices and/or protocols. According to one embodiment, a user connects to servers 710 via client software. The client software may include a browser application such as 20 Netscape Navigator m or Microsoft Internet Explorer m on the user's personal computer, which communicates to servers 710 via the Hypertext Transfer Protocol (hereinafter "HTTP"). Among other embodiments, software such as Microsoft's Word, Power Point, or other applications for composing and presentations may be configured as client decoder/player. In other embodiments included within the scope of the invention, clients may communicate with servers 710 via cellular 25 phones and pagers (e.g., in which the necessary transaction software is electronic in a microchip), handheld computing devices, and/or touch-tone telephones. [000671 Servers 710 may also communicate over a larger network (e.g., network 725) to other servers 750-752. This may include, for example, servers maintained by businesses to host their Web sites - e.g., content servers such as "yahoo.com." Network 725 may include router 720. 30 Router 720 forwards data packets from one local area network (LAN) or wide area network (WAN) to another. Based on routing tables and routing protocols, router 720 reads the network address in each IP packet and makes a decision on how to send if based on the most expedient route. Router 720 works at layer 3 in the protocol stack.
WO 2005/073920 PCT/AU2005/000108 - 18 [00068] According to one embodiment of the present method and system, components illustrated in Figure 1 may be distributed throughout network 700. For example, video sources may be connected to any client 730-735 or 760-762, or sever 710 or 750-752. Live feed source 104, encoder 112, decoder 118 and display 122, may reside in any client or server, as well. 5 Similarly, all or some of the components of Figure 1, may be fully contained within a signal server, or client. [00069] In one embodiment, servers 750-752 host video acquisition device 104 and encoder 112. Video sources connected to clients 760-762 provide source video to servers 750-752. Servers 750-752 encode and compress the live source video and deliver the compressed video file 114 upon 10 a client request. Upon client 730-732 request, servers 750-752 transmit the compressed video file 114 over network 116 to the client 730-733 via server 710. In addition, server 710 and the client 730-733 may be connected via a dial-up connection between 9.6 kBps and 56 kBps. Clients 730 733 hosts decoder 118, and upon receiving the compressed video file 114, decodes the file 114 and provides the decoded video file 120 to an attached playback device. Numerous combinations may 15 exist for placement of encoder 112, decoder 118 and video acquisition device 104. Similarly, encoder 112, decoder 118 and live feed source 104 may exist in software executed by a general purpose processor, a dedicated video processor provided on an add-on card to a personal computer, a PCMCIA card, an ASIC (application specific integrated circuit) or similar devices. Additionally, decoder 118 may reside as a software program running independently, or as a plug-in to a web 20 browser. Decoder 118 may be configured to format its video output to have compatibility with existing playback devices that support motion JPEG, MPEG, MPEG-2, MPEG-4 and JVT standards. AN EXEMPLARY COMPUTER ARCHITECTURE 25 [00070] Having briefly described an exemplary network architecture which employs various elements of the present invention, a computer system 600 representing exemplary clients 730-735 and/or servers (e.g., servers 710), in which elements of the present invention may be implemented will now be described with reference to Figure 8. [00071] One embodiment of computer system 800 comprises a system bus 820 for 30 communicating information, and a processor 810 coupled to bus 820 for processing information. Computer system 800 further comprises a random access memory (RAM) or other dynamic storage device 825 (referred to herein as main memory), coupled to bus 820 for storing information and instructions to be executed by processor 810. Main memory 825 also may be used for storing temporary variables or other intermediate information during execution of instructions by processor WO 2005/073920 PCT/AU2005/000108 - 19 810. Computer system 800 also may include a read only memory (ROM) and/or other static storage device 826 coupled to bus 820 for storing static information and instructions used by processor 810. [00072] A data storage device 827 such as a magnetic disk or optical disc and its corresponding 5 drive may also be coupled to computer system 800 for storing information and instructions. Computer system 800 can also be coupled to a second I/O bus 850 via an I/O interface 830. Multiple I/O devices may be coupled to I/O bus 850, including a display device 843, an input device (e.g., an alphanumeric input device 842 and/or a cursor control device 841). For example, video news clips and related information may be presented to the user on the display device 843. 10 [00073] The communication device 840 is for accessing other computers (servers or clients) via a network 725, 740. The communication device 840 may comprise a modem, a network interface card, or other well-known interface device, such as those used for coupling to Ethernet, token ring, or other types of networks. 15 A GENERALIZED MAPPING [00074] The above-described system and method for encoding/decoding video information can now be generalized to a method and system for encoding data values having one or more multi dimensional parameters by mapping those parameters to respective one-dimensional parameters. These one-dimensional parameters may then be stored in a table of encoded data values in which 20 their respective encoded counterparts represent the original data values. Redundant ones of the encoded data values, meaning those which are identical or sufficiently similar to one another, may share common table entries, for example by making use of pointers or other entries in the table. Decoding is accomplished by essentially reversing the encoding process with reference to certain reference data values. 25 [00075] The present methods and systems may be applied in a variety of contexts. For example, as discussed above the present invention finds particular application in the field of video and/or audio encoding/decoding. More broadly, the present invention may be used in any application in which data values are represented by one or more multi-dimensional parameters, such as color, position, orientation, or other parameters. 30 [00076] Turning now to Figure 9, one embodiment of methods of the present invention is illustrated in a graphical fashion by the transformation of a table 910 of raw data values into a table 920 of encoded data values. The raw data values that make up table 910 may represent any digital information. Each data value has an associated row 912 of parameters (P) that describe the data value. For example, if the data values represent pixels of a frame of video information, the WO 2005/073920 PCT/AU2005/000108 - 20 parameters may represent the color, luminance and chrominance information associated with the pixels. In general there are "x" data values and "n" parameters per data value in table 912, where x and n are integers. [00077] Some of the parameters may be one-dimensional parameters, meaning that the entire 5 parameter is described by a single value. Referring to the earlier example, in the case where the data values represent pixels of a frame of video information, the luminance and chrominance parameters may be one-dimensional parameters. [00078] Other parameters, however, may be two-, three- or, generally, multi-dimensional in nature, meaning that more than one value is needed to describe the parameter. In the case of the 10 data values representing pixels, the color parameter is, in general, multi-dimensional because typically more than one value is required to describe the parameter. For example, values for red, green and blue are often required in order to fully describe the color parameter of a pixel data value, making the color parameter three-dimensional in nature. Each dimension of a data value parameter may be represented as Pvm, where v represents the parameter number, m represents the 15 sub-parameter number and both are integers. [00079] In accordance with the present invention, the multi-dimensional parameters for each data value in table 910 are mapped to respective one-dimensional parameters in the table 920 of encoded data values. This is illustrated in detail in Figure 10. A parameter Py having a number of dimensions (or sub-parameters) Pyi, Py, . . . Pyn is mapped to a one-dimensional parameter Pye to be 20 used in table 920. This has the effect of reducing the total number of bits required to represent the data value, provided the mapping is something other than merely aggregating the values associated with the multi-dimensional parameter into a single value. By way of example, in the case of the pixel data values the original multi-dimensional color parameter (having red, green and blue dimensions say) may be mapped to a single color value (color) having a fewer number of bits than 25 the aggregate of the red, green and blue values. [000801 Of course any such mapping may be done in a variety of ways. In one embodiment of the present invention the mapping is done so as to describe a raw, multi-dimensional parameter solely in terms of its dominant constituent when encoded. For example, consider a raw pixel data value that has a color parameter with its red dimension equal to "rrr", its blue dimension equal to 30 "bbb" and its green dimension equal to "ggg' (where rrr, bbb and ggg are all integers). If the red dimension was the dominant value (e.g., in terms of its magnitude), then the resulting encoded version of this parameter might simply have a one-dimensional color parameter that indicated "red", with perhaps some appropriate magnitude value that may or may not be the same as the raw pixel data value's red dimension. Indeed, if this technique is used in combination with a scaling of WO 2005/073920 PCT/AU2005/000108 -21 the magnitude of the dominant parameter constituent (or even a value which takes into account all or some of the sub-parameters values) to arrive at the encoded pixel parameter (Pye) value, significant savings in terms of the number of bits required to represent the parameter (and, hence the data value as a whole) may be achieved. 5 [00081] Other ways in which the mapping may be performed include using a weighted dimension approach. That is, the values of the raw data value parameter dimensions may be separately weighted according to their individual impact on the overall parameter being described and then the weighted dimension value having the greatest magnitude selected as the dominant constituent. Alternatively, an averaging of the parameter dimension values (weighted or not) may 10 be performed to arrive at the encoded, one-dimensional parameter value. [00082] It should be noted that what is being proposed herein in not simply mapping an existing parameter to another parameter space in order to change the representation of that parameter. For example, this is not a proposal to simply map a color parameter initially described by RGB color components to YUV or another color space. Instead, the present invention involves making a 15 deliberate decision to eliminate certain parameter information (e.g., color information) in order to develop a description of a data value that is smaller (e.g., in terms of the number of bits) than the original. [00083] As the table 920 of encoded data values is being constructed, a further technique may be employed in order to further reduce the number of bits required to represent the original table of 20 raw data values. In cases where the various parameter values of a newly encoded data value are identical or sufficiently similar (e.g., within a tolerance limit) to the values of a previously encoded data value's parameters, then this entire list of parameters need not be repeated within table 920. Instead, one need only store a pointer to the previously encoded data value as an indication that the same encoded parameter values should be used. Assuming the number of parameters is such that it 25 requires fewer bits to represent the pointer than it does to represent the parameters themselves, this will further reduce the number of bits required to represent the data values in their encoded form. In some cases, it may be beneficial to use this approach for individual parameters (rather than for entire data values) within table 20, for example where a pointer can be represented in fewer bits than a parameter. Note that this portion of the encoding process is a lossless process, whereas other 30 portions of the encoding process may be lossy in nature. [00084] When employed as part of a data encoding / decoding scheme, the present methods allow for the transmission of the table 920 of encoded data values rather than the table 910 of raw data values between a transmitter and a receiver. Because the number of bits required to represent the encoded data value parameters is less than the number of bits required to represent the raw data WO 2005/073920 PCT/AU2005/000108 - 22 value parameters, the present methods will achieve bandwidth savings during such transmissions. Viewed another way, the present methods allow for the use of lower bandwidth communication links to support such transmissions. This is especially useful when low-bandwidth communication links such as dial-up communication links (typically 14.4 - 56.6 kBps) and/or mobile phone (e.g., 5 GSM, or CDMA) communication links (typically 9.6 kBps) are being relied upon to transport otherwise large data files, such as audio/video files. [00085] In order to properly decode the table 920 of encoded data values at a receiver either of two methods may be employed. In a first method, a set of reference data values may be transmitted to the receiver along with the table 920 of encoded data values. These reference data values may 10 include a small number of raw data values having a complete set of the multi-dimensional parameters. Using this information, the receiver may construct a table of decoded data values having multi-dimensional parameters for further use. [00086] For example, in the case of data values representing pixels of video frames, a small number of reference pixel data values may be transmitted to the receiver. In one embodiment, 15 reference pixels for red, green, blue and black (representing a maximum color intensity in each dimension) may be so provided. Each reference pixel includes a multi-dimensional color parameter; hence the "red" pixel will have RGB values, as will the green pixel, the blue pixel and the black pixel. [00087] The receiver may construct a table of decoded pixel values by using the appropriate 20 reference pixel and scaling the RGB values therefor as appropriate. For example, if the encoded data value recites a red pixel (meaning that during encoding the corresponding raw pixel was determined to have a dominant red dimension color parameter) then the receiver would use the RGB values for the red reference pixel (perhaps scaled according to the magnitude of the color parameter for the encoded data value) in creating the corresponding entry for the pixel of interest in 25 the table of decoded data values. [00088] The second method for decoding the table 920 of encoded data values involves providing the receiver with prior knowledge of the encoding process used. That is, if the receiver is provided with information similar to that which could be obtained using the reference data values prior to receiving the transmitted table of encoded data values, then the transmission need not 30 include the reference data values in order to decode the information included in the table. Such knowledge may be provided in a variety of forms, including a lookup table at the receiver. [00089] This use of a lookup table also has application to the encoding process. For example, if the encoder is provided with a sufficiently large lookup table that encompasses all or a large number of the possible combinations of data value parameter sets likely to be encountered in WO 2005/073920 PCT/AU2005/000108 - 23 encoding operations, then encoding becomes a simple matter of translating the data value to a pointer that references the appropriate lookup table entry. If the receiver is provided with a similar lookup table then decoding is performed by mapping the pointer from the table of encoded data values to the appropriate lookup table entry and using the indicated set of data value parameters. 5 This approach can be combined with the above described method for reducing the number of encoded data value table entries using pointers to/from redundant ones thereof to even further reduce the amount of data included in the table. [00090] It should be apparent that the present methods (with the exclusion of the encoder and decoder lookup table-based approach discussed immediately above) for encoding/decoding data 10 values is a "lossy" process in that some of the original information for multi-dimensional parameters is destroyed as part of the encoding process. However portions of the process (e.g., the use of pointers within tables) are "lossless". Thus, the present technique may not be suitable for all applications. Nevertheless, where perfect reproduction of an original data set is not required, the present methods may be employed to dramatically reduce the volume of data to be transmitted 15 between an encoder and a decoder or to be stored in a data archive. [00091] Thus, a method and system for encoding data values having one or more multi dimensional parameters has been described. Although the present invention was discussed with reference to presently preferred embodiments, the full scope of the invention should be measured only in terms of the claims that follow.

Claims (86)

1. A method, comprising: encoding a frame of audio/video data, segment by segment, comprising a number of 5 pixels each having a plurality of pixel color components by: creating a frame group table of encoded pixel values in which each pixel entry includes a dominant pixel color component of the plurality of pixel color components; and determining a set of segment reference pixels for each encoded segment, wherein each one of the segment reference pixels is a pixel within each one of the encoded segments having a 10 most intense dominant pixel color value.
2. The method of claim 1, wherein each pixel entry further includes at least one pixel parameter value.
3. The method of claim 2, wherein the pixel parameter values include at least one of luminance, chrominance, and color depth information. 15
4. The method of claim 2, including scaling down each pixel parameter value prior to storing each pixel entry into the frame group table.
5. The method of claim 1, wherein prior to encoding the frame of audio/video data, creating a frame group file to store a header, the frame group table, and each set of segment reference pixels. 20
6. The method of claim 5, wherein after creating the frame group table, writing a pointer to a next frame group to the frame group file.
7. The method of claim 5, including storing audio data within the frame group file.
8. The method of claim 1, including encoding audio data into the frame group table.
9. The method of claim 1, including encoding audio synchronization tags into the frame 25 group table.
10. The method of claim 1, wherein determining the set of segment reference pixels includes comparing, on a pixel by pixel basis for each segment, a current pixel color value with a previously WO 2005/073920 PCT/AU2005/000108 -25 stored dominant pixel color value and storing the plurality of pixel color components and pixel parameters of the pixel with the most intense dominant pixel color component.
11. The method of claim 1, wherein each pixel entry further comprises scaled down non dominant pixel color components. 5
12. The method of claim 1, wherein the plurality of pixel color components comprises at least one of the sets of primary color components, red, green, and blue, or cyan, magenta, and yellow.
13. The method of claim 12, wherein the dominant pixel color components include red, green, blue, and black.
14. The method of claim 13, wherein the dominant pixel color component black is 10 determined by comparing an average of the red, green, and blue pixel component values to a black threshold value.
15. The method of claim 1, wherein an encoded segment is a row of pixels within the frame of audio/video data.
16. The method of claim 1, wherein creating a frame group table further comprises scaling 15 down the dominant pixel color component.
17. The method of claim 1, wherein the frame group table includes pixel entries for a plurality of frames.
18. The method of claim 1, wherein redundant encoded pixel values of the frame group table share common table entries. 20
19. The method of claim 18, wherein redundant encoded pixel values share identical dominant pixel color components and identical pixel parameter values.
20. The method of claim 18, wherein redundant encoded pixel values share dominant pixel color components and pixel parameters values that are similar to one another within a tolerance range. 25
21. An encoder, comprising: the encoder to create, pixel by pixel, a frame table of encoded segments of audio/video data, wherein each pixel is represented by a single pixel color component; and WO 2005/073920 PCT/AU2005/000108 - 26 the encoder to create a set of color scaling pixels for each encoded segment, wherein each color scaling pixel is selected based upon a pixel having a most significant color intensity value with respect to similarly colored pixels within the segment.
22. The encoder of claim 21, wherein the frame table data of encoded segments of 5 audio/video data for each pixel includes pixel parameter values for at least one of, luminance, chrominance, and color depth.
23. The encoder of claim 22, wherein the pixel parameter values are scaled down values.
24. A method, comprising: decoding, on a pixel-by-pixel basis, audio/video data using a table of encoded pixel 10 parameter values, wherein each pixel is represented by an entry including a dominant pixel color component; and scaling a set of segment reference pixels comprised of segment reference pixel values according to each entry in the table of encoded pixel parameter values to produce decoded pixels comprised of decoded pixel parameter values. 15
25. The method of claim 24, wherein each set of the segment reference pixels corresponds to an encoded segment of a frame.
26. The method of claim 25, wherein the set of the segment reference pixel values comprises a unique set of color pixels for each encoded segment, wherein each segment reference pixel represents a pixel with a most intense dominant pixel color component for each encoded segment. 20
27. The method of claim 26, wherein the set of segment reference pixels comprises a representative red pixel, green pixel, blue pixel, and black pixel.
28. The method of claim 24, wherein the table of encoded pixel parameter values further comprises at least one of luminance, chrominance, and color depth.
29. The method of claim 24, wherein the set of the segment reference pixel values further 25 comprises a dominant color pixel value, non-dominant pixel color values, and luminance and chrominance values. WO 2005/073920 PCT/AU2005/000108 - 27
30. The method of claim 29, wherein scaling the set of segment reference pixel values comprises scaling the segment reference pixel's dominant color pixel value, non-dominant pixel color values, and luminance and chrominance values.
31. The method of claim 24, wherein the table of encoded pixel values further comprises 5 redundant entries, wherein each one of the redundant entries is decoded by recalling previously decoded pixel parameter values associated with each one of the redundant entries.
32. The method of claim 24, wherein the table of encoded pixel parameter values further comprises non-dominant pixel color components.
33. The method of claim 32, wherein the set of segment reference pixels are comprised of 10 full-scale pixel parameter values.
34. The method of claim 33, wherein scaling the set of segment reference pixel values further comprises scaling each of the full-scale pixel parameter values with the each corresponding encoded pixel parameter values.
35. The method of claim 24, further comprising synchronizing audio data associated with the 15 decoded table of encoded pixel parameter values.
36. The method of claim 24, wherein prior to decoding the audio/video data, receiving a file including the table of encoded pixel parameter values and the set of segment reference pixel values.
37. The method of claim 36, wherein the file further comprises a header comprised of video frame information and audio information. 20
38. The method of claim 37, further comprising processing the file by using the header to determine data locations within the file, including the beginning and end of the table of encoded pixel parameter values and the corresponding segment reference pixel values.
39. The method of claim 24, wherein after scaling the set of segment reference pixel values according to each entry in the table of encoded pixel parameter values, communicating the decoded 25 pixels to a playback device.
40. The method of claim 39, further comprising communicating and synchronizing audio data to the playback device. WO 2005/073920 PCT/AU2005/000108 -28
41. The method of claim 39, wherein prior to communicating decoded pixel parameter values to the playback device, converting the decoded pixel parameter values to another display format.
42. A method of communicating live audio/visual information over a communications link, comprising: 5 encoding, segment by segment, frames of audio/video data, including a number of pixels each having a plurality of pixel color components by creating a frame group table of encoded pixel values in which each pixel entry includes a dominant pixel color component of the plurality of pixel color components; determining a set of segment reference pixels for each encoded segment, wherein each 10 one of the segment reference pixels is comprised of segment reference pixel parameter values and is a pixel within each one of the encoded segments having a most intense dominant pixel color value; communicating the frame group table and the segment reference pixels over a network to a receiver; and 15 at the receiver, decoding the frame group table on a pixel-by-pixel basis by scaling the segment reference pixel parameter values according to each entry in the frame group table of encoded pixel parameter values to produce decoded pixels comprised of decoded pixel parameter values.
43. The method of claim 42, wherein each pixel entry further comprises at least one of 20 luminance, chrominance, and color depth information.
44. The method of claim 43, further comprising scaling down each pixel parameter value prior to storing each pixel entry into the frame group table.
45. The method of claim 42, wherein the set of segment reference pixels comprises a representative red pixel, green pixel, blue pixel, and black pixel. 25
46. The method of claim 42, wherein prior to encoding the frame of audio/video data, creating a frame group file to store a header, the frame group table, and each set of segment reference pixels.
47. The method of claim 46, wherein after creating the frame group table, writing a pointer to a next frame group to the frame group file. WO 2005/073920 PCT/AU2005/000108 - 29
48. The method of claim 46, including storing audio data within the frame group file.
49. The method of claim 42, wherein determining the set of segment reference pixels includes comparing, on a pixel by pixel basis for each segment, a current pixel color value with a previously stored dominant pixel color value and storing the plurality of pixel color components 5 and pixel parameters of the pixel with the most intense dominant pixel color component.
50. The method of claim 42, wherein the plurality of pixel color components comprises at least one of the sets of primary color components, red, green, and blue, or cyan, magenta, and yellow.
51. The method of claim 50, wherein the dominant pixel color components include red, 10 green, blue, and black.
52. The method of claim 42, wherein an encoded segment is a row of pixels within the frame of audio/video data.
53. The method of claim 42, wherein creating a frame group table further comprises scaling down the dominant pixel color component. 15
54. The method of claim 42, wherein the set of the segment reference pixel values further comprises a dominant color pixel value, non-dominant pixel color values, and luminance and chrominance values.
55. The method of claim 42, wherein redundant encoded pixel values of the frame group table share common table entries. 20
56. The method of claim 55, wherein redundant encoded pixel values share identical dominant pixel color components and identical pixel parameter values.
57. The method of claim 55, wherein redundant encoded pixel values share dominant pixel color components and pixel parameters values that are similar to one another within a tolerance range. 25
58. The method of claim 55 wherein each one of the common table entries is decoded by recalling previously decoded pixel parameter values associated with each common table entry. WO 2005/073920 PCT/AU2005/000108 -30
59. The method of claim 42, wherein scaling the set of segment reference pixel values comprises scaling the segment reference pixel's dominant color pixel value, and luminance and chrominance values.
60. The method of claim 42, wherein the table of encoded pixel parameter values further 5 comprises non-dominant pixel color components.
61. The method of claim 60, wherein the set of segment reference pixels are comprised of full-scale pixel parameter values.
62. The method of claim 61, wherein scaling the set of segment reference pixel values further comprises scaling each of the full-scale pixel parameter values with the each corresponding 10 encoded pixel parameter values.
63. The method of claim 42, further comprising synchronizing audio data associated with the table of encoded pixel parameter values.
64. The method of claim 42, wherein after scaling the set of segment reference pixel values according to each entry in the table of encoded pixel parameter values, communicating the decoded 15 pixels to a playback device
65. The method of claim 64, further comprising communicating and synchronizing audio data to the playback device.
66. The method of claim 42, wherein the communications link is a low-bandwidth communications link. 20
67. A system, comprising: an encoder to encode, segment by segment, frames of audio/video data, including a number of pixels each having a plurality of pixel color components by creating a frame group table of encoded pixel values in which each pixel entry includes a dominant pixel color component of the plurality of pixel color components and to determine a set of segment reference pixels for each 25 encoded segment, wherein each one of the segment reference pixels is comprised of segment reference pixel parameter values and is a pixel within each one of the encoded segments having a most intense dominant pixel color value; WO 2005/073920 PCT/AU2005/000108 -31 a server to communicate the frame group table and the segment reference pixels over a network to a receiver; and a decoder coupled to the receiver to decode the frame group table on a pixel-by-pixel basis by scaling the segment reference pixel parameter values according to each entry in the frame 5 group table of encoded pixel parameter values to produce decoded pixels.
68. A method, comprising encoding data values by mapping multi-dimensional parameters of the data values to respective one-dimensional parameters and creating a table of encoded data values in which the data values are represented by their respective encoded counterparts utilizing the one-dimensional parameters and in which redundant ones of the encoded data values share 10 common table entries.
69. The method of claim 68, wherein the data values comprise pixel information.
70. The method of claim 68, wherein the data values comprise position information.
71. The method of claim 68, wherein redundant encoded data values share identical parameter values. 15
72. The method of claim 68, wherein redundant data values share parameters values which are similar to one another within a tolerance range.
73. The method of claim 68, further comprising transmitting the table of encoded data values to a receiver.
74. The method of claim 73, further comprising decoding the table of encoded data values at 20 the receiver using the table of encoded data values and a set of reference information.
75. The method of claim 74, wherein the reference information is transmitted together with the table of encoded data values.
76. The method of claim 74, wherein the reference information is stored at the receiver prior to the transmission of the table of encoded data values. 25
77. The method of claim 74, wherein the reference information comprises a lookup table.
78. A method, comprising encoding a data values having one or more multi-dimensional parameters by combining a lossy encoding process in which the one or more multi-dimensional WO 2005/073920 PCT/AU2005/000108 -32 parameters of the data values are mapped to respective one-dimensional parameters and stored in a table of encoded data values, with a lossless encoding process in which redundant ones of the encoded data values are arranged to share common entries in the table.
79. The method of claim 78, wherein the data values comprise pixel information. 5
80. The method of claim 78, wherein the data values comprise position information.
81. The method of claim 78, wherein the redundant ones of the encoded data values share identical parameter values.
82. The method of claim 78, wherein the redundant ones of the encoded data values share parameters values which are similar to one another within a tolerance range. 10
83. The method of claim 78, further comprising transmitting the table of encoded data values to a receiver.
84. The method of claim 83, further comprising decoding the table of encoded data values at the receiver using the table of encoded data values and a set of reference information.
85. The method of claim 84, wherein the reference information is transmitted together with 15 the table of encoded data values.
86. The method of claim 84, wherein the reference information is stored at the receiver prior to the transmission of the table of encoded data values.
AU2005207982A 2004-02-02 2005-01-28 Systems and methods for encoding and decoding video information Abandoned AU2005207982A1 (en)

Applications Claiming Priority (9)

Application Number Priority Date Filing Date Title
US10/770,952 2004-02-02
US10/771,096 US20050180641A1 (en) 2004-02-02 2004-02-02 System and method for transmitting live audio/video information
US10/770,432 US20050169365A1 (en) 2004-02-02 2004-02-02 Data encoding using multi-dimensional redundancies
US10/771,096 2004-02-02
US10/770,432 2004-02-02
US10/771,097 2004-02-02
US10/770,952 US20050196049A1 (en) 2004-02-02 2004-02-02 System and method for encoding live audio/video information
US10/771,097 US7505045B2 (en) 2004-02-02 2004-02-02 System and method for decoding live audio/video information
PCT/AU2005/000108 WO2005073920A1 (en) 2004-02-02 2005-01-28 Systems and methods for encoding and decoding video information

Publications (1)

Publication Number Publication Date
AU2005207982A1 true AU2005207982A1 (en) 2005-08-11

Family

ID=34831423

Family Applications (1)

Application Number Title Priority Date Filing Date
AU2005207982A Abandoned AU2005207982A1 (en) 2004-02-02 2005-01-28 Systems and methods for encoding and decoding video information

Country Status (2)

Country Link
AU (1) AU2005207982A1 (en)
WO (1) WO2005073920A1 (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111385579B (en) * 2020-04-09 2022-07-26 广州市百果园信息技术有限公司 Video compression method, device, equipment and storage medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5467110A (en) * 1989-05-12 1995-11-14 The Regents Of The University Of California, Office Of Technology Transfer Population attribute compression
EP0705027B1 (en) * 1994-09-30 2000-11-22 SANYO ELECTRIC Co., Ltd. Image communication system and method for data compression of colour images using colour tables and region-based processing
JPH08139936A (en) * 1994-11-04 1996-05-31 Canon Inc Coding method
EP1235185B1 (en) * 2001-02-21 2011-11-23 Boly Media Communications Inc. Method of compressing digital images

Also Published As

Publication number Publication date
WO2005073920A1 (en) 2005-08-11

Similar Documents

Publication Publication Date Title
US20050180641A1 (en) System and method for transmitting live audio/video information
US6639945B2 (en) Method and apparatus for implementing motion detection in video compression
US7181072B2 (en) Intra compression of pixel blocks using predicted mean
US7162091B2 (en) Intra compression of pixel blocks using predicted mean
US6507672B1 (en) Video encoder for digital video displays
WO2006024077A2 (en) System and method for encoding and decoding video
US6008847A (en) Temporal compression and decompression for video
US7991052B2 (en) Variable general purpose compression for video images (ZLN)
US20130114687A1 (en) Fixed intra run-level mode for cavlc in hevc
US8537898B2 (en) Compression with doppler enhancement
CN108366300A (en) Image received device
US6968012B1 (en) Methods for encoding digital video for decoding on low performance devices
US6584226B1 (en) Method and apparatus for implementing motion estimation in video compression
US6975767B1 (en) System and method for encoding and decoding video
US7010033B2 (en) System and method for compressing and encoding video
US20060008154A1 (en) Video compression and decompression to virtually quadruple image resolution
US7505045B2 (en) System and method for decoding live audio/video information
US20050196049A1 (en) System and method for encoding live audio/video information
CN101395924A (en) Differential coding with lossy embedded compression
AU2005207982A1 (en) Systems and methods for encoding and decoding video information
WO2005074143A1 (en) System and method for encoding/decoding live audio/video information
US20050207657A1 (en) System and method for encoding and decoding video
US7483576B2 (en) System and method for decoding video
Gao et al. Standard designs for cross random access point reference in video coding
WO2005074146A1 (en) Data encoding using multi-dimensional redundancies

Legal Events

Date Code Title Description
MK4 Application lapsed section 142(2)(d) - no continuation fee paid for the application