COMPRESSION SYSTEM AND METHOD FOR USE IN A SET TOP BOX ENVIRONMENT
Technical Field
This invention relates to image compression. More particularly, this invention relates to image compression of Internet images for use in a set top box environment..
BACKGROUND OF THE INVENTION
Set top boxes provide an interface between a television display or peripheral and a multimedia source that includes audio and video content. The set top boxes are connected by cable systems (coaxial, fiberoptic, etc.), satellite-based systems, and other terrestrial-based systems to one or more media providers. Because the multimedia sources have predorr nantly provided video that contains moving images, the multimedia providers generally rely on Motion Picture Experts Group (MPEG) standards for compression of the video to reduce bandwidth requirements. To further increase the speed of decompression, the set top boxes generally employ hardwired circuitry and/or hardware acceleration for performing MPEQ decompression. In the future, set top boxes will also be required to interface with distributed communications systems such as the Internet. Most of the web pages currently on the Internet employ still images. The MPEG formats used in the set top box environment are less desirable for still images. Web pages typically employ a graphics interchange format (GIF) and Joint Photographic Experts Group (JPEG) format for still images. GIF was developed in 1987 at the request of CompuServe who needed a platform-independent image format that was suitable for transfer across slow
connections. GIF is a compressed, lossless format that employs compression at a ratio of between 3:1 and 5:1. GIF is an 8 bit format that supports up to 256 colors. There are two GIF standards, 87a and 89a, which were developed in 1987 and 1989, respectively. The 89a standard has additional features such as improved interlacing, the ability to define a transparent color, and the ability to store multiple images in one file to create a basic form of animation.
. Joint Photographic Expert Group (JPEG) format was designed for compressing either full color (24 bit) or gray-scale digital images of natural or real world scenes. JPEG works well on photographs, naturalistic artwork, and similar material. JPEG does not work very well on lettering, simple cartoons, or black and white line drawings. JPEG does not handle moving images. JPEG is "lossy" which means that the image produced after decompression is not identical to the original image. The JPEG algorithm achieves an increase in compression over GIF by exploiting the limitations of the human eye. The human eye does not perceive small color details as well as small details of light and dark. JPEG is intended for compressing images that will be viewed by humans. A useful property of JPEG is that the degree of lossiness can be varied by adjusting compression parameters. This means that an image creator can trade-off file size against output image quality.
By comparing a compressed image with a real world scene, JPEG loses far less information than GIF. JPEG, however, is not going to displace GIF entirely. For some types of images, GIF is superior in image quality, file size, or both. GIF transparency and animation are not available in JPEG Generally speaking, however,
JPEG is superior to GIF for storing full color or gray scale images of realistic scenes.
Any continuous variation in color such as that which occurs in highlighted or shaded
areas will be represented more faithfully and in less space by JPEG than by GIF. GIF does significantly better on images with only , a few distinct colors, such as line drawings and simple cartoons. Not only is GIF lossless for such images, but it often compresses them more than JPEG can. Computer-drawn images, such as ray-traced scenes, usually fall between photographs and cartoons in terms of complexity. The more complex and subtly rendered the image, the more likely that JPEG will do well compressing it. JPEG has a hard time with very sharp edges, such as a row of pure- black pixels adjacent to a row of pure-white pixels. Sharp edges such as these are rare in scanned in photographs but are fairly common in GIF files such as borders, overlaid text, etc.
As the set top boxes attempt to interface with the Internet, the set top boxes must be able to handle GIF, JPEG and MPEG images quickly. Unfortunately, the set top boxes are unable to perform decompression of all three formats quickly enough to make conventional set top boxes a commercially acceptable interface for the Internet.
SUMMARY OF THE INVENTION An image display system includes a web server with web pages containing images in a first format. The first format includes a Graphics Interchange Format (GIF), a Joint Photographies Experts Group (JPEG) format and/or a Motion Pictures Experts Group (MPEG) format. A distributed communications system is connected to the web server. A proxy server is connected to the distributed communications system. The proxy server decompresses the first format images. The proxy server converts the first format images to a second format that includes either JPEG or
MPEG image formats. The proxy server compresses the second format images. A set
top box that is connected to the proxy server contains either hardwired circuits or hardware acceleration for quickly decompressing the second format images and for generating image frames. A display that is connected to the set top box allows viewing of the image frames. Other features, objects, and advantages will be readily apparent from the specification, the claims and the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a functional block diagram of a set top box compression system according to the invention;
FIG. 2A is a functional block diagram of a proxy server;
FIG. 2B is a data flow view illustrating the operation of the proxy server of
FIG. 1 in further detail;
FIG. 3A is a block diagram of the set top box of FIG. 1 in further detail; FIG. 3B is a data flow view illustrating the operation of the set top box;
FIG. 4A illustrates a graphics interchange format (GIF) frame that includes a transparent color;
FIG. 4B illustrates an MPEG image I-frame;
FIG. 4C illustrates a translucency MPEG I-frame; FIG. 5 illustrates a GIF animation sequence that includes multiple frames;
FIG. 6A illustrates the assembly of multiple GIF frames into a single MPEG
I-frame in a uniform manner;
FIG. 6B illustrates the assembly of multiple GIF frames into a single MPEG
I-frame in a non-uniform, asymmetric manner;
FIG. 6C illustrates the assembly of multiple GIF frames into multiple MPEG I- frames.
FIG. 7A is a flowchart illustrating steps for transforming GIF and JPEG images into MPEG I-frame(s); FIG. 7B is a flowchart illustrating steps for transforming MPEG and GIF images into JPEG images;
FIG. 8A is a flowchart illustrating steps for converting a GIF image with animation into MPEG I-frame(s);
FIG. 8B is a flowchart illustrating steps for converting a GIF image with translucency into MPEG I-frame(s);
FIG. 8C illustrates steps for converting a GIF image with translucency and animation into MPEG I-frame(s);
FIG. 8D illustrates steps for converting a GIF image with neither translucency nor animation into MPEG frame(s); FIG. 9 illustrates the addition of a low pass filter to improve transformation of
JPEG and GIF images into MPEG I-frame(s);
FIG. 10 illustrates steps for converting a JPEG image into MPEG I-frame(s);
FIG. 11A is a flow chart illustrating steps for converting a GIF image with animation into JPEG frame(s); FIG. 1 IB is a flow chart illustrating steps for converting a GIF image with translucency into JPEG frame(s);
FIG. 11C illustrates steps for converting a GIF image with translucency and animation into a JPEG frame(s);
FIG. 11D illustrates steps for converting a GIF image with neither translucency
nor animation to JPEG frame(s); and
FIG. 12 illustrates steps for converting an MPEG image into JPEG frame(s).
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Referring to FIG. 1, in a first embodiment a compression system for converting JPEG and GIF images into MPEG format for use in a set top box environment is illustrated at 10. In a second embodiment, the compression system 10 converts MPEG and GIF images to JPEG format. The compression system 10 includes one or more web servers 14-1, 14-2, ... , 14-n that are connected to a distributed communication system 18 such as the Internet. One or more proxy servers 22-1, 22-2, ..., 22-n are connected to the distributed communication system 18. A set top box 26 is connected directly to at least one of the proxy servers 22 and/or is connected through the distributed communications system 18 to at least one of the proxy servers 22. A display 30 is connected to the set top box 26. Skilled artisans can appreciate, that connections between the distributed communications system 18 and the proxy servers 22 and between the proxy servers 22 and the set top box 26 can be a cable connection such as coaxial cable or fiberoptic cable, a satellite-based connection, a terrestrial-based connection, or any other suitable connection. An IEEE 1394 wire may also be employed. Additionally, the set top box 26 can be integrated with electronics associated with the display 30 if desired. The display 30 can be a TV, a network TV display, or other suitable displays.
In use, each web server 14 delivers web pages that contain GIF, JPEG and
MPEG compressed images. A user associated with the set top box 26 and the display
30 requests access to the web pages provided by the web server 14 using a browser
that is associated with the set top box 26. In the first embodiment, the proxy server 22 converts the GIF and JPEG images delivered by the selected web server 14 to compressed MPEG I-frame(s) which are output to the set top box 26. In the second embodiment, the proxy server 22 converts MPEG and GIF images to JPEG images which are output to the set top box 26. The set top box 26 contains hardwired circuits or hardware acceleration to quickly decompress the MPEG I-frames or JPEG images. The set top box 26 transforms the MPEG I-frames or JPEG images into frames for output to the display 30.
Referring now FIG. 2A, the proxy server 22 is illustrated in further detail. A proxy server includes an input/output (I/O) interface 72 that is connected to a processor 74 and memory 78. The memory 78 preferably includes read only memory (ROM), random access memory (RAM) and/or external storage such as floppy drives, hard drives, optical drives, and other suitable storage media. The memory 78 includes an operating system server module 84, a compression module 86, a storage module 88, a first format to third format converter module 90, and a second format to third format converter module 92. In the first embodiment, the first format is the GIF format, the second format is the JPEG format, and the third format is the MPEG format. In the second embodiment, the first format is the GIF format, the second format is the MPEG format and the third format is the JPEG format. The proxy server 22 is preferably connected to the set top box through a broadcast network 95 using coaxial cable, fiberoptic cable, phone lines, satellites or other connections. The proxy server 22 is connected to the distributed communications system 18 in any of the ways previously described. For example, the proxy server 22 can be connected to the distributed communications system 18 using a local area net work (LAN) 96, a wide
area network (WAN) 98, a modem 100, etc. The proxy server 22 typically includes a display 102, a keyboard 104, a mouse 106, and other I/O devices 108.
Referring now to FIG. 2B, in use the browser in the set top box 26 transmits a uniform resource locator (URL) for a particular web page selected by the user to the proxy server 22. A set top box manager 120 receives the URL from the set top box 26. A lookup manager 122 checks memory to determine whether a web page that is associated with the URL is located in the memory 78 of the proxy server 22. If it is, the compressed MPEG I-frames in the first embodiment or JPEG images in the second embodiment are transmitted to the set top box 26 immediately. If not, a web server interface 130 contacts the web server and downloads the web page associated with the URL. If the web pages are in MPEG format, in the first embodiment or JPEG format, in the second embodiment, the web server interface 130 transmits the web pages to the set top box 26. Otherwise, a sort manager 134 sorts the web pages into JPEG and GIF formats in the first embodiment or into GIF and MPEG formats in the second embodiment.
The JPEG format web pages are converted from the JPEG format to the MPEG format at 136 in the first embodiment. The MPEG format web pages are converted from MPEG format to JPEG format in the second embodiment at 136. The GIF format web pages are converted from the GIF format to the MPEG format at 138 in the first embodiment. The . GIF format web pages are converted from the GIF format to the JPEG format in the second embodiment at 138. Compression occurs at
140. The compressed MPEG images in the first embodiment or the compressed JPEG images in the second embodiment are transmitted to the set top box 26. The proxy server 22 receives images that are in the MPEG, GIF and JPEG formats from one or
more web servers 14.
Referring now to FIG. 3A, the set top box 26 is illustrated in further detail and includes an input/output interface 150, a processor 154, and memory 156 that can include random access memory (RAM), read only memory (ROM), and other electronic storage such as a hard drive, a floppy drive, an optical storage device, or any other suitable memory. When the set top box 26 is turned on, an operating system 160 is loaded into the memory 156. A set top box browser 162 is also loaded into the memory 156 when needed. The set top box 26 is connected to a broadcast network 163. The set top box 26 is connected to the distributed communication system 18 in any of the ways described above with respect to FIG 2A. To that end, the I/O interface 150 can be connected to a local area network (LAN) 164, a wide area network (WAN) 168, a cable or phone modem 172, and/or to satellite or terrestrial- based systems. A display 30 is connected to the I/O interface 150. The set top box 26 can include I/O devices 176 such as a keyboard 180, a remote control 184, a joystick (not shown), a game pad (not shown), etc. The set top box 26 further includes a hardwired or hardware accelerated MPEG decompression circuit 188 in the first embodiment or a hardwired or hardware accelerated JPEG decompression circuit in the second embodiment 188.
Referring now to FIG. 3, the set top box 26 requests the URL for a selected web page using the browser 162. The proxy server 22 receives the request. In the first embodiment, the proxy server 22 converts the JPEG or GIF images that are associated with the requested web page to MPEG I-frames and transmits MPEG I- frames to the set top box 26. The set top box 26 receives compressed MPEG I-frames . from the proxy server 22. The MPEG decompression circuit 188 decompresses the
MPEG I-frames and generates frames for output to the display 30.
In the second embodiment, the proxy server 22 converts MPEG and GIF images to JPEG images and transmits the JPEG images to the set top box 26. The set top box 26 receives the compressed JPEG images from the proxy server 22. The JPEG decompression circuit 188 decompresses the JPEG images and generates frames for output to the display 30.
Referring now to FIGS. 4A-4C, a GIF frame 200 is illustrated and includes an image 202 and a transparent or translucent area 204 that surrounds the image 202. During conversion in the proxy server 22, the proxy server 22 separates the transparent or translucent area 204 from the GIF image 202. In the first embodiment, the proxy server 22 creates a first MPEG I-frame 206 from the GIF image 202. Depending upon the amount of information contained in the transparent or translucent area 204, a second MPEG I-frame or GIF image 208 can be created therefrom.
In the second embodiment, the proxy server 22 creates a first JPEG frame 206 from the GIF image 202. Depending upon the amount of information contained in the transparent or translucent area 204, a second JPEG frame 208 or a GIF image is created therefrom. If the area 204 is transparent, a single bit for each pixel is sufficient to describe the transparent mask. In this situation, the transparent information is preferably sent as a GIF image. If the area 204 is translucent, 8 bits or more may be employed to describe the translucency mask for each pixel in the frame 200. In these circumstances, an MPEG I-frame (first embodiment) or JPEG frame (second embodiment) is preferably used.
Referring now to FIG. 5, an animated GIF image 210 is illustrated and includes multiple GIF frames 212, 216, 220, 224, 228 and 232. Each frame contains
an image 236, 238, 242, 246, 250 and 254. Translucency information may also be contained in each GIF frame and is designated at 260, 264, 268, 272, 276 and 280. Each frame may also contain timing information which is generally designated TI, T2, T3, T4, T5, and T6. The timing information designates how long a particular frame is displayed.
In the first embodiment, the proxy server 22 converts the GIF frames 212, 216, 220, 224, 228 and 232 into a single MPEG I-frame that is generally identified at 284. The proxy server 22 creates a custom MPEG header information identifying a region (Rl, R2, R3, R4, R5 and R6) within the MPEG I-frame 284 that contains each GIF frame and the timing (TI, T2, T3, T4, T5 and T6) for each GIF frame. For example, a first region field (Rl) in the MPEG header contains I-frame coordinates for the GIF frame 212. A first timing field (TI) in the MPEG header contains timing for the GIF frame 212. Similarly, the GIF frames 216, 220, 224, 228 and 232 are associated with R2 and T2, R3 and T3, R4 and T4, R5 and T5, and R6 and T6, respectively. In the second embodiment, the proxy server 22 converts the GIF frames 212,
216, 220, 224, 228 and 232 into a single JPEG frame 284. The proxy server 22 creates a custom JPEG header information identifying a region (Rl, R2, R3, R4, R5 and R6) within the JPEG frame 284 that contains each GIF frame and the timing (TI, T2, T3, T4, T5 and T6) for each GIF frame. For example, a first region field (Rl) in the JPEG frame contains coordinates for the GIF frame 212. A first timing field (TI) in the JPEG header contains timing for the GIF frame 212. Similarly, the GIF frames
216, 220, 224, 228 and 232 are associated with R2 and T2, R3 and T3, R4 and T4,
R5 and T5, and R6 and T6, respectively.
Referring now to FIGS. 6A, 6B and 6C, additional ways of placing GIF
images that are generally identified at.286, 287 and 289 in MPEG I-frame(s) 284'; 284" '-1 and 284"'-2 (in the first embodiment) are illustrated. While each region 286 in FIG. 6A is approximately the same size, the regions may have different sizes as can be seen in FIG. 6B. The relative size of each region and the relative orientation of each GIF image within the MPEG I-frame will depend upon the characteristics of the GIF images. In FIG. 6C, the image is split across several MPEG I-frames 284" '-1 and 284" '-2. Skilled artisans can appreciate that in the second embodiment the GIF images are placed in one or more JPEG frames (in the second embodiment) in a similar manner. Referring now to FIG. 7A, a flowchart 300 that illustrates steps for converting
GIF and JPEG images from the web servers 14 to the MPEG format in the first embodiment is illustrated. In step 304, the proxy server 22 (or the processor 74 that is associated with the proxy server 22) reads an image header. If the image is a GIF image, as determined at step 308, control continues with step 312 where control determines whether the GIF image contains animation only. If it does, control continues with FIG. 8A. Otherwise, control continues with step 316 where control determines whether the GIF image includes translucency information. If translucency information is involved, control continues with FIG. SB. Otherwise, control continues with step 318 where control determines whether the GIF image includes both animation and translucency. If both are involved, control continues with FIG. 8C.
Otherwise, control continues with FIG. 8D. If the image is not a GIF image, control continues with step 322 where control determines whether a JPEG image is present. If a JPEG image is present, control continues with FIG. 10. Otherwise, control performs other processing on the image. If the image is an MPEG image, the proxy server 22
forwards the MPEG image without processing.
Referring now to FIG. 8A, the steps for converting a GIF image with animation to an MPEG I-frame are illustrated. At step 330, control decompresses the GIF image. At step 334, control recovers GIF frames within the GIF image animation. At step 336, control creates an MPEG header and an MPEG I-frame for the frames as previously described. At step 338, control compresses the MPEG I- frame or frames and transmits the MPEG I-frame(s) to the set top box 26 and/or stores the MPEG I-frame(s) at the proxy server 22.
Referring now to FIG. 8B, control converts a GIF image with translucency information. At step 342, control decompresses the GIF image. At step 346, control recovers image information. At step 350, control recovers translucency information. At step 354, control creates an MPEG header and I-frame for the image. At step 358, control creates a GIF header and frame or an MPEG header and I-frame for the translucency information. At step 362, control compresses the MPEG I-frames and transmits the MPEG I-frame(s) to the set top box 26 and/or stores the MPEG I- frame(s) at the proxy server 22.
Referring now to FIG. 8C, control converts a GIF image with transparency and animation information. At step 370, control decompresses the GIF image. At step 374, control recovers animation frames. At step 378, control recovers translucency information for each frame. At step 382, control creates MPEG headers and I-frames for the images in the frames. At step 386, control creates GIF header and frames or MPEG headers and I-frames for the translucency information
(depending upon the amount of information present). At step 390, control compresses the MPEG I-frames and transmits the MPEG I-frame(s) to the set top box 26 and/or
stores the MPEG-I frame(s) at the proxy server 22.
Referring now to FIG. 8D, control decompresses GIF images that do not contain either translucency or animation information. At step 400, control decompresses the GIF image. At step 404, control recovers image information. At step 408, control creates MPEG headers and I-frames for the image. At step 412, control compresses the MPEG I-frames and transmits the MPEG I-frame(s) to the set top box 26 and/or stores the MPEG I-frames at the proxy server 22.
Referring now to FIG. 9, a filter is employed to improve the results of the MPEG compression. In a preferred embodiment, the filtering is low-pass filtering using a conventional psycho-visual model. Low-pass filtering is employed because MPEG uses a fewer number of bits to quantize high-frequency components of an image. Low-pass filtering reduces inter-block artifacts that would otherwise occur without filtering. The low-pass filtering is performed prior to compression. Skilled artisans can appreciate that each of the conversion processes described in FIGS. 8A-8D can include as a filtering step prior to MPEG compression. Other types of filtering to optimize the results of the compression and subsequent decompression are also contemplated. In step 420, the GIF image is decompressed. In step 422, the image information is recovered. In step 426, the image is passed through a low-pass filter. In step 430, an MPEG header and an MPEG I-frame is created for the image. In step 434, the MPEG I-frame is compressed and transmitted to the set top box 26 and/or stored at the proxy server 22.
Referring now to FIG. 10, steps for converting JPEG images to MPEG I- frames is illustrated. At step 440, the JPEG image is decompressed. At step 444, the image is recovered. At step 448, an MPEG header and an MPEG I-frame are created. ■
At step 452, control compresses the MPEG I-frame and transmits the MPEG I-frame to the set top box 26 and/or stores the MPEG I-frames at the proxy server 22.
Referring now to FIG 7B, a flow chart 480 that illustrates steps for converting GIF and MPEG images from the web servers 14 to the JPEG format in the second embodiment is illustrated. In step 482, the proxy server (or the processor 74 that is associated with the proxy server 22) reads an image header. If the image is a GIF image, as determined at step 484, control continues with step 488 where control determines whether the GIF image contains animation only. If it does, control continues with FIG. 11A. Otherwise, control continues with step 492 where control determines whether the GIF image includes translucency information. If translucency information is involved, control continues with FIG. 11B. Otherwise, control continues with step 494 where control determines whether the GIF image includes both animation and translucency. If both are involved, control continues with FIG. 1ΪD. If the image is not a GIF image, control continues with step 496 where control determines whether an MPEG image is present. If an MPEG image is present, control continues with FIG. 12. Otherwise control performs other processing on the image. If the image is a JPEG image, the proxy server 22 forwards the JPEG image without processing.
Referring now to FIG. 11 A, the steps for converting a GIF image with animation to a JPEG image is illustrated. At step 500, control decompresses the GIF image. At step 504, control recovers GIF frames with the GIF image animation. At step 508, control creates a JPEG header and a JPEG frame. At step 512, control compresses the JPEG frame and transmits the JPEG frame to the set top box 26 and/or stores the JPEG frame at the proxy server 22 for subsequent use.
Referring now to FIG. 11B, control converts a GIF image with translucency information. At step 520, control decompresses the GIF image. At step 524, control recovers image information. At step 528, control recovers translucency information. At step 532, control creates a JPEG header and a JPEG frame for the image. At step 536, control creates a GIF header and frame or a JPEG header and JPEG frame for the translucency information depending upon the number of bits required to describe the transparent or translucent information. At step 540, control compresses the JPEG frames and transmits the JPEG frames to the set top box 26 and/or stores the JPEG frames at the proxy server 22 Referring now to FIG. 11C, control converts a GIF image with transparency and animation information. At step 550, control decompresses the GIF image. At step 554, control recovers animation frames. At step 558, control recovers translucency information for each frame. At step 560, control creates JPEG headers and frames for the images in the frames. At step 564, control creates GIF headers and frames or JPEG headers and frames for the translucency information depending upon the amount of transparent or translucent information" present. At step 566, control compresses the JPEG frames and transmits the JPEG frames to the set top box 26 and/or stores the JPEG frames at the proxy server 22.
Referring now to FIG. 11D, control decompresses GIF images that do not contain either translucency or animation information. At step 570, control decompresses the GIF image. At 574, control recovers image information. At step
578, control creates JPEG headers and frames for the image. At step 585, control compresses the JPEG frames and transmits the JPEG frames to the set top box 26 and/or stores the JPEG frames at the proxy server 22.
Referring now to FIG. 12, steps for converting MPEG images to JPEG images is illustrated. At step 570, the MPEG image is decompressed. At step 574, the image is recovered. At step 578, a JPEG header and a JPEG frame are created. At step 582, control compresses the JPEG frame and transmits the JPEG frame to the set top box 26 and/or stores the JPEG frame at the proxy server 22.
Since the process to convert JPEG and GIF images to MPEG format or to convert MPEG and GIF images to JPEG format may be a rate-limiting step, the proxy server 22 or another off-line computer system preferably performs off-line conversion and storage for web sites that will be accessed regularly. The proxy servers 22 and/or web servers 14 can store web pages and/or imager and other information in memory cache and/or disk cache to improve loading speeds. In addition, if multiple set top box users are accessing a web site, the converted MPEG or JPEG images can be stored in the memory 78 that is associated with the proxy server 22.
Filtering prior to JPEG compression is preferably performed to improve the results of JPEG compression in a manner similar to that described above in FIG. 9.
As can be appreciated, the hardwired or hardware accelerated MPEG or JPEG compression circuit 188 substantially decreases the response times for the GIF, JPEG or MPEG images for the set top box 26. As a result, the set top box 26 becomes a viable delivery portal to the Internet while continuing to provide other multimedia content.
Those skilled in the art can now appreciate from the foregoing description that the broad teachings of the present invention can be implemented in a variety of forms.
Therefore, while this invention has been described in connection with particular examples thereof, the true scope of the invention should not be so limited since other
modifications will become apparent to the skilled practitioner upon a study of the drawings, specification, and the following claims.