SYSTEM AND METHOD FOR SCALING A VIDEO SIGNAL BACKGROUND OF THE INVENTION
Field of the Invention
This invention relates generally to the field of video conversion systems. More particularly, the invention relates to a system and method which determines the proper horizontal and vertical scaling parameters for displaying a digital video signal.
Description of the Related Art
A "standard" television image (i.e., as defined by the National Television Standards Committee; hereinafter "NTSC"), has an aspect ratio of 4:3 (i.e., image width : image height). As illustrated in Figure 1, three areas are defined within a standard television image: an "active" area 105; a "safe action" area 120; and a "safe title" area (defined by dotted line 130 in Figure 1). The "active" area 105 represents the entire television image as it is scanned and broadcast (i.e., all of the transmitted video information). The "safe action" area 120 is the portion of the active area 105 which will be visible on a typical television set 110. In other words, it is the portion of the active area which is not hidden by the bezel surrounding the television's 110 picture tube (TV sets in the United States are configured to "overscan" so that portions of the transmitted video image around the periphery of the television screen cannot be seen).
The concept of a "safe title" area was initially developed to deal with the problem of imperfections on early television sets (e.g., image distortion towards the edges of the picture tube). Thus, the "safe title" area is where text, such as movie titles or news anchor names, will be clearly displayed on most television sets.
As illustrated in Figure 2, in contrast to a standard television 210, a "wide screen" television 220 typically has an aspect ratio of 16:9. Wide screen
sets may or may not be "high definition television" (hereinafter "HDTV") compatible. The most common HDTV standard, introduced by the Advanced Television Standards Committee (hereinafter "ATSC"), includes up to 1080 lines of video data with up to 1920 pixels /line (interlaced). By contrast, non- HDTV wide screen TVs can have significantly fewer lines (e.g., 525) and fewer pixels /line. Both non-HDTV wide screens and most HDTV wide screens are capable of displaying standard NTSC images.
In order to take advantage of the improved aspect ratio of a wide screen system, several different video encoding formats currently exist, each of which use different horizontal and vertical scaling variables for storing the underlying video content. Two general types of scaling formats exist: "anamorphic" scaling formats, in which the horizontal and vertical portions of the encoded image are scaled by different amounts; and "symmetric" scaling formats, in which the horizontal and vertical portions of the image are scaled evenly. The particular type of scaling format used is based on the native aspect ratio of the underlying video content and the aspect ratio of the television on which it will be displayed (i.e., 16:9 in the case of most wide screen systems).
Figure 3 illustrates some of the concepts associated with anamorphic scaling. To encode video in an anamorphic format, the image may be stretched in the vertical direction to include more video data than the total number of scan lines available on a standard television. This is done to pack as much resolution into the video signal as possible. Thus, the actual encoded frame 305 and images within it (e.g., oval 306) appear to be stretched in the vertical direction if displayed on a standard television monitor. The frame 306 and images 316 are displayed properly on a wide screen monitor (e.g., circle 316 appears as a circle and not an oval) because wide screen televisions may be configured to scale the horizontal and vertical portions of the encoded image by different amounts.
One problem that exists with the foregoing configuration, however, is that although various different scaling formats - each of which uses different horizontal and vertical scaling variables - no reliable mechanism currently exists for determining the appropriate horizontal and vertical scaling variables to use when decoding and displaying different video formats on a wide screen system. Although content providers sometimes attempt to encode aspect ratio data within video streams (e.g., on DVDs), this data is frequently inaccurate. Moreover, many televisions and DVD players are not capable of accepting this aspect ratio data. As such, video content may appear distorted when displayed on certain wide screen systems (i.e., those that use the improper scaling factors). In addition, there is no reliable mechanism currently available for determining the ideal wide screen scaling factors when decoding/displaying video images initially encoded for display on standard TVs (including those which factor in the overscanning effect and those which do not).
Accordingly, what is needed is and improved system and method for determining the ideal horizontal and vertical scaling parameters for decoding and displaying digital video content.
SUMMARY
A method is disclosed for determining scaling variables for video comprising the steps of: identifying an image within the video which has a predetermined size and shape when scaled properly; and adjusting the scaling variables for the video until the image matches the predetermined size and shape.
Also disclosed is a system for determining scaling variables for video comprising: a database for storing one or more images; and a pattern comparator for comparing the images with images in a scaled video, and selecting particular scaling variables responsive to matching one of the images with an image included in the scaled video data.
Also disclosed is a system for scaling a video for wide screen television using proper horizontal and vertical scaling variables comprising: video identification means for determining the type of video signal to be displayed; scaling variable selection means for selecting the ideal horizontal and vertical scaling variables based on said type; and scaling means for scaling the video by an amount specified by the horizontal and vertical scaling variables to produce scaled video.
BRIEF DESCRIPTION OF THE DRAWINGS
A better understanding of the present invention can be obtained from the following detailed description in conjunction with the following drawings, in which:
FIG. 1 illustrates how the active area of a standard television image is displayed on a television screen.
FIG. 2 illustrates differences in aspect ratio between a standard television and a wide screen television.
FIG. 3 illustrates the concept of anamorphic encoding and how an anamorphic image is mapped to a wide screen system.
FIG. 4 illustrates how a first type of symmetric image should be mapped to a wide screen system.
FIG. 5 illustrates how a second type of symmetric image should be mapped to a wide screen system.
FIG. 6 illustrates how a third type of symmetric image should be mapped to a wide screen system.
FIG. 7 illustrates how a fourth type of symmetric image should be mapped to a wide screen system.
FIGS. 8a-b illustrate generally how different types of anamorphic images should be mapped to a wide screen system.
FIG. 9 is a table which classifies several known video encoding types into eight different categories.
FIG. 10 is a table which identifies particular movies encoded using each of the eight different video encoding types.
FIG. 11 illustrates one embodiment of a system for determining the appropriate scaling variables for a particular video.
FIG. 12 illustrates another embodiment of a system for determining the appropriate scaling variables for a particular video.
FIG. 13 illustrates one embodiment of a system for determining the appropriate scaling variables using pattern matching techniques.
FIG. 14 illustrates another embodiment of a system for determining the appropriate scaling variables using pattern matching techniques.
FIG. 15 illustrates another embodiment of a system for determining the appropriate scaling variables for a particular video.
FIG. 16 illustrates another embodiment of a system for determining the appropriate scaling variables using pattern matching techniques.
DETAILED DESCRIPTION
In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of the invention.
Embodiments of the present invention include various steps, which will be described below. The steps may be embodied in machine-executable instructions. The instructions can be used to cause a general-purpose or special-purpose processor, which is programmed with the instructions to perform certain steps. Alternatively, these steps may be performed by specific hardware components that contain hardwired logic for performing the steps, or by any combination of programmed computer components and custom hardware components.
Elements of the present invention may also be provided as a computer program product which may include a machine-readable medium having stored thereon instructions which may be used to program a computer (or other electronic device) to perform a process. The machine-readable medium may include, but is not limited to, floppy diskettes, optical disks, CD-ROMs, and magneto-optical disks, ROMs, RAMs, EPROMs, EEPROMs, magnet or optical cards, propagation media or other type of media /machine-readable medium suitable for storing electronic instructions. For example, the present invention may be downloaded as a computer program product, wherein the program may be transferred from a remote computer (e.g., a server) to a requesting computer (e.g., a client) by way of data signals embodied in a carrier wave or other propagation medium via a communication link (e.g., a modem or network connection).
EMBODIMENTS OF A SYSTEM AND METHOD FOR SCALING A VIDEO SIGNAL
The scaling/zoom variables for several different types of encoded video are listed in Figure 9 and some examples of actual movies encoded for wide screen display are enumerated in Figure 10. Each type of encoded video and corresponding scaling/zoom variables (hereinafter "scaling variables") will now be described with respect to Figures 4-8. The manner in which the appropriate scaling variables are determined by various embodiments of the system and method is described below with respect to Figures 11-14.
Scaling of Symmetric-Encoded Images
Figure 4 illustrates how the active area of a typical 16 mm movie image 405 may be converted and displayed (as image 406) on a wide screen system 420 (e.g., an HDTV). The characteristics and related scaling variables associated with this type of encoded video image are listed in row 1 of Table 900. As indicated, the entire active area of the image 405 is scaled to fit vertically within the wide screen 420 (i.e. so that the upper and lower boundaries of the image correspond with the upper and lower edges of the wide screen). This is done because the entire active area 405 of the 16 mm movie was meant to be displayed and viewed (i.e., the active area 405 was not originally produced for television where overscanning clips the outer edges of the image). Due to aspect ratio differences of the 16 mm image 405 and the wide screen, two areas 410 and 411 of the wide screen will not be used (i.e., will typically appear as black or solid color fields). The 16 mm image is encoded in a symmetrical format, so the horizontal and vertical scaling variables will be the same (e.g., both equal to 1.00 in the illustrated embodiment).
Figure 5 illustrates the manner in which a standard television video image 505 (i.e., video originally produced for TV) may be scaled and displayed
on a wide screen system 520. In this case, the image is scaled so that the top and bottom edges of the safe action region 540 match up with the top and bottom of the wide screen 520, respectively. The portions of the active area 505 which are not displayed - regions 530-531 - contain video footage which would not be visible on a standard TV due to overscanning. Because these regions were not intended by the videographer to be seen, it is not necessary to display them on the wide screen system. Thus, this scaling factor may be useful for scaling standard NTSC broadcasts (e.g., TV newscasts).
It should be noted, however, that the scaling factors described above for 16 mm movies may also be used to display standard television material. In this configuration, the portions of the TV image outside of the safe action area 540 (i.e., the portions which were not intended to be visible during normal viewing) are visible on the wide screen. For example, errors in production (such as microphones suspended overhead) can sometimes be seen in this region, which can be interesting. The decision as to which scaling variables to use will be based on the preferences of each end user.
Figure 6 illustrates an embodiment in which the active area of the video image 605 is scaled so that the upper and lower portions of the safe title region, defined by inner doted line 640, correspond to the upper and lower regions of the wide screen, respectively. This scaling configuration is useful for displaying a particular type of letterboxed movie in which blacked out boundaries 650 on the top and bottom of the image approximate the boundaries 640 of the safe title region when viewed on a standard TV. Using the proper terminology, these movies are "letter-boxed" within the safe title region for the purpose of preserving more of the original movie image (i.e., to the left and right). Because of the 16/9 aspect ratio of wide screen, the active area of the original movie can fit within the wide screen with only limited blacked out areas on the top and bottom of the picture. Thus, one appropriate scaling factor for this type of encoded movie is one which - as illustrated in
Figure 6 - matches the safe title border on the top and bottom of the image with the top and bottom of the wide screen, respectively.
The active region 705 of Figure 7 is a fully letter-boxed movie. In other words, as described in the background section, the entire original movie image is fit width-wise into a standard TV frame. When a movie encoded in this manner is played on a wide screen system, the "appropriate" scaling variables are those in which the left and rights edges of the TV image are matched up with the left and right edges of the wide screen, respectively. But, unlike anamorphic scalings, in this type of encoding, the horizontal and vertical scaling factors are the same.
Scaling of Anamorphic-Encoded Images
All of the images described above were encoded "symmetrically," meaning that the horizontal and vertical portions of the video image were scaled the same amount during encoding. Accordingly, the horizontal and vertical scaling variables (i.e., the "H Zoom/Scale" and "V Zoom/Scale" values) are equal for each of the first four types of encoded images set forth in rows 1-4 of Table 900 (which correspond to the embodiments illustrated in Figures 4-7, respectively).
As described above, however, many images encoded specifically for display on a wide screen system are encoded in an "anamorphic" format, using different horizontal and vertical scaling values. As indicated in rows 5-8 of Table 900, and as illustrated in Figure 8a-b, at least four different anamorphic horizontal and vertical scaling variables currently exist for encoded movies. However, no system or method currently exists for reliably determining the appropriate scaling values to be applied when playing back a particular movie. As such, many DVD players and /or wide screens play back anamorphic- encoded movies using improper scaling values, resulting in image distortion (e.g., "stretching" of the image).
Scaling Analysis
One embodiment of a system for determining appropriate scaling variables (symmetric or anamorphic) for a video is illustrated in Figure 11. In this embodiment, the appropriate H/V scaling variables 1130 are transmitted as part of the encoded video stream (e.g., an MPEG-2 video stream). A video player/decoder 1120 decodes the video stream and separates the underlying video data 1140 from the H/V scaling variable information 1130. An appropriate H/V scaling variable 1130 is then transmitted to a sealer unit 1150 which scales the video data 1140 for a wide screen system display 1160 in the appropriate manner. It should be noted that the video player /decoder 1220 and/or the H/V sealer unit 1260 may be included within a DVD player, a digital satellite receiver, a digital cable box, a computer, or any other device used to receive and play back digital video.
In another embodiment of the system and method illustrated in Figure 12, the H/V scaling variables 1245 are not necessarily transmitted with the encoded video signal 1210. Rather, in this embodiment, the video player/decoder 1220 communicates over a network 1230 with one or more servers 1250. Depending one the particular embodiment, this communication may occur over various media types, including (but not limited to) standard telephone lines, digital cable lines, digital subscriber lines ("DSL"), wireless broadcast, digital cellular or any other digital communication media.
The video player /decoder 1220 communicates video identification ("ID") information 1240 to the server 1250 which the server 1250 uses to retrieve the appropriate H/V scaling variables 1245 from a database 1255. The video ID information 1240 may be the title and/or serial number of the video stream 1210 being played. This information is typically stored on DVDs. In addition, each DVD has a unique "fingerprint" which can be generated and used as the video ID information 1240. For example, a fingerprint may be generated by executing a checksum on a known unique portion of a DVD.
Regardless of what ID information is used to identify the particular DVD being played, the underlying principles of the system remain the same.
If the video player /decoder 1220 is part of a digital satellite or cable receiver, the video ID information 1240 may be retrieved from the schedule for the particular channel being viewed. For example, cable and digital broadcast companies broadcast a schedule of the video content to be aired for each channel. Consequently, this schedule may be used as the video ID information 1240 for retrieving the appropriate scaling variables 1245 from the database 1255.
Once retrieved, the H/V scaling information 1245 is then transmitted back from the server 1250 and is used by the H/V sealer unit 1260 to scale the incoming video signal using the correct H/V scaling variables. Although Figure 12 illustrates the H/V scaling variables 1245 being initially transmitted to the video player/decoder 1220 and then to the H/V sealer unit 1260, in one embodiment the H/V scaling variables 1245 are transmitted directly to the H/V sealer unit 1260 from the server 1250.
In one embodiment of the system, the video player /decoder 1220 itself contains a local database (not shown) for storing H/V scaling variables 1245. This local database may be updated each time a new H/V scaling factor is retrieved from database 1255 (and associated with specific video material identified with the video ID information 1240).
The database 1255 on server 1250 (or the local database) in one embodiment of the system is updated with new H/V scaling variables 1245 on a periodic basis as new video material becomes available to the public. There are several ways in which the appropriate H/V scaling variables 1245 may be determined for each new video. One method is to search through the video for an image of known dimensions (e.g., one which is supposed to be perfectly round such as a wheel of a car or a clock). If the image is oval rather than
round, then the H/V scaling variables 1245 being used are incorrect. Thus, several different sets of H/V scaling variables 1245 may be tested until the correct set is found. Once the correct H/V scaling variables 1245 have been identified, database 1255 (or the local database) is updated accordingly.
In one embodiment, illustrated in Figure 15, the video data and associated H/V scaling variables 1545 are transmitted directly from the player decoder 1520 to the wide screen system 1570. This embodiment may be implemented on wide screen systems which are capable of accepting scaling variables and scaling an incoming video signal based on the variables.
In any of the embodiments described herein, H/V scaling variables 1245, 1545 and video data may be transmitted from the video player /decoder 1220, 1520 to the sealer unit 1260 or to the wide screen system 1570 using either terrestrial or wireless transmission media. For example, in one embodiment the player/decoder 1220, 1520 uses a wireless infrared signal to transmit the scaling variables 1245, 1545 to the wide screen display 1570 (or sealer unit 1260, depending on the configuration). Other possible transmission channels include, but are not limited to, Sony S-Link channels, IEEE 1394 channels, Bluetooth channels, and/or various other terrestrial and wireless channels. It should be noted, however, that the underlying principles of the invention are not limited to any particular transmission channel.
The foregoing process can either be carried out manually as described, or automatically using video pattern matching techniques. One embodiment of a system which automatically determines the appropriate H/V scaling variables 1345 for a particular video is illustrated in Figure 13. In this embodiment, a database 1355 (which may reside remotely on a server or locally on a the digital video player system, or both) is loaded with images which are known to be found on most videos. For example, the database may be loaded
with video images of the various video distributor and/ or movie producer logos such as New Line Cinema,™ Universal Pictures,™ or Columbia Pictures.™
The output of the H/V sealer unit 1360 is fed back into a pattern comparator 1315 which performs one or more pattern matching techniques to match the standard logo (or other image) stored on database 1355 with the scaled image. An image match indicates that the logo or other image is being displayed properly and, therefore, the appropriate H/V scaling variable 1345 has been identified and will be used for the remainder of the video playback.
In one embodiment, if a match cannot be found on a local database 1355, then the system will search for an appropriate image on a remote database (not shown). The remote database in this embodiment is one which is updated with new logos (or other known images) on a regular basis. One embodiment of the system will allow a user to override the pattern comparator 1315 and manually select particular H/V scaling variables 1345 (e.g., in case he wants to watch a video with a certain amount of distortion).
In an alternative embodiment illustrated in Figure 14, unsealed video images are transmitted to the pattern comprator 1415 for comparison to images stored on the database 1455. Accordingly, in this embodiment, a single logo may be stored in numerous different unsealed formats on the database 1455. Each unsealed logo will have slightly different characteristics uniquely identifying the incoming video type (i.e., which may, for example, correspond with one of the eight types set forth in Table 900). Each of the different unsealed image formats (i.e., represented by image data 1450) are compared to the incoming unsealed video and, when a match is found, the pattern comparator will select appropriate H/V scaling variables 1445 for playback.
In one embodiment, illustrated in Figure 16, the video data and associated H/V scaling variables 1645 are transmitted directly from the player decoder 1620 to the wide screen system 1670. As with the case with the
embodiment illustrated in Figure 15, this embodiment may be implemented on wide screen systems which are capable of accepting scaling variables and scaling an incoming video signal based on the variables. Moreover, as described above, various transmission channels may be used to transmit scaling variables 1645 and/or video data to the side screen system (e.g., Sony S- Link channels, IEEE 1394 channels, Bluetooth channels, and/or various other terrestrial and wireless channels, . . . etc).
Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the present system and method. For example, the discussion above focused on scaling variables for wide screen systems having an aspect ratio of 16:9. It will be appreciated, however, to one skilled in the art that the system and method may be practiced on wide screen systems having various other aspect ratios (e.g., 18:10). Accordingly, the scope and spirit of the invention should be judged in terms of the claims which follow.