Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/16—Analogue secrecy systems; Analogue subscription systems
  • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
  • H04N7/17345—Control of the passage of the selected programme
  • H04N7/17354—Control of the passage of the selected programme in an intermediate station common to a plurality of user terminals
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q20/00—Payment architectures, schemes or protocols
  • G06Q20/08—Payment architectures
  • G06Q20/12—Payment architectures specially adapted for electronic shopping systems
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
  • G06Q20/00—Payment architectures, schemes or protocols
  • G06Q20/08—Payment architectures
  • G06Q20/12—Payment architectures specially adapted for electronic shopping systems
  • G06Q20/123—Shopping for digital content
  • G06Q20/1235—Shopping for digital content with control of digital rights management [DRM]
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
  • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/80—Details of filtering operations specially adapted for video compression, e.g. for pixel interpolation
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N7/00—Television systems
  • H04N7/16—Analogue secrecy systems; Analogue subscription systems
  • H04N7/173—Analogue secrecy systems; Analogue subscription systems with two-way working, e.g. subscriber sending a programme selection signal
  • H04N7/17309—Transmission or handling of upstream communications
  • H04N7/17318—Direct or substantially direct transmission and handling of requests

Definitions

• the present invention relates generally to methods and apparatus for prepurchase browsing of video material, and more particularly to methods and apparatus to limit the content of video clips without detracting from providing a useful browsing system to a prospective customer.
• Any browsing system must protect the associated copyrightable material or intellectual property from misappropriation by browsers or customers, while encouraging customers to select and purchase the material.
• the owners of the copyrightable material are unlikely to approve any browsing system to be implemented therewith unless such protection is inherent in the system.
• Browsing when used as a sales tool for video sequences, must balance the needs and wants of the customer and merchant: the merchant must offer the customer a preview to allow the customer to select what he or she wants; but the merchant does not want to give away the product without payment from the customers.
• key-frame browsing discourages theft by not giving the customer the full clip in advance, it may discourage the purchase of some clips because it is not full motion video.
• Perceptual studies show that people can recognize important features from heavily-filtered image sequences. The present inventors discovered that such sequences can be generated directly from a compressed video sequence, saving both bandwidth and computation time.
• One known browser divides the video sequence into several equal segments and denotes the first frame of each segment as its key frame. This approach provides a rather hit-or-miss view of the contents of the video. Another known browser stacks every frame of the sequence. This browser provides the user with rich information regarding the camera and object motions. However, a scholar/researcher using a digital video library or a customer of a pay-per-view system is more likely to be more interested in the contents (who, what, where) than how the camera was used during the recording. To address the content-based browsing needs, a third known browser uses Rframes (representative frames) to organize the visual contents of the video clips. Rframes may be grouped according to various criteria to aid the user in identifying the desired material.
• storyboard browsers can prevent the majority of video data from falling prey to intellectual property thefts, however, all these storyboard browsers inherently eliminate motion from the browsing copy.
• motion is an important aid to recognition and understanding of video. Browsing of sports, action sequences in news, and entertainment may require motion to identify the material of interest.
• Even scholarly collections may offer some material on a pay-per-use basis, much as the Vanderbilt Television News Archive and the Purdue C-SPAN Collection. Organizations which own the copyrights on this material may sell some of it in secondary markets and wish to maintain its value, they will often charge users small sums even for non-commercial use.
• Browsers may also provide playback facilities which use key frames as one method of nonlinear access to the source material. Browsing of pay-per-view material requires a scheme which encourages the customer to rent the source material but not to use the browsing version of the material. If the original source material is shown in its entirety (or even a substantial fraction of the source's length) during browsing, the customer may be tempted to view the material once during browsing without paying the rental fee; the customer can also videotape the screen during browsing to obtain a permanent record of the browsing material. If one is to provide full-motion browsing copies of video material, they must provide some way to limit the amount of information in the browsing copy. Perceptual studies show that people do not need all the visual information to recognize important features of images and that the qualities which viewers classify as high image quality do not match those features required for accurate recognition of the video material. Key frames are a form of temporal sampling of the video material.
• the present invention provides an improved method and apparatus for pre ⁇ purchase browsing of video material.
• the customer is allowed to access and view full motion video clips that are a spatially filtered version of the video sequence.
• the filtered sequence retains enough information to make the material recognizable and attractive to the customer, but is not of sufficiently good image quality to tempt the customer to substitute the free browsing copy for the purchasable video source material.
• Browsing copy is extracted directly from Motion JPEG and MPEG video streams, in one embodiment of the invention, for providing substantial savings in the computational, storage and transmission costs.
• Temporal approximation and modelling of compressed video data is used to further enhance the performance of various embodiments of the present theft-resistant browser system invention.
• spatial filtering in conjunction with temporal filtering, is used to provide visual information essential for browsing without plausible the customer to substitute the free browsing copy for the purchased source material.
• payment means are provided for permitting a customer to pay a progressively higher fee, for progressively increased quality of the browsing copy.
• Fig. 1A is a pictorial presentation of a full video image having a resolution of 320 X 240 pixels;
• Fig. IB shows the image of Fig. 1A lowpass filtered by (DC+2AC) a resolution of 80 X 60 pixels;
• Fig. IC shows the image of Fig. 1A lowpass filtered via use of only DC coefficients at a resolution of 40 X 30;
• Fig. 2A shows an Intra-coded image divided into 8 X 8 blocks
• Fig. 2B shows an enlargement of an 8 X 8 block of Fig. 2A;
• Fig. 2C shows blocks of pixels within a video frame, including a reference block P ref , motion vectors and original blocks P,, P 2 , P 3 , and P 4 , for example;
• Fig. 3A shows a video frame that has been lowpass filtered by (DC+2AC) ;
• Fig. 3B shows the video frame of Fig. 3A after highpass filtering thereof;
• Fig. 4A shows a sequence of 20 frames from a newsprogram that have been lowpass filtered by (DC+2AC) ;
• Fig. 4B shows the sequence of lowpass filtered frames of Fig. 4A after being highpass filtered
• Fig. 5A shows an unfiltered video image of a frame from a newsprogram
• Fig. 5B shows a lowpass filtered frame of the original video image of Fig. 5A
• Fig. 5C shows a highpass filtered frame of the image of Fig. 5A
• Fig. 6 shows a block diagram of a system for one embodiment of the invention
• FIG. 7 shows a flowchart of the programming steps for a payment subsystem portion of the system of Fig. 6;
• Fig. 8 shows a simplified partial pictorial partial block diagram of the programming steps associated with a filtering subsystem of the system of Fig. 6;
• Fig. 9 is a flowchart for the reconstruction of DC images for one embodiment of the invention.
• Fig. 10 is a flowchart for the reconstruction of (DC+2AC) images for one embodiment of the invention.
• Fig. 11 illustrates a DC+2AC image reconstruction of 2 X 2 block portions of a frame using a 2D inverse discrete cosine transform for each 2 X 2 block.
• spatial filtering of each frame is provided as an alternative means of providing a subset of the original source material to the customer during browsing.
• Spatial filtering provides a mechanism similar to the browsing area of a magazine shop, which provides a space which allows the user to look through all the magazines to determine their content but is not sufficiently comfortable to encourage customers to read the magazines cover-to-cover without buying them.
• Spatial filtering provides a full-motion, full-length version of a video sequence which still protects the intellectual property rights of the copyright holders.
• the present browsing mechanism extracts the filtered video stream directly from Motion JPEG (Joint Photography Expert Group) or MPEG (Motion Picture Expert Group) video streams.
• Highpass and/or lowpass filtering may be used in various embodiments of the invention.
• Reduced lowpass-filtered images called DC images and DC+2AC images were tested by the present inventors, and found to be effective for theft-resistant browsing.
• the small size permits fast computation and fast data transfer, which in turn provides substantial savings in the computational, storage and transmission costs.
• Motion JPEG and the I-frames in MPEG divide the original image into 8 X 8 blocks 5, for example, as shown in Fig. 2A, and apply a two-dimensional Discrete-Cosine Transform (DCT) to each block.
• the pixel values in a block are denoted by f(i,j), 0 ⁇ i,j ⁇ 7, and the DCT terms are denoted by C(i,j), 0 ⁇ i,j ⁇ 7.
• motion compensation is applied to P-frames (predicted frames) and B-frames (bidirectional frames) ; the residue is coded using DCT.
• the DC term c(0,0) is related to the pixel values f(i,j) as follows:
• a DC-image has one pixel per block in the original image, the pixel value being the average of the pixel values of the pixels in the corresponding block.
• a sequence of DC-images is called a DC-sequence.
• (DC+2AC) image 3 of size 80 X 60, and DC image 4 of size 40 X 30, are shown respectively in Figs. 1A, IB and ic, for example.
• the (DC+2AC) image is comprised of blocks each of 2 X 2 pixels. Each block is obtained from the DC term C(0,0), and two AC terms C(1,0) and C(0,1) .
• the DC-image 4 may be thought of as a lowpass filtered, and a subsampled version of the original image 2. Although it is much smaller than the original image 2, the DC image 4 still retains a significant amount of information. However, details have been suppressed in DC image 4. In the example shown in Fig. IC of DC image 4, the words and numbers are no longer discernible, yet the overall information has been preserved. Embodiments of the invention for producing such images are given in detail below.
• the DC image is the direct current equivalent to the average relative value of the 8 X 8 block.
• each 8 X 8 block has sixty-four coefficients, of which one is the DC coefficient, and the remainder of the coefficients are called the AC coefficients.
• Fig. 2B shows an 8 X 8 block portion 5 of the frame of Fig. 2A.
• the DC image portion or coefficient is in the upper left-hand corner, and AC coefficients AC01 and AC10 are adjacent to the DC coefficient, in this example.
• the DC term of a DCT block is a scaled version of the block's average value.
• N 8
• the average value is one-eighth the DC term of the DCT block.
• a DC-image is formed from the collection of scaled DC coefficients.
• the DC-image is reduced eight times in each dimension and may be thought of as a lowpass filtered and subsampled version of the original image. Although it is much smaller than the original image, it still retains significant amount of information. However, details have been suppressed.
• approximation algorithms have been developed to construct images that are very close to the actual DC images. The construction is described below.
• DC images can be constructed directly from a compressed MPEG stream.
• the following example focuses upon P-frames and B- frames.
• P ref be the current block of interest
• P 17 P 2 , P 3 and P 4 be the four original neighboring blocks from which P ref is derived.
• the 2-D DCT of a block P be DCT(P)
• its (i,j) component be (P) s .
• the DC value of DCT(P ref ) as derived by Yeo and Liu in their above-mentioned paper follows below.
• the DCT coefficients of P ref are as follows:
• DCT ⁇ P IBt ⁇ DCT(S i:L ) DCT ⁇ P i ) DCT ⁇ S i2 ) (2 ) i-i
• S-,- are matrices of the form
• the DC value of the block with largest subblock of interest is used. This provides a zero-order approximation. The DC sequence constructed by this approximation is still viable for the purposes of the present invention.
• the DC+2AC reduced image is reduced four times in each dimension.
• an intra-coded block it is formed by taking an 2 X 2 inverse DCT using only the DC and two AC coefficients, c(0,l) and c(l,0).
• approaches similar to first-order approximation used for DC images are used. Only the DC and two AC coefficients and two AC coefficients are used in the anchor frames to reconstruct the DC and two AC coefficients of the present frames. Higher order terms in equation (2) are ignored. In this case, reconstructions are as follows in equations (7) , (8) , and (9) :
• Formation of the DC image is a first step in spatial filtering of the image sequence for browsing.
• the computational complexity is extremely low because only a small portion of data is involved.
• a Sobel operator is used as the high-pass filter on the DC sequences. While other more sophisticated schemes are possible, the Sobel operator is used for its simplicity.
• the Sobel edge detector is defined by two masks ⁇ ! and S 2 :
• FIG. 3A a typical (DC+2AC) frame 6 is shown.
• Fig. 3B the frame 8 illustrates the frame 6 of Fig. 3A after highpass filtering. From the results, one can see that the essence of the video is still preserved for understanding of the image information conveyed, while permitting effective and yet theft- resistant browsing of the materials via the filtered frames 10 of Fig. 3C. Similar comments apply to Figs. 4A and 4B showing lowpass and highpass filtered versions, 10, 12, respectively, of twenty frames of a news program; and to Figs. 5A, 5B, 5C showing original, lowpass and highpass versions 14, 16, 18, respectively, of a newscaster frame.
• the browsing system 20 includes a video monitor 22, a browsing control system 24, a payment system 26, a filtering system 28, and a computer memory device 30.
• the browsing control 24 can be provided by the keyboard of a computer having incorporated therein a mouse or joystick type control, for example.
• the payment system 26 and filtering system 28 can be provided through appropriate programming of an associated computer (not shown for the sake of simplicity) , for example.
• the memory device 30 can be provided, for example, by a disk or other storage device, for storing a collection of video programs in compressed motion JPEG or MPEG formats, for example.
• the software based payment system 26, and filtering system 28 are described below in greater detail relative to the flowcharts of Figs. 7 and 8, respectively.
• the payment system 26 in this example, is a software based system driven by a user operating the browsing control or keyboard 24 in typically a menu driven format.
• the user when first signing onto the system, provides the user's account number as shown in step 28, whereafter the next step is for the system to check the account number against valid account numbers, to determine whether the users account number is valid, as shown in decision step 30. If the account is not valid, the system outputs a signal which displays to the user on monitor 22 the invalid account status, and denies the user access to the video programs.
• step 32 the user is then requested in step 32, via prompts on monitor 22, to select the particular available video program of interest, and also to select the level of quality of the browsing copy to be reviewed of the selected video program.
• step 34 payment information is gathered in an account update step 34, and the user is provided with either or both of a display of the users account on monitor 22, and/or a printout via a printer (not shown for sake of simplicity) .
• an output signal 36 is provided for the index of the selected video program
• another output signal 38 is provided for indicating the index of the selected level of quality for the browsing copy of the selected program 36.
• the index of selected program 36 is used by the computer for retrieving the associated video program from the memory 31 storing the various video databases or available programs, and providing it to filtering system 28, as shown.
• the software based spatial filtering system 40 is responsive to the index of selected quality 38 for adjusting the degree of filtering to the index 38. As shown, depending upon the level of the index of selected quality 38, the spatial filtering system 40 may provide the maximum amount of filtering through use of only the DC coefficient and/or highpass filtering 44, or perhaps a lesser degree of filtering through use of coefficients including (DC+2AC) , all the way to the extreme of no filtering for providing the original resolution.
• step 42 the selected video program is reconstructed through use of the spatial filtering level selected, and the resultant browsing video copy is supplied to the user via display of the browsing material on monitor 22.
• step 44 a decision is made whether highpass filtering of the lowpass filtered video program is to be made. If not, the lowpass filtered video program is passed through to the user for display on monitor 22. If highpass filtering is required, the lowpass filtered video is passed through highpass filter 46 (typically edge detection via a Sobel filter) , and then to the user or monitor 22.
• highpass filter 46 typically edge detection via a Sobel filter
• a flowchart for reconstruction step 42 for DC images includes steps 48, 50, 52, and 54.
• step 48 each new frame is lowpass filtered via the filter selection of step 40 being DC, in this example.
• step 50 a determination is made as to whether the frame is intra coded. If the answer is "yes”, step 54 is entered for extracting DC coefficients from the current frame, and proceeding to step 44. If the answer is "no”, step 52 is entered for extracting DC coefficients from anchor frames, and then proceeding to step 44.
• step 42 An alternative reconstruction step 42 is shown in Fig. 10 for reconstructing [DC+2AC] images.
• step 56 each new frame is lowpass filtered via a filter selection of [DC+2AC] , in this example.
• step 58 a determination is made as to whether the current frame is intra coded. If not, step 60 is entered for extracting DC, AC01, and AC10 coefficients from anchor frames, and then proceeding to step 64 for making a 2D inverse discrete transform for each 2 X 2 block, in this example.
• the resultant [DC+2AC] images are then passed to step 44.
• step 62 is entered for extracting DC, AC01, and AC10 coefficients from the current frame, and followed by step 64 for proceeding as previously indicated.
• step 64 the DC+2AC image reconstruction for a 2 X 2 block in an upper left corner is illustrated.

Landscapes

• Engineering & Computer Science (AREA)
• Business, Economics & Management (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Accounting & Taxation (AREA)
• Finance (AREA)
• Strategic Management (AREA)
• Physics & Mathematics (AREA)
• General Business, Economics & Management (AREA)
• General Physics & Mathematics (AREA)
• Theoretical Computer Science (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)

Applications Claiming Priority (3)

Publications (1)

Family

ID=23739512

Family Applications (1)

Country Status (4)

Families Citing this family (3)

Family Cites Families (1)

• 1996
  • 1996-05-07 EP EP96915545A patent/EP0826196A1/en not_active Withdrawn
  • 1996-05-07 CA CA002219789A patent/CA2219789A1/en not_active Abandoned
  • 1996-05-07 WO PCT/US1996/006403 patent/WO1996036013A1/en not_active Application Discontinuation
  • 1996-05-07 JP JP8534180A patent/JPH11505093A/ja active Pending

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events