HK1069054B - Method for providing digital cinema content based upon audience metrics - Google Patents

Description

Method for forming digital cinema content according to audience measurement standard

Technical Field

The present invention relates generally to digital cinema and, more particularly, to a method and apparatus for forming various digital cinema scene content, which is provided by advertisers and is selectably displayable in accordance with demographics and associated user metrics.

Background

In the conventional model of production and distribution of motion picture films, the entertainment content of the film is invariably and limited due to practical limitations of the film-on-reel (film-on-reel) process. Whatever the content on the disc, any and all viewers are presented, regardless of the composition of the viewers. Typically, different versions of a movie are produced to provide, for example, versions with different normalization rates due to different content (e.g., R is a limit rating, or PG is a recommendation to view under parental guidance). However, the type of this version is strictly fixed; there is no opportunity to adapt the movie content to the viewer even if the modified version is more appropriate.

Demographic metrics have been used as a basis for entertainment, advertising, and the delivery of promotional material to viewers. In various embodiments, U.S. patent No. 5758257 (invented by Herz et al) discloses a method of processing a material using profile data of a customer so that the material can be enjoyed by a cable television customer; U.S. patent application publication 2002/0156677A1 (by Pdters et al) discloses a method and system for advertising in public spaces based on demographic and preference information obtained from digital devices such as Personal Digital Assistants (PDAs) carried by similar individuals; U.S. patent No. 5155591 (by Wachob) discloses a method of demographic-dependent commercial advertising to television viewers based on demographic and preference profile data for each television viewer location; and us patent No. 6128663 (by Thomas) discloses a method of providing specified internet content to a requester based on demographic and preference information.

Demographic metrics have also been used by film producers and distributors to derive raw feedback data on the responses of the entire audience and the relative degree of success or failure of a film to business. However, demographic audience data has not heretofore been used to tailor the content of a movie to a particular audience, nor has other commercially available demographic data been used to assimilate that audience viewing a presentation at a particular viewing location.

With the advent of digital cinema, particular attention has been given to potential technical advantages that can provide more flexible projection arrangements and the possibility of color improvement and other image improvements. In addition to these technical and performance advantages, compelling advantages and possibilities are shown for more economical movie distribution, for improved tuning to market and audience demographics, for local customization of some types of content, and for overall improvements in a more entertaining movie theater experience. Due to these advantages, it is possible to increase the financial income of movie production companies that produce and distribute movie contents and movie theaters that show the contents to viewers.

The traditional model of motion picture film distribution is characterized by a strict control of the entire distribution and presentation process, which allows local adjustment with minimal or no flexibility. Referring to fig. 1, a block diagram of a conventional motion picture film distribution system 10 is shown. The movie production company 20 obtains content, and in particular individual content movies 30, from content providers 22 that produce film parts, commercials, movie trailers, previews, and other content for theater presentation. The film production company 20 clips, controls and prepares film prints 24 and provides these film prints to a movie theater 28 via a distribution network 26. In the conventional mode shown in fig. 1, the movie production company 20 decides the contents to be shown at each movie theater 28. Such control to determine the content of the presentation is not only relevant to the film portion itself, but also to the commercials or trailers, such as predictions of future boxes and the like. In a conventional arrangement, the theater 28 shows the movie portion and other related content according to the instructions of the film production company 20. Typically, in practice, the manager or owner of the theater 28 is required to combine the movie trailer (including advertising and other content) with the main movie portion in order to show the movie according to the specific requirements of the movie production company 20. Due to the rather inflexible arrangement shown in fig. 1, there is no opportunity to dynamically add or change image content.

The major revenue for the movie production company 20 is derived from the advertising and promotional material. Advertisers pay a large fee for the production of a movie in order to have their products appear in the movie. For example, car manufacturers pay a premium to have their cars as part of the set. Similarly, soft drink manufacturers pay for their brands, vending machines, or billboard advertisements to appear in the background, or have actors wear T-shirts with their group identification. Any such "hidden" advertising revenue is owned by the motion picture production company 20 using conventional, film-based print preparation and distribution methods. Theater 28 does not benefit from such advertising revenue. Of course, for some national and international advertising campaigns, advertisers benefit by publishing movies domestically and worldwide. However, there are often scenarios where advertising or promotional content is being filmed for local merchants, where the product or service is known only in certain areas. For example, the scene background may take a picture of an exit from a grocery store, an appliance store, a coin-operated telephone, a car gas station, or an exit from a product that is only partially present. With this content, the potential revenue from the re-sale of the promotional space is not shunted using the conventional film-based print and distribution method shown in fig. 1.

In summary, it can be seen that with the conventional film-based printing and distribution method shown in fig. 1, only the film production company 20 controls the content of the film prints 24 distributed to the movie theater 28. Theater 28 has no opportunity to share promotional revenue. In a similar situation, there is no opportunity to augment other types of local content, including content not specifically targeted for advertising or promotion, using conventional film-based print and distribution methods. For example, including audience members in the projected movie itself may have entertainment value, thereby enhancing the overall entertainment experience. As just one example, a crowd scene in a movie may allow for additional images that may be added using audience members. This capability can be used to increase viewer interest and reaction to the movie.

One of the potential advantages of digital cinema is how images are stored and displayed by the data. Referring to fig. 2, there is shown a block diagram of a digital cinema production and distribution system 100 for providing a cinema image from a motion picture production company 110 to a post-production facility 111, which digitizes the cinema image and provides the digitized image to a projection system 140, typically a movie theater, via a transport system 130. (the post-production facility 111 may be under the control of the film-producing company 110, or may (typically will) exist as a separate entity throughout the system 100.) at the post-production facility 111, the film 112 is digitally controlled, the film including image content, such as film portions, commercials, trailers, and the like. The data cinema system 114 converts the film content into digital image content and provides the digital image content to a presentation system 118, which is typically supported by a disk array 120, with presentation resolution suitable for the film image data being distributed and presented. Presentation system 118 may also receive input from auxiliary input devices 116 such as data tapes, DS tapes, and data Cam devices. The entire processed digital cinema output is then provided as a data stream to the transmission system 130, which may use a satellite 138 in communication with the transmitter 122 connected to the post-production facility 111. Other alternative transmission media include optical cable connectors 136, or transmission using optical media 134 such as DVDs or optical disks, or transmission using magnetic media 132 such as data tapes. Digital image data of the master is received at projection system 140, for example, by receiver 148, optical media reader 144, magnetic media reader 142, or by data or fiber optic cable connector 136. A movie operating system 146, typically supported by disk array 120, accepts digital input data, processes the input data stream for projection, and provides the data for forming images and projection by one or more digital projectors 150.

Various methods have been developed for the overlay of images in digital image processing. For example, U.S. patent No. 6243104 (by Murray) discloses an overlay technique for delivering pictorial content as digital data provided over the internet. U.S. patent nos. 6069637 (invented by Gaglione et al) and 6362816 (invented by Kawanami et al) disclose techniques for combining different types of images, including images that may have different resolutions. In television broadcasting, the two-dimensional blue screen method has been widely used to superimpose moving images one on top of another. Weather forecasting is the best embodiment to apply these methods. The meteorologist appears in front of the dynamically changing map; in fact, the meteorologist stands in front of the blue screen; the weather image is recorded by a separate camera and superimposed on the unblocked blue pixel.

More advanced three-dimensional methods and techniques have been developed and used to place advertising and promotional content in video data streams for television broadcasting. Similar embodiments are found in background advertising during a sporting event. In a baseball game, for example, a play scene viewed from the pitcher's pedal shows a billboard behind the batter. The billboard is added into a broadcasting studio; in fact, the pitcher only sees the dark wall behind the batter because the bright white board distracts the pitcher. Advanced techniques for integrating this type of advertising material have been developed so that electronically composed advertisements can be naturally presented to television viewers. For example, U.S. patent No. 5264933 (issued to Rosser et al) discloses a technique for inserting logos in a video clip that is broadcast. Us patent No. 5353392 (invented by Luquet et al) discloses a method for identifying a target area in successive image frames for replacement with advertising or other material, including a method for adjusting the camera focal length and elevation and azimuth changes and a method for allowing the target to block a portion of the target area. U.S. patent No. 6381362 (by deshopande et al) discloses a virtual replacement display area for providing inserted advertisements in a video presentation. U.S. patent No. 5731846 (invented by Kreitman et al) discloses a method for image conversion of combined video images. Us patent No. 6184937 (invented by Williams et al) discloses a method of image insertion with good quality audio.

U.S. patent application publication No. US2002/010042a1 (by khoo et al) discloses a method and system for product replacement and advertisement display in a digital data stream broadcast to individual television viewers. In the arrangement disclosed at 2002/0100042, the studio can replace the advertisements inserted into the broadcast movie based on the acquired individual demographic data of the individual viewer. To provide alternative image content, the image processing means identifies an area that can be cropped and inserts scaled advertising images into the area, particularly replacing one product for another based on individual viewer demographic data. 2002/0100042, to form a plurality of different movie presentations (each showing specifying a particular viewer demographic profile), or to form individual personalized movie presentations "on the air" according to demographic metrics. 2002/0100042 can be used to generate alternative advertising content, however, it has the disadvantage of limiting the usability and potential effectiveness of this feature. One drawback relates to the level of control of the studio when using the method; no local or regional modification for advertising is possible with alternative scene content. Moreover, since the replacement of scene content can only be done at the studio, the method disclosed in 2002/0100042 does not provide metrics by which the advertiser can adequately evaluate the value of the replaced scene content. For example, a brief glance at the billboard during a chase scene may be less valuable to advertisers than a productive display appearing in the background during a separate talk scene. Yet another disadvantage of the method disclosed in 2002/0100042 relates to the visual characteristics of the scene, i.e., the overall "look and feel" of the scene. Any alternative scene content must be adapted to the visual characteristics of many scenes, such as camera view angle, brightness and focus. The replacement scene content may be in the background or foreground of the scene and may not appear very realistic if set under ideal focus conditions. Thus, the solution disclosed by 2002/0100042 is less suitable for allowing insertion of replacement scene content at a local or regional level, or for allowing the replacement scene content to be aesthetically tailored to the visual characteristics of the scene, due to its tight control of content, limited availability of information for advertisers and those who synthesize the replacement content into the scene itself, and limited flexibility of use outside the studio environment.

Thus, while there are technical tools for altering the movie image based on the viewer's demographics to add content to successive image frame portions, this type of image processing is typically only performed in the originating television studio or display facility. There is no way to use these techniques to locally modify a motion picture image; movie theaters cannot take advantage of the promotional opportunities for local and regional advertising by inserting image content. Advertisers have limited information available to evaluate the potential effectiveness of replaceable scene content. For example, in addition to advertisements, movie theaters cannot add types of content that enhance the entertainment level of a movie fan's experience, such as adding images of audience members.

In addition to the commercials and promotional content in the movies, there may also be "movie trailers" for promotion, which include commercials that are played at intervals between shows. In the traditional film-based model, these movie trailers are spliced with movie films at the movie theater following the instructions of the movie production company and distributor. Other image content used in a movie environment includes advertisements that are slideshows between presentations. In conventional mode, the same advertising or promotional content is shown regardless of when the film is shown, regardless of the viewer demographics. It will be appreciated that it would be valuable to be able to alter the material of a movie trailer or slide show to suit the composition of the audience.

In the traditional film environment, some promotional and advertising revenues are owned by film production companies for displaying product or group logos in a film, or for showing commercials in the middle of a show. However, the local exhibitor of the film has no opportunity to profit; the content of the presentation, including any benefits derived from product placement or other advertising, is controlled by the film production company and distributor. In contrast, in television broadcasts, specific advertisement slots are allocated for local commercial sales. However, the movie presentation is not interrupted for commercial purposes and the local exhibitor is not able to obtain any promotional revenue opportunity from the movie content or movie trailer.

It is well known that advertising is most effective when targeted according to an ideal demographic profile. Even though the same product is often displayed differently when being promoted to different audiences. Thus, for example, very identical breakfast cereals may be touted as healthy, fiber-rich, low-fat, and nutritious to promote to those audiences who pay attention to them, while promoting to the rest of the audience may tout it as interesting, colorful, sweet, and pleasant crunchy. In television advertising, the demographic profile of the viewer is inferred from assumptions about the effectiveness of the relevant display content and statistics derived from viewer surveys. These demographics are then used, perhaps to varying degrees, to schedule and purchase advertising slots for profile (profiled) group viewers. However, in a cinema setting, living audiences are sitting in their seats and thus the demographic data is more reliable. The same movie may be shown to people with different demographics depending on the time of day and other factors. It would be of significant value to advertisers to be able to convey their commercials to such viewers who are prone to profiling, particularly since movie viewers generally do not have the level of inattention that is the case with home viewers to which advertisers are confronted.

Thus, it can be seen that movie theater operators would benefit from the ability to add and change specific content of the movie and associated display content for advertising, promotion, and entertainment improvement, particularly when such modifications are based on demographic information obtained from the audience. The local application of effective space for promotional content based on audience metrics can help movie exhibitors with advertising revenue that is not possible when applying conventional movie-based exhibition techniques.

Disclosure of Invention

The present invention is directed to overcoming one or more of the problems set forth above. Briefly, in accordance with one aspect of the invention, the invention provides a method for screening a motion picture received as a digital data stream having a plurality of original scene sequences and having selectable scene content in or from one of the original scene sequences. The method comprises the following steps: (a) acquiring data on each of a plurality of viewers when a movie is shown; (b) receiving digital image data of an alternate sequence of scene content for one of the original sequences of scenes in the motion picture; (c) substituting said alternate sequence of scene content into said digital data stream for one of said original sequences of scenes at said presentation based on said data relating to said viewer, thereby altering selectable scene content for presentation based on said viewer related data; and (d) charging a fee for presenting the replacement sequence of scene content.

In accordance with another aspect of the present invention, a method is provided for synthesizing advertising promotional content of an advertiser into a digital cinema presentation for an audience. The method according to this aspect comprises the steps of: (a) receiving a movie digital data stream comprising at least one variable scene sequence, a first scene content replacement sequence and a second scene content replacement sequence, wherein the first or the second scene content replacement sequence is inserted into the movie digital data stream as part of the variable scene sequence; (b) acquiring data related to audiences when the digital film is shown; (c) receiving a fee from an advertiser for showing either the first scene content replacement sequence or the second scene content replacement sequence in the at least one variable scene sequence; and (d) inserting either said first scene content replacement sequence or said second scene content replacement sequence into said movie digital data stream in said at least one variable scene sequence, the selection of said first or second scene content replacement sequence being dependent on said obtained viewer-related data.

According to another aspect of the present invention, the present invention provides a method for inserting promotional content of an advertiser into a digital movie for presentation to an audience. The method according to this aspect comprises the steps of: (a) identifying at least one replaceable promotional scene sequence in the digital cinema data stream; (b) charging a fee from the advertiser to purchase the at least one sequence of replaceable promotional scenes for airing while showing the digital movie; (c) receiving a digital data stream of a movie comprising said at least one replaceable promotional scene sequence and comprising at least one scene content replacement sequence; (d) acquiring data related to audiences when the digital film is shown; and (e) inserting said at least one scene content replacement sequence into said movie digital data stream for playout based on said viewer-related data.

It is a feature of the present invention to provide the opportunity to change the scene content in a particular scene of a digital cinema presentation in one or more presentations.

An advantage of the present invention is the ability to adjust promotional material in a motion picture presentation based on the demographics and preferences of the audience.

An advantage of the present invention is that it enables the owner of a movie theater to benefit from the opportunity to market when showing a movie. Whereas previously only film production companies controlled the use of promotional content and gained revenue. In addition to image content, related audio content may also be provided with the inserted image content.

Another advantage of the present invention is the ability to enable local movie theater owners or other exhibitors to dynamically add images of people to selected portions of the movie being shown, thereby enhancing the entertainment and participation of certain audiences and providing local advertisers with the opportunity to display their advertisements.

These and other objects, features and advantages of the present invention will become more apparent to those skilled in the art from the following detailed description, which, taken in conjunction with the annexed drawings, shows and describes an embodiment of the present invention.

Drawings

Fig. 1 is a schematic block diagram showing a conventional system for producing and distributing a motion picture film.

Fig. 2 is a schematic block diagram illustrating a mastering (mastering) and digital cinema data providing system, such as may be used in practicing the present invention.

Figures 3a-3d show a sequential process of identifying a variable content display area, separating the area, providing replacement content inserted into the area, and inserting the replacement content according to metadata associated with a digital cinema image data stream.

Fig. 4 is a schematic diagram showing a relationship between key parts for providing alternative contents in a digital cinema using a chromakeying technique.

Figure 5 is a schematic diagram illustrating one arrangement for obtaining demographic data relating to an audience.

Fig. 6 is a schematic diagram showing how an image data stream of a digital cinema can be constructed from a sequence of alternate scenes.

Fig. 7 is a flow chart showing the processing steps for obtaining alternate content in the preferred embodiment.

FIG. 8 is a flowchart showing process steps for obtaining and applying alternative content provided by an advertiser.

Fig. 9 is a flowchart showing processing steps for acquiring and applying alternative content provided by a movie production company.

Fig. 10 is a flowchart showing a process flow for acquiring and applying alternative audio content.

Detailed Description

The present description will be directed in particular to elements forming part of, or forming part of, the apparatus in accordance with the present invention. It is to be understood that elements not specifically shown or described may take various forms well known to those skilled in the art.

Although the preferred embodiment of the present invention is primarily directed to the use of demographic factors such as age and gender, other types of audience metrics can be used to affect the scene content in a movie. Relevant audience metrics may include, for example, the number of attendees, measured or perceived behavioral responses, the volume of sound, information based on reported or perceived likes or dislikes, or purchasing behavior such as purchasing cola, popcorn, and others. Even if the expected transaction amount varies due to the time schedule of the showing or the moment of the showing at the same time, other factors may be widely considered as types of audience metrics for influencing the movie content, as described below.

Obtaining audience metrics

Demographic and other relevant data relating to the audience may be obtained in a variety of ways. Referring to fig. 5, components of a projection system 140 for collecting and making demographic and presence data related to audience members 80 are shown. At a demographic data entry point 70, such as when ticketing or checking tickets, an employee 72 enters information about audience members 80 into a console 74. The database 76 stores and makes available audience metric data to the movie operating system 146, which also receives digital camera data 82 from the transmission system 130, as described in the central preceding background section. The movie operating system 146 adjusts the digital movie content projected by the projector 150 based on demographic data and other audience metrics.

The demographic data typically includes age and gender information of the audience members 80, and may include additional information. There are a variety of other methods that may be used to obtain demographic data. Theater employee 72 would collect and report this data using manual or automated methods. For example, an electronic camera 84 may be used in conjunction with pattern recognition software to obtain information about each audience member 80 when approaching or sitting. For example, the audience members 80 themselves may be invited to enter demographic or preference information on the portable terminal, such as at ticket sale, at entry, or while seated.

Regardless of how the viewer demographic or other viewer metric data is obtained, this data may be used as an alternative sequence for selecting and adjusting scene content. The sequence of scene content being adjusted may be image content in image frames or may be a sequence of frames of an event, as described below.

Modifying portions of scene content in a frame

Referring to fig. 3a, 3b, 3c and 3d, an exemplary sequence is shown illustrating how the present invention defines and processes variable content display regions 204 according to audience metrics for an image frame 200. In a series of movie image frames 200, the sequence starts with the original shot scene shown in fig. 3a, with an alternative image component 202. For example, in a preferred embodiment, the alternative image component 202 includes some form of advertising, such as advertising on a billboard, bus or other vehicle, indicia in or on a shop window, logos on a kiosk, or some other identifiable component that appears in the background scene content. As shown in fig. 3b, the variable content display area 204 has a shape and size defined by a plurality of vertices 206 forming boundary coordinates. Given the given information, including the shape, size and location of the variable content display area 204, the alternate content 208 may be pre-formed, as shown in the embodiment shown in fig. 3 c. The alternate content 208 is then adjusted to the shape and other characteristics of the variable content display area 204 to provide alternate image content 202 'in the modified image frame 200'.

The ability to replace prepared replaceable image content 202' as shown in fig. 3d relates to a feature of digital imaging whereby the image frame 200 is displayed as a two-dimensional matrix of pixels, as is well known to those skilled in the art of digital imaging. Each pixel has one or more corresponding data values that determine how the pixel is to be displayed. Referring again to fig. 2, for the presentation of the modified image frame 200 in fig. 3d, digital pixel data of variable alternative image content 202' is inserted into the digital data stream of the digital projector 150. Note that the pixels in the frame 200 that are located on or near the boundary of the variable content display area 204 may be adjusted in some manner to improve the video conversion between the originally captured content and the alternate content 208. Such adjustment, for example, when switching between pixels originally captured and pixels containing the replacement content, may be performed by blurring or feathering. Preferably, although not necessarily, this is done by modifying the pixels adjacent to, but not within, the variable content display region.

Metadata for variable content display area 204

Referring to fig. 2 and the sequence of fig. 3a-3d together, the present invention provides a method for modifying image frames 200 transmitted from post-production facility 111 as raw digital data at projection system 140.

In order to be able to modify the image frame 200, the motion picture production company 110 must provide sufficient metadata with the digital image data stream. As an absolute minimum, this metadata must include information about the replaceable image content 202 that the frame 200 includes, as well as information about the shape, size, and location of the variable content display region 204 in each frame 200. At a minimum, this type of data gives "native" information identifying which pixels of the image frame 200 may be replaced in the corresponding image data stream. Alternatively, a mask or selection channel may be provided that specifies the replaceable area. However, as will be appreciated by those skilled in the visual arts, merely identifying the location of the variable content display area 204 itself may not give sufficient information for a reliable, visually appropriate replacement of the variable alternative image content 202'. That is, additional information related to the visual characteristics of the scene, such as camera angle, focal length, brightness conditions, image size, etc., must be taken into account in order to appropriately adjust the image data to form a visually acceptable modified image frame 200'. The focus information also helps to determine the appearance of the changeable alternative image content 202' relative to other components of the image frame 200. Also, both the size and viewing factors may be easily changed from one image frame 200 to the next, at least in part.

Referring to table 1, there are shown several brief, simple embodiments of metadata with identified significant metadata fields provided for different image frames 200. The key metadata would include the following:

(a) the frame id number. As described above, each frame 200 has a frame number marked in sequence starting from the first frame of the movie;

(b) boundary coordinates. As shown in fig. 3b, these identified vertices 206 of the variable content display area 204 are typically groups of pixels, and using the familiar cartesian coordinate model, some points within or along the boundary of the frame 200 are typically labeled as starting points (origin) or (0, 0) locations;

(c) camera focus setting and camera angle data. These parameters are necessary to adjust the alternate content 208 in order to obtain the proper viewing angle and perspective. Applying image data conversion can typically compensate for failures to provide an appropriate viewing angle when obtaining the replacement content 208. For example, a change to specify the rectangular shape of the insert, such as a billboard, so that the final appearance matches the perspective taken by the camera, as indicated in U.S. patent No. 6075542, "method of compositing two digital images" (invented by Fredlund et al), which is incorporated herein by reference;

(d) brightness and contrast data. This data may include information regarding overall scene brightness conditions necessary to adjust the alternate content 208 to fit the image frame 200 in a natural display. The creator of the movie may specify defining brightness, contrast, saturation or other image parameters that help make the inserted pixels have a natural appearance; and

(e) if necessary, other descriptive information. For example, if the scene environmental conditions include reflections, rain, snow or fog, complexity may be increased. Depending on the location of the variable content display area 204 within the frame 200, some compensation for projection light loss may be required. The focus information may be important for determining image sharpness and other parameters.

The contents of Table 1 are given for illustrative purposes only; alternative and/or additional metadata fields may be provided for more accurately determining the viewing conditions of the scene. For example, the data format of the metadata may be a simple, open data format, such as using the familiar comma-separated fields. More complex encoding schemes may be used, such as those employing compression, security algorithms, one of which is the need for decoding or other existing data conversion mechanisms.

Metadata relating to each image frame 200, labeled non-image metadata, may be transmitted with the image data of the image frame 200, but formed within the image data stream or for the same transmission period of the image data stream. Alternatively, the metadata may be formed on a separate medium or as a file that is transferred along with the digital camera imaging data. Alternatively, a separate channel may be provided to transmit the metadata at a separate time or in synchronization with the transmission of the image data itself.

In practice, the default content for any identified variable content display area 204, which serves as the alternative image content 202, is provided by the movie production company 110. Thus, the operator of the projection system 140 has the alternative image content 202 'replaced with variable alternative image content 202' or maintains the option of the original alternative image content 202. In a movie theater environment, for example, there may be situations where it may or may not be appropriate or advantageous to provide such a replacement, for example, based on audience demographics.

It must also be noted that the metadata provided for the alternate scene content also provides metrics useful to advertisers and those selling potential advertising space. In evaluating the potential value of an opportunity to use the alternative image content 202 for placement of a product, for example, an advertiser may make a decision based on metadata regarding duration, focus, and particular sequence of light conditions in a movie.

Audience metrics may also be used to more efficiently find the alternate content 208. There may be many available candidate images that are selected based on viewer demographics and other viewer metrics as the alternative content 208. For viewers composed primarily of children less than ten years of age, for example, the particular alternative content 208 may be directed to the interests of a younger viewer. For viewers composed primarily of adults, other alternative content 208 for this older population may be used. In this manner, demographic data and other audience metric criteria may be used to determine the content of the image frame 200' modified in any one presentation.

Table 1 metadata example

Frame # (start, end)	Boundary coordinates	Characteristics of scene	Variable content
				000,423	(137，144)(658，147)(661，398)(140，387)	Camera focus setting: 16 feet distal azimuth: +4.5 degrees from normal: -11.23 degree brightness: grade of sunlight: 88 focal point: jennifer with sunglasses	The following steps are described: milk carrier advertisement size: 48H X64W inches
23433，23507	(66，215)(312，228)(301，396)(80，378)	A camera: pan, arc distance (558, 232)13.2 feet azimuth: +23.8 degrees from normal: +22.3 degree luminance: no dust, grade: 62 focal point: jack, walking in a hurry	The following steps are described: display size of the storefront: 35H X116.7W inch
				6727727，6745421	(2341，816)(2215，145)......	Camera focus setting: 46.3 feet azimuth: +15.4 degrees from normal: -23 degree brightness: rain night grade: 21 focus: remote urban landscape	The following steps are described: flag size of taxi: 12H X55W inch

Other selections of alternative image content 202

In the embodiment shown in fig. 3a, the alternative image component 202 includes advertising as part of the background scene content. However, in the broadest understanding, the alternative image component 202 comprises any portion of the image frame 200. Using the method of the present invention, any background component can be identified as a replaceable image component 202 that meets the conditions for replacement with the appropriate metadata provided. Foreground components and characters may even be added.

Referring to fig. 4, an arrangement of components for obtaining alternate content 208 that can add an image of a person or object to an image frame 200 is shown. The subject 160 in front of the blue screen 162 is captured by an electronic camera 164 and the resulting digital image is input into the movie operating system 146. According to the program instructions, the movie operating system 146 combines the digital image data with the image data of the digital movie acquired through the delivery system 130 as described above. The movie operating system 146 then transmits the digital image data stream to the digital projector 150 for projection of the modified image frame 200'.

Chroma-keying or "blue-screen" techniques are known for image overlay in television distribution. Using a conventional chroma keying method, an authoritative meteorologist points to a background map or a moving display; however, in a film studio, the meteorologist is actually pointing at an empty blue screen, and hopefully the individual shooting background display studio personnel correctly combine the images and form a reliable display.

According to the arrangement of fig. 4, an image 160' of the subject 160 may be captured for projection in a movie. Image 160' is inserted as replacement content 208 into image frame 200 via chroma keying techniques. Using chroma keying, the movie operating system 146 determines whether blue pixels or pixels that are part of the image 160 ' are detected from the blue screen 162 and reacts accordingly to dynamically separate the image 160 ' of the subject 160 so that the image 160 ' can be provided as the alternate content 208. With this effect, for example, audience members that were filmed prior to the show may "appear" in the appropriate scenes of the movie, such as the crowd scenes or various types of adventure or comedy scenes. For this purpose, the electronic camera 164 and blue screen 16 may be located, for example, in a restroom of a movie theater or other location suitable for capturing images of audience members.

Applying the technique shown in fig. 4 to insert an object or person into a motion picture image frame 200 may require different metadata than the alternative embodiment described above with reference to fig. 3a-3d and table 1. For example, instead of an exact description of the geometric characteristics of the variable content display area 204, a more general description of the background and targeting information may be provided, for example, using the techniques disclosed in U.S. patent No. 5867282 (Fredlund et al), which is incorporated herein by reference. Some camera angle, focal length, and brightness data are also needed. The additional metadata would represent all types of scenes and preferred subject types (e.g., age or gender), actions or gestures, and other information that would help to best fit the image 160' to the scene.

As previously mentioned, it may be beneficial to select one out of a number of available sequences of preliminary alternative image content 202', for example, when suggested by the viewer demographics. For advertising content, for example, a first particular sequence of alternative image content 202 'may be most appropriate for viewers less than the decade of life, while a second particular sequence of alternative image content 202' may be most appropriate for an older audience population.

Application of alternate sequences of scene content

In addition to serving as a basis for selecting alternate content in the image frames 200, audience metrics may also serve as a basis for selecting one or more frame sequences, where each frame sequence is a set of consecutive frames that provide a scene or some temporary sequence of scenes for a movie. For example, a particular scene or portion of a scene may be deleted or shortened for a teenager viewer, or, conversely, the scene may be expanded for a younger viewer and shortened for a older viewer. Different versions of the same scene can be easily replaced in a movie theater using digital cinema technology, which was previously extremely difficult with traditional film screening.

Referring to the schematic block diagram of fig. 6, there is shown an arrangement of the present invention by which the movie operating system 146 combines the image display data stream 180 with alternative scene content portions selected based on the viewer demographics of the database 76 or based on other factors. With this capability, the movie production company 110 provides a composite movie 170 that includes multiple image data streams. In the simplified embodiment shown in fig. 6, the image data flow is as follows:

(a) base content 172, including one or more scenes that do not change during the presentation of different scenes;

(b) selectable scenes 174a1 and 174a2, one and only one selectable scene 174a1 or 174a2 being included in the display image data stream 180; and

(c) selectable outcomes 176z1, 176z2, and 176z3, one and only one of which 176z1, 176z2, and 176z3 are included in the display image data stream 180.

In practice, one of the selectable scenes 174a1 or 174a2 is a default value, set in the original image data stream of the entire movie. Similarly, one of the outcomes 176z1, 176z2, and 176z3 may be selected as a default value. Thus, if no decision is made based on audience metrics, the default original scene is used in the digital image data stream.

For the arrangement of fig. 6, the movie production company 110 provides all the components of the composite movie 170. In addition, the movie production company 110 also provides a description of one of the selectable scenes 174a1 and 174a2 and the selectable outcomes 176z1, 176z2, and 176z 3. In a preferred embodiment, the selectable scene metadata 182 has the information in table 2, including scene delineation, coding demographics, time, and other information. The movie presenter may then use the selectable scene metadata 182 to determine which scene clips to present based on audience metrics and other factors. Preferably, the program controlled movie operating system 146 makes the decision based on audience metrics and related variables. If this determination is made, then the selected material may be embedded in the image display data stream 180 using image processing logic on the movie operating system 146.

Using the arrangement shown in fig. 6, a screener may provide multiple different image display data streams 180 for the same portion of the movie, depending on such other factors as the number of attendees, traffic conditions, audience demographics of audience behavior, or like time of day, season, weather conditions, or popcorn inventory. For example, during busy hours, it may be beneficial to play a selectable outcome 176z1 that is somewhat shorter than selectable outcome 176z 3. It should also be noted that the image display data stream 180 may be dynamically combined based on previous decisions or even decisions made during the presentation. For example, if the demographic database 76 is accessible from a remote location, the image display data stream 180 may be assembled at a remote server by the film production company 110 or by a film distributor.

The same method used to specify selectable scenes 174a1, 174a2 or selectable outcomes 176z1, 176z2 or 176z3 can also be used to specify advertisements or promotional trailers or slide show content that is part of the image display data stream 180. That is, the selectable content may be part of the movie itself, or may be some other component, such as a trailer.

Table 2 example of scene metadata 182

Serial number	Scene	Selectable scene segments	Duration (seconds)	Grade	Demographic ratings
						28	Storm on the sea	Attack of blood in sharks	114	PG	14000010075
28	Storm on the sea	The shark attacks and causes a little blood	103	G	040000306f3
						28	Storm on the sea	Attack by sharks, suggested	64	G	04000040782
28	Storm on the sea	Attack by sharks accompanied by lively chewing	132	R	2400007663a
						187	Maritime staff conversation	Report by Barnacle Bill, vivid	32	R	340000356ea
187	Maritime staff conversation	Barnacle Bill, Mild	30	PG	07000070089

Arrangement for resale of advertising and promotional spaces

It is clear that the method of the present invention provides a profit opportunity for movie theater owners and other film exhibitors. The sale of advertising space in a film being shown, which is traditionally controlled only by the film production company 110, can now benefit film exhibitors. To apply this new capability, a movie has a content tracking list that includes information about where in the movie the replacement content 208 is used. By way of example, table 3 shows a content tracking list with a small number of expected information data fields.

The sequence number ID indicates a sequence of scenes in the movie. Other information provides sufficient data for forming an opinion of what type of advertising or promotional space is available, for indicating the duration of the space to be available, for listing potential or actual customers of the space, and for specifying the cost of the space to be available per unit time. With this information, the theater owner or local exhibitor may then solicit local customers to present them with the opportunity to provide alternate content 208 for one or more segments in the movie. The capture of the alternate content 208 would typically be performed by an external agent, specifying available scene sequence information (as shown in the embodiment of table 3) and metadata (as shown in the embodiment of table 1).

Table 3 content tracking examples

Serial number	Scene	Target	Duration of time	Customer	Cost/second
						1	Daytime	Grocery store sign	30 seconds	GiantBig Eagle	$300.
33	Dust-free	Canopy of gas station	120 seconds	Mak’sSunoilco	$280.
						42	Dust-free	Small restaurant floor top mark	8 seconds	Joe’s Diner	$240.
353	At night, sunny day	Automobile franchise	12 seconds	Loopy’sStudebaker	$280.

Referring to fig. 7, a flow chart of steps for obtaining alternate scene content in a preferred embodiment is shown. In an initial delivery step 242, the movie production company 110 provides the exhibitor 240 with a digital movie as a digital movie image data stream. (the intermediate role of the post-production facility 111 has been removed from the description in order to simplify the description in connection with fig. 7-10.) additionally, the movie production company 110 also provides information on alternative scene content as metadata, whether of the variable content display area 204 as shown in the embodiment in fig. 7 or of the complete scene as described later. In a sell provision step 244, exhibitor 240 sells available alternative scene content to advertiser 260. The advertiser 260 purchases promotional space at a purchase step 248. Upon sale to the advertiser 260, the exhibitor 240 defers delivery of the cost of using the space to the film production company 110 in a purchase step 246. In the implementation step 250, the movie production company 110 provides the exhibitor 240 with implementation information, such as coding, for the space and any additional metadata needed, for example.

Following the flow in FIG. 7, in a designation step 252, the exhibitor 240 provides the metadata information to the advertiser 260. The advertiser 260 then requests that promotional image content be produced. Next, in a content formation step 254, advertiser 260 provides appropriately authored content for use in the digital cinema image data stream. The exhibitor 240 then inserts the alternate content, whether it be within the frame 200 as the alternate content 208 or as a complete sequence of frames of the alternate scene content, as described below.

The transaction sequence of fig. 7 allows for any number of modifications within the scope of the present invention. For example, the movie production company 110 may simply provide the metadata, as well as any necessary decoding information, as part of its sale, rental, or movie distribution. Alternatively, the exhibitor 240 may be required to pay for the use of showing any locally sold promotional content. For example, the movie production company 110 may provide digital movie content, metadata, encoding capable of decryption, and any other data transmitted from a network server or using any other suitable data transmission medium (e.g., optical disk or tape media).

Exhibitor 240 may enter into any number of various possible agreements with advertiser 260. In particular, the audience metric data is used to determine what, if any, alternative scene content to use. For example, the advertiser 260 may insist that specific promotional content is incorporated into the movie digital image data stream only when there are at least 200 males between 16 and 44 in the audience. Billing settings may be negotiated to adjust the pricing of added promotional content based on audience metrics such as size or demographics. Advertisers 260 can bid for the same sequence of scenes so that different promotional content can be seen in different showings, including the ability to determine which promotional content to show based on audience metrics. The same data network or some other network connection used to transport the movie may be used as a back channel (back channel) to feed payment confirmation or audience metric data back to the movie production company 110 or advertiser 260. In this broadest scenario, advertiser 260 can be a company, individual, organization, or any organization or representative that is trusted to offer promotional displays in return for a reward, which produces products for sale or provides certain services.

To date, other options that apply audience metrics, scales, can also be used, such that the cost of an advertiser 260 depends on the number of people in the audience who meet certain demographic metrics. Thus, for example, if the target audience is female between the ages of 16 and 35, the cost of promotional content assessed to the advertiser 260 will vary depending on the number of viewers meeting these metrics.

Other embodiments

The actual playout of the digital cinema display data with the inserted replacement content 208 and any accompanying audio content is done dynamically in "real-time" as the movie is being played back using the method of the present invention. Alternatively, the playout may be performed in advance, and the played-out version temporarily stored, for example, on disk array 120 for later viewing, as shown in fig. 2.

It must be noted that the movie production company 110 may specify any number of sequences of frames 200 that can accept the replacement content 208. The exhibitor 240 or theater owner may choose to modify some or none of the sequence of frames 200. The encoding allows the motion picture production company 110 to decide whether to modify the content of the frame 200 based on consideration received from the owner or operator of the projection system 140.

For any of the variable content display areas 204, the exhibitor or theater owner may select among a set of possible candidates for the alternate content 208. Thus, for example, different advertisers may have their promotional material appear differently at different shows. The advertiser 260 can use alternative scenarios to promote the same product in different ways depending on audience metrics, such as emphasizing the trophism of breakfast cereals (e.g., if the audience age may be 50 or higher) or the enjoyment, sweetness of the same breakfast cereals (if the audience metrics indicate the age of younger audiences), as described above in the background section.

A default image may be provided as the alternate scene content 208 for use when the exhibitor 240 does not sell one or more of the variable content display areas 204.

Transaction flow for selectable content

Referring to fig. 8, a flow diagram is shown for using alternate scene content 208 as a source of revenue in an embodiment in which exhibitor 240 plays a more passive role than the flow shown in fig. 8. In an initial delivery step 242, the movie production company 110 provides the digital movie to the exhibitor 240 while providing the alternate content for the variable content display area 204 or the entire scene. In a sell offer step 244, company 110 offers advertiser 260 an opportunity to purchase alternate scene content 208, which is shown based on audience metrics in one or more movie shows. The replacement scene content 208 may be produced by the movie production company 110 or the advertiser 260 according to metadata provided by the movie production company. In a purchase step 248, the advertiser 260 purchases promotional space. Then, in the implementation step 262, the movie production company 110 provides the exhibitor 240 with implementation code for decoding and using the alternative content for exhibition purchased by the advertiser 260. At presentation, the exhibitor 240 obtains audience metrics in a metrics collection step 264. Alternate scene content is then presented 208 according to appropriate audience metrics. The screening action is recorded and reported to the film production company 110 in a reporting step 266.

The arrangement in fig. 8 allows for alternative scene content 208 to be presented with a variety of alternatives, such as the following:

(a) when presented individually, the decision to present any particular one of a set of promotional scenes or to include particular frame 200 content may be based on the obtained audience metrics. Thus, for example, if a particular movie is particularly attractive to a crowd of a particular age, the alternate scene content 208 for that crowd is shown. For example, the decision to show one or other alternative movie trailers at any one time, or simply to show default content, would be based on collecting audience metrics at the time of the showing;

(b) the billing level may be adjusted based on audience metrics. Thus, for example, upon determining that the audience size has at least reached the threshold value for the target demographic group, the exhibitor 240 may levy a premium for presenting the alternate scene content 208; conversely, if the audience metrics indicate that the target demographic group is rare at a particular show, the bonus level is lowered accordingly;

(c) alternate scene content 208 from different sources, such as from different advertisers 260, available for each showing of the movie, may decide whether to show promotional content a or promotional content B while the audience is seated;

(d) the network connection may be used as a "conduit" to report audience metrics or any of the transaction steps shown in fig. 8. In particular, purchasing step 248 or implementing step 262 may advantageously employ a network connection, thereby allowing for differentiation of scene content and pricing of each showing based on, for example, audience metrics.

It will be appreciated that the ability to replace scene content based on the characteristics of the viewer is beneficial to advertisers 260 who will exhibit greater advertising effectiveness for a particular market. Similarly, exhibitors 240 and movie production companies 110 benefit by being able to indicate to advertisers 260 a more efficient use of promotional space.

Still referring to FIG. 8, the metric collection step 264 may be performed in a variety of ways, applying various techniques from manual statistics and reporting to using pattern recognition tools to automatically obtain data about audience members.

Referring to fig. 9, a flow diagram of using alternate scene content 208 as a source of revenue in another embodiment is shown. In an initial delivery step 242, the film production company 110 provides the digital cinema to the exhibitor 240 and provides a list of alternate content produced by the film production company 110 that is available for the variable content display area 204 or all of the scenes in one or more sequences of frames 200, whether video content or audio content, or both video content and audio content. Is provided with default content; however, the replacement content may be replaced if purchased. In this manner, local advertisers may purchase alternative content produced by a film production company for presentation to viewers having a predetermined demographic profile. In a negotiation step 268, the exhibitor 240 or an agency mandated by the exhibitor or film production company 110 negotiates the sale of replacement content to advertisers 260 that is to be shown based on audience metrics for one or more film shows. In order step 270, advertiser 260 purchases alternate content based on the negotiation results. In the delivery step 272, the movie production company 110 provides the replacement content that has been ordered, along with any required fulfillment data, such as a decode key or metadata. At presentation, the exhibitor 240 obtains audience metrics at a metrics collection step 264. The purchased replacement scene content is then projected 208 according to appropriate audience metrics. The screening action is recorded and reported to the film production company 110 in a reporting step 266.

Selectable audio content

It must be emphasized that the replacement content 208 may comprise video or audio content, or both. If the alternate content 208 is video content only, the original soundtrack is used; if the replacement content 208 is audio only, the original video frames are used. A third possibility is to overwrite the existing content or to mix the audio in the replacement content 208 with the existing content if the replacement content 208 has both video and audio content. Although the above description is primarily directed to the use of alternate video content, the transactions shown in the figures in fig. 7, 8, 9 may also be used for alternate scene content, which may be either audio content or video content, or both.

When applying the method of the invention, it is optional to introduce the processing of the audio content as part of the replacement content 208. The channel data may be substituted for or blended with portions of the movie channel data to coordinate with any sequence of modified image frames 200'. The audio content may replace existing audio content or be selectively inserted into the movie as one or more sound materials. For example, the jingle of an advertisement may be inadvertently heard in the background of a movie scene. Alternatively, different soundtrack content may be selected for use based on audience metrics. For a selected movie, this can even select an alternate channel based on audience metrics, for example.

Referring to fig. 10, a flow chart illustrating the steps of obtaining audio content as replacement content 208, selecting particular audio content based on audience metrics, and obtaining revenue from the campaign is shown. In an initial delivery step 242, the movie production company 110 provides the exhibitor 240 with a digital movie as a digital movie image data stream. In addition, the movie production company 110 also provides information about the audio content in the form of metadata, in which replacement or mixing is allowed. In a sell offer step 274, the exhibitor 240 sells the advertising customer 260 alternate scene content for which promotional space is applicable. In a purchase step 276, the advertiser 260 purchases promotional space for the sound material. At the delivery and implementation step 302, for example, the movie production company 110 provides the exhibitor 240 with implementation information, such as a code, for the sound material and any additional metadata that is needed. The audio content itself is typically provided by the advertiser 260, although the movie production company may provide alternative audio content for local or regional purchase.

Next in FIG. 10, exhibitor 240 obtains audience metrics at data collection step 264. Based on the resulting audience metrics, the exhibitor 240 selects which of the alternative sound materials to present at a particular presentation in a presentation step 300. The presenter 240 then inserts the selected sound material in the presentation. The activity is recorded and reported to the film production company 110 in a reporting step 266.

The transaction flow in fig. 10 allows any number of modifications within the scope of the present invention. For example, the movie production company 110 may simply provide the metadata and any necessary encoded information suitable for audio insertion as the replacement content 208 as part of its sale, rental, or movie distribution. Alternatively, exhibitor 240 may be required to pay a fee for showing any locally sold promotional content. Additional billing settings may be negotiated to adjust the pricing of additional promotional content based on audience metrics such as size or age demographics. Advertisers 260 may include, for example, musicians who purchase promotional space. The movie production company 110 or exhibitor 240 may provide the existing audio content to the advertiser 260 for mixing with the additional content.

In addition to affecting the composition of the image display data stream 180, audience demographic data or associated audience metrics may also be used as a basis for selecting advertising or promotional content for presentation in rest rooms or other places outside of movie theaters.

Parts list

10-film distribution system

20 film production company

22 content provider

24-film print

26 issuing network

28 cinema

30 content film

70 demographic data entry points

72 employee

74 control desk

76 database

80 audience members

82 digital cinema data

84 camera

100 digital film production and distribution system

110 film production company

111 post production facility

112 film

114 data cinema system

116 auxiliary input device

118 broadcast system

120 disk array

122 transmitter

130 conveying system

132 magnetic media

134 optical medium

136 optical cable connection

138 satellite

140 projection system

142 magnetic media reader

144 optical media reader

146 film operating system

148 receiver

150 digital projector

160 main body

160 images

162 blue screen

164 electronic camera

170 composite movie

172 base content

174a1, 174a2 optional scene

176z1, 176z2, 176z3 optional outcome

180 image display data stream

182 scene metadata

200 image frames

200 modified image frame

202 variable replaceable image component

202 replace alternative image content

204 variable content display area

206 vertex

208 replacement content

240 projector

242 original transfer step

244 sell providing step

246. 248 purchase step

250 implementation step

252 specifying step

254 content providing step

260 advertiser

262 implementation step

264 metric fear gathering step

266 reporting step

268 step of discussion

270 ordering step

272 transmitting step

274 selling and providing step

276 purchase step

300 projection step

302 transfer and implementation steps

Claims (7)

1. A method of projecting a motion picture received as a digital data stream having a plurality of original scene sequences and having selectable scene content in one or more of the original scene sequences, said method comprising the steps of:

(a) acquiring audience data relating to a plurality of audience members watching a movie while showing the movie;

(b) receiving digital image data of a replacement sequence for replacing scene content in a movie, the scene content having one or more complete frames of an original scene sequence;

(c) substituting a replacement sequence of one or more complete frames for replacement scene content into a digital data stream for one of said original scene sequences at said presentation time based on said viewer data, thereby altering selectable scene content for presentation based on said viewer data; and

(d) a fee is charged for presenting the replacement sequence of scene content.

2. A method of projecting a motion picture as claimed in claim 1 wherein said scene content replacement sequence received in step (b) comprises a series of complete image frames.

3. A method of projecting a motion picture as claimed in claim 1 wherein said scene content replacement sequence received in step (b) comprises image content inserted into image frames provided in the motion picture.

4. A method of showing a movie as claimed in claim 1, wherein the scene content replacement sequence received in step (b) comprises slide show images.

5. A method of projecting a motion picture as claimed in claim 1 wherein step (c) of substituting said alternate sequence of scene content into said digital data stream of the motion picture comprises adjusting said digital image data of said alternate sequence of scene content to conform to scene characteristics of one or more successive frames.

6. A method of projecting a motion picture as claimed in claim 1 wherein said data relating to said viewer acquired in step (a) is a factor in setting the amount of said fee.

7. A method of projecting a motion picture as claimed in claim 1 wherein said data obtained in step (a) relating to said viewer is a factor in selecting a particular alternate sequence of said scene content from a particular advertiser.