US8922718B2 - Key generation through spatial detection of dynamic objects - Google Patents
Key generation through spatial detection of dynamic objects Download PDFInfo
- Publication number
- US8922718B2 US8922718B2 US12/909,508 US90950810A US8922718B2 US 8922718 B2 US8922718 B2 US 8922718B2 US 90950810 A US90950810 A US 90950810A US 8922718 B2 US8922718 B2 US 8922718B2
- Authority
- US
- United States
- Prior art keywords
- region
- dynamic object
- video
- interest
- suppression
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/64—Circuits for processing colour signals
- H04N9/74—Circuits for processing colour signals for obtaining special effects
- H04N9/75—Chroma key
Definitions
- the present invention relates to the insertion of virtual graphics into a video stream such that these graphics may appear to be behind other graphics or complex colored objects in the video stream and/or in front of other complex colored objects or background structures in the video stream.
- one approach that may be used is to place a logo without occlusion in a region of the scene where occluding objects are unlikely. This may lead to less than ideal placement from a visibility standpoint, and may still be susceptible to the occasional occlusion failure of projectile objects. Furthermore, there maybe failure due to overlap with other graphics which are likewise positioned in the region of the scene where occluding objects are unlikely.
- post-production approaches such as roto-scoping may be employed, where as the boundary between an object and the background is defined though manual operations. This is a time consuming approach operationally with application that are limited to pre-produced content.
- FIG. 1 illustrates an exemplary video scene according to an exemplary embodiment of the present invention.
- FIG. 2 illustrates a first block diagram of a suppression key generation system according to an exemplary embodiment of the present invention.
- FIG. 3 illustrates a first dynamic object that is overlaid onto the video depicting the event and, upon its detection, its corresponding suppression key according to a first exemplary embodiment of the present invention.
- FIG. 4 illustrates a second dynamic object that is in the video scene and, upon detection, its corresponding suppression key according to a second exemplary embodiment of the present invention.
- FIG. 5 illustrates a block diagram of a video insertion system according to an exemplary embodiment of the present invention.
- FIG. 6 illustrates a block diagram of a video pipe-line according to an exemplary embodiment of the present invention.
- FIG. 7 is a schematic diagram of an example computer system used to integrate virtual graphics into a video depicting an event according to an exemplary embodiment of the present invention.
- An event such as a sporting contest, a musical concert, a speech, television sitcom, or a television reality show to provide some examples, may be viewed by an actual viewer that is physically present to observe the occurrence of the event or by a remote viewer.
- the remote viewer observes a video depicting the event using a television platform, an internet platform, a mobile platform, or any other suitable platform that is capable of displaying the video.
- the video includes one or more video scenes which are provided to the remote viewer by a broadcaster.
- the video may be broadcast near live as the event is happening in real-time, or may be a delayed broadcast occurring after the event is complete.
- the one or more video scene captures the event for the remote viewer as the event occurs and may be overlaid with virtual graphics to enhance observation of the event by the remote viewer.
- FIG. 1 One such exemplary video scene that may be produced by the broadcaster is illustrated in FIG. 1 .
- FIG. 1 illustrates an exemplary video scene according to an exemplary embodiment of the present invention.
- a video scene 102 illustrates a hockey scene from a hockey contest that is recorded onto video.
- the video scene 102 is not limited to the hockey scene as illustrated in FIG. 1 , those skilled in the relevant art(s) will recognize the video scene 102 may depict any other suitable video scene, such as a video scene from a different sporting contest or any other event that may be recorded onto the video such as a musical concert, a speech, television sitcom, or a television reality show to provide some examples, without departing from the spirit and scope of the present invention.
- the broadcaster of the event typically overlays one or more virtual graphics onto the video depicting the event to produce the video scene 102 .
- These virtual graphics represent graphical images that are seen or visualized by the remote viewer of the video scene 102 ; however, these graphics remain unseen or hidden to the actual viewer.
- the broadcaster may be a traditional provider of television channels such as a television network or a cable company, or any other suitable provider of video to multiple recipients, such as video over the web.
- a first type of virtual graphic represents a static graphic 104 that is displayed to the remote viewer on a permanent or semi-permanent basis during the event. For example, the static graphic 104 may display a score of the hockey contest, period of the hockey contest, and time left in the period.
- a second type of virtual graphic represents a dynamic graphic 106 that contains information that is displayed to the remote viewer on a periodic basis that may appear and disappear during the event. For example, the dynamic graphic 106 may display time left in a power play.
- a third type of virtual graphic represents a realistic graphic 108 that is realistically rendered and mixed to appear, to the remote viewer, as part of the event. For example, the realistic graphic 108 may display advertisements appearing on the perimeter hockey glass to the remote viewer.
- a physical location the dynamic graphic 106 overlaps a physical location of the realistic graphic 108 .
- the placement of the dynamic graphic 106 and the realistic graphic 108 onto the video depicting the event appears realistic in an overlap region to the remote viewer, namely the dynamic graphic 106 appears to be in front of the realistic graphic 108 in the video scene 102 .
- the dynamic graphic 106 may be overlaid into the video scene 102 after the realistic graphic 108 to give the video scene 102 this realistic appearance.
- a complex background 110 present within the onto the video depicting the event precludes the use of conventional occlusion techniques to properly overlay the realistic graphic 108 and the dynamic graphic 106 to give this realistic appearance to the remote viewer.
- the complex background 110 upon which the realistic graphic 108 is overlaid, represents a region of the onto the video depicting the event that is characterized as having an inconsistent color signature as is necessary for the conventional occlusion techniques.
- the complex background 110 represents the actual viewers seated behind the perimeter hockey glass upon which the realistic graphic 108 is to be overlaid. The color signature produced by the actual viewers of the event varies during the course of the hockey contest.
- the conventional occlusion techniques may be used to integrate realistic virtual graphics into the video depicting the event for a simple background 112 .
- the simple background 112 represents a region of the video depicting the event that is characterized as having a consistent color signature which may be used by the conventional occlusion techniques to integrate realistic virtual graphics.
- the simple background 112 represents a portion of the ice upon which the virtual graphics may be overlaid by the conventional occlusion techniques.
- these techniques may not apply in complex backgrounds, such as complete background 110 to provide an example, because the movement within this region, such as movement of spectators in this region, and color variation within this region, such as color variations of the spectators themselves, may cause inconsistencies in color signature.
- the conventional occlusion techniques may not properly integrate the realistic virtual graphics into these regions to give the realistic appearance to the remote viewer.
- the virtual graphics may be inserted without occlusion, but the graphics will be overlaid in front of objects, such as the static graphic 104 , the dynamic graphic 106 , and/or the hockey players 114 to provide some examples, that may appear in the insertion region, again giving an unrealistic appearance to the remote viewer.
- FIG. 2 illustrates a first block diagram of a suppression key generation system according to an exemplary embodiment of the present invention.
- a suppression key generation system 200 generates a suppression key that may be used to suppress regions in a video depicting the event for the integrating of realistic virtual graphics into the video depicting the event to produce the video scene 102 .
- the suppression key allows for proper integrating of the realistic virtual graphics into the video depicting the event to provide the realistic appearance of these virtual graphics to the remote viewer.
- the suppression key may represent an extraction key that is used to extract a foreground object such as the static graphic 104 , the dynamic graphic 106 , and/or the hockey players 114 to provide some examples, from the video depicting the event, and render this foreground object into another video scene. For example, as shown in FIG.
- the suppression key allows for the proper placement of the dynamic graphic 106 and/or the realistic graphic 108 onto the video depicting the event to appear realistic in the video scene 102 to the remote viewer.
- the suppression key generation system 200 includes a video analysis module 204 , a key generator module 206 , and, optionally, a video reception module 202 and/or a video combination module 208 .
- the video reception module 202 , the video analysis module 204 , the key generator module 206 , and/or the video combination module 208 may be implemented in hardware or as software routines that are executed on one or more processors.
- the video reception module 202 receives a region of interest 252 within the video depicting the event.
- the region of interest may represent one or more regions of the video depicting the event.
- the region of interest 252 may be a fixed screen area, such as the complex background 110 or the simple background 112 , that is bounded by a polygon, a curved shape fixed to the video scene 102 , or any other suitable region of the video scene 102 that will be apparent to those skilled in the relevant art(s) from the teachings herein without departing from the spirit and scope of the present invention.
- the region of interest 252 may be tied to a physical scene that is depicted in the video such as a virtual insertion region or a physical scene, or a portion thereof, that is in view to the remote viewers to provide some examples.
- the region of interest 252 may be a video image, such as a field or a frame to provide some examples, in its entirety.
- the region of interest 252 may represent one or more images, or one or more portions thereof, from among a sequence of images that comprise the video depicting the event.
- the one or more images of a video sequence may be one or more fields or frames of the video sequence.
- the region of interest 252 may be one or more fixed location within a particular field or frame or image.
- the region of interest 252 may be tied to an approximate location of objects or groups of objects that move around a scene.
- the video analysis module 204 determines through spatial means a presence of a dynamic object within the region of interest 252 to provide a dynamic object indicator 254 .
- the dynamic object may be characterized as a static graphic in a fixed location within the video scene 102 , which appears/disappears to the remote viewers and updates, such as the static graphic 104 to provide an example.
- This form of the dynamic object may be represented by the score of the hockey contest, the period of the hockey contest, and the time left in the period.
- the dynamic object may be characterized as a dynamic moveable graphic that may appear to move, namely, to expand and/or contract, across the video scene 102 to the remote viewers, such as the dynamic graphic 106 to provide an example.
- the dynamic object may be represented by the time left in the power play.
- the dynamic object may be characterized as a dynamic graphic, such as the dynamic graphic 106 to provide an example, that appears to be physically present in the scene such that it pans into and out of view of the remote viewers.
- the dynamic object may be characterized as a physical object that is physically present at the event, that is, capable of being seen by the actual viewers of the event, such as the hockey players 114 , their associated equipment, or a scoring object to provide some examples.
- the dynamic object may be characterized as a dynamic graphic, such as the dynamic graphic 106 to provide an example, that may include a realistic virtual graphic.
- the determination of the video analysis module 204 may be manually made by an operator reviewing a spatial presentation of the region of interest 252 who manually signals whether the dynamic object is present using an input device or who electronically triggers a tally signal from a video switcher or graphics engine. Alternately, the determination may be automated using image processing matching techniques, examples of which are described below in FIG. 3 and FIG. 4 , and may additionally involve defining the screen position or shape of the object. In another alternate, the determination may be made by a hybrid method combining the manual detection and the automated detection.
- the key generator module 206 provides a suppression key 256 in response to the dynamic object indicator 254 .
- the suppression key 256 may represent one or more key values corresponding to the dynamic object when the dynamic object indicator 254 indicates the presence of the dynamic object within the region of interest 252 .
- the suppression key 256 may represent a default value when the dynamic object indicator 254 indicates the absence of the dynamic object within the region of interest 252 .
- the suppression key 256 sequence for the region of interest 252 has a frame by frame correspondence to the video scene 102 .
- the suppression key 256 may be pre-determined to match expected virtual graphics.
- the suppression key 256 may use alternate representations suitable for keying, mixing, blending, overlapping, and/or cropping of graphics together, video channels together, or a combination of graphics and video. Additional alternate forms of the suppression key 256 may include alpha channel, depth map, bitmask, grayscale mask, polygon shape, contour boundary of an object, outline of an object, or any other suitable form that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention. These alternate representations of the suppression key 256 may be considered a suppression key.
- the suppression key 256 may represent an extraction key that is used to extract a foreground object within the region of interest 252 , and render this foreground object into another video sequence.
- a person may be embedded into a scene to appear to be interacting with other people that are physically captured in the video sequence. This may be useful in live events, where it is valuable for announcers to interact with commentators or participants of the event, but there is not sufficient time to move between locations. Alternately, a commentator may be able to walk virtually within the event when analyzing a replay of the event.
- the video combination module 208 utilizes the suppression key 256 representation to combine the region of interest 252 with a virtual graphic, combine virtual graphic together, to combine video channels together, and/or to combine a virtual graphic or multiple virtual graphic with one or more video channels to provide a video scene 258 such as the video scene 102 to provide an example.
- FIG. 3 illustrates a first dynamic object that is overlaid onto the video depicting the event and, upon its detection, its corresponding suppression key according to a first exemplary embodiment of the present invention.
- a dynamic object 300 may be characterized as a static graphic in a fixed location within the video scene 102 , such as the region of interest 252 to provide an example.
- the dynamic object 300 may represent an exemplary embodiment of the static graphic 104 .
- the dynamic object 300 may include one or more static regions 302 that are characterized as being static to the remote viewer for the entire event.
- the one or more static regions 302 may include a first region corresponding to the broadcaster, denoted MSG in FIG. 3 , and a second and a third region corresponding to the participants in the hockey contest, denoted WSH and NYR in FIG. 3 .
- the dynamic object 300 may additionally include one or more periodic regions 304 that are characterized as being periodically updated to the remote viewer during the event.
- the one or more periodic regions 304 may include a first region corresponding a score of the participants in the hockey contest, denoted as 0 and 1 in FIG. 3 , and a second region corresponding to the period of the hockey event, denoted as 1st in FIG. 3 .
- the dynamic object 300 may further include one or more frequently updating regions 306 that are characterized as being frequently updated to the remote viewer during the event.
- the one or more frequently updating regions 306 may include a region corresponding to a remaining time in the period of the hockey contest, denoted as “:29.7” in FIG. 3 .
- the dynamic object 300 may yet further include one or more temporary regions 308 that are characterized as temporarily appearing to the remote viewer.
- the one or more temporary regions 308 may include a region corresponding to an upcoming announcement for the event, denoted as COMING UP IN THE MERCEDES INTERMISSION REPORT AL WITH BRANDON DUBINSKY in FIG. 3 .
- the dynamic object 300 may include any combination of the one or more static regions 302 , the one or more periodic regions 304 , and/or the one or more temporary regions 308 .
- the video analysis module 204 may utilize automated detection methods to detect the presence of the dynamic object 300 within the region of interest 252 . These automated detection methods may favor the use of the static regions 302 and/or the periodic regions 304 over the temporary regions 308 because the video analysis module 204 may search the region of interest 252 for a single pattern for an extended period of time.
- the video analysis module 204 may utilize a vision processing algorithm to train the automated detection methods on the dynamic object 300 , or portions thereof, before and/or during the event to generate a spatial model of the dynamic object 300 .
- the spatial model of the dynamic object 300 may include one or more reference images corresponding to a pixel-by-pixel mapping of at least one of the static regions 302 , the periodic regions 304 , and/or the temporary regions 308 and its intended location within the region of interest 252 .
- the video analysis module 204 may make a pixel-by-pixel spatial comparison between the one or more reference images in the spatial model and the region of interest 252 during the event to determine the presence of the dynamic object 300 .
- the video analysis module 204 determines the dynamic object 300 to be present when the region of interest 252 , or a portion thereof, substantially matches the one or more reference images in the spatial model.
- the spatial model of the dynamic object 300 includes a spatial color model with or without the one or more reference images.
- the one or more reference images stored may retain one or more channel color spaces values, such as three channel color space values, for example, RGB, YUV, HSV, etc., to provide an example, one or more channel chroma values, and/or may be a single channel intensity or luma image.
- the one or more reference images may represent full video fields or frames or some region of interest within the full video field or frame.
- the one or more reference images may be subject to image processing techniques including, but not limited to, contrast stretching, histogram or other normalization, edge enhancement, zero mean or other suitable filtering technique such as high pass, band pass, and/or low pass filtering, that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- This one or more reference images may be templates used for correlation comparison or other types of matching techniques.
- the one or more reference images may be scaled or expanded representations of captured images.
- the spatial model of the dynamic object 300 may include one or more reference images corresponding to a color distribution for at least one of the static regions 302 , the periodic regions 304 , and/or the temporary regions 308 and its intended location within the region of interest 252 .
- the video analysis module 204 may make a color comparison between the one or more reference images in the spatial model and the region of interest 252 during the event to determine the presence of the dynamic object 300 .
- the video analysis module 204 determines the dynamic object 300 to be present when the color distribution of the region of interest 252 , or a portion thereof, substantially matches the color distribution of the reference images in the spatial model.
- the video analysis module 204 may compute object features in the one or more reference images from the spatial model of the dynamic object 300 . These object features may be extracted from the region of interest 252 and matched to the spatial model of the dynamic objects 300 to identify the presence of the dynamic object 300 in the region of interest 252 .
- the video analysis module 204 may utilize character recognition to validate the presence of the dynamic object 300 .
- the video analysis module 204 may compare the one or more reference images in the spatial model with the region of interest 252 to generate a suppression key on a field basis.
- the dynamic object 300 may be dynamic as in the case of a DVE, and a spatial-temporal extension to the above techniques may be employed to detect the dynamic object 300 appropriately over time.
- the object features may be characterized as salient features of an object that were extracted from the one or more reference images. These salient features may include scale-invariant descriptors such as keypoints or image templates to provide some examples.
- the object features may be extracted from the region of interest 252 and matched to a database of features in an object model of the dynamic object 300 that is derived from the one or more reference images. The strength of the matching as well as the matched feature locations may be used to determine the presence of the dynamic object 300 within the region of interest 252 .
- Detection of the object features may employ a variety of techniques such as edge detection, line detection, boundary detection, blob detection, corner detection, and/or any other suitable detection process that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- the location and nature of these detected features may be used to determine the presence of the dynamic object 300 in the region of interest 252 .
- the object model may include the image processing thresholds or algorithm settings to perform these aforementioned detection techniques.
- the image processing thresholds or algorithm settings may be determined using the one or more reference images, but are not necessarily derived from the images themselves.
- the object model, the spatial model, and/or the spatial color model may include the object features, the image processing thresholds, and/or the algorithm settings.
- the key generator module 206 provides the suppression key 256 corresponding to the dynamic object 300 upon its detection within the region of interest 252 .
- the suppression key 256 represents a region within the video scene 102 that is to be suppressed allowing for the integrating of virtual graphics into this suppressed region and/or extracted from the region of interest 252 and merged into another region of interest or another video scene.
- pixels corresponding to the dynamic object 300 within the suppression key 256 may be assigned a value of a large value, representative of the color black, to allow for suppression other graphics, such as the realistic graphic 108 to provide an example, that overlap the intended location of the dynamic object 300 in the video scene 102 .
- the pixels corresponding to the dynamic object 300 within the suppression key 256 may be assigned a small value representative of the color white.
- the pixels corresponding to the dynamic object 300 within the suppression key 256 may be assigned any suitable value between the large value and the small value that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- the suppression key 256 may include multiple suppression regions such as a first suppression region 324 , a second suppression region 326 , and a third suppression region 326 to provide some examples. Those skilled in the relevant art(s) will recognize that the suppression key 256 may include a lesser or a greater number of suppression regions without departing from the spirit and scope of the present invention.
- the pixels of each of the multiple suppression regions of the suppression key 256 may be assigned to similar and/or dissimilar values. For the example illustrated in FIG. 3 , pixels of the first suppression region 324 are assigned to a first value, typically a small value representative of approximately zero percent opacity, to allow for suppression of other graphics that overlap the intended location of the one or more static regions 302 , and the one or more periodic regions 304 .
- Pixels of the second suppression region 326 are assigned to a second value, typically a smaller value representative of less than approximately one hundred percent opacity, to allow for suppression, or a partial suppression, of the other graphics that overlap the intended location of the one or more frequently updating regions 306 .
- Pixels of the third suppression region 328 are assigned to a third value, typically a small value representative of approximately zero percent opacity, when the temporary regions 308 are present in the video scene 102 or to a fourth value, typically a large value representative of approximately one hundred percent opacity, when the temporary regions 308 are no longer present in the video scene 102 .
- opacity value near zero may represent areas for suppression and opacity values near one hundred percent may represent areas without suppression.
- the convention of near one hundred percent opacity may be used in regions of suppression, and near zero percent opacity for regions with minimal suppression.
- sample instances of virtual graphics with their appropriate location in the video scene 102 may be obtained before and/or during the event, and the suppression key 256 may be generated from a polygon region manually positioned around the graphic location.
- the suppression key 256 can be generated by detecting the boundary colors of the dynamic object 300 , or its regions, or by manually tracing the outline of the dynamic object 300 , or its regions, and flood-filling the internal holes in the dynamic object 300 , or its regions.
- the suppression key 256 may be formulated heuristically by comparing pixel color in the dynamic object 300 , or its regions, to the color outside of the dynamic object 300 , or its regions, and assigning a highest key value to pixels that mostly differ from the pixels outside of the dynamic object 300 .
- the suppression key 256 may represent a complex decal that is generated off-line and stored as a bitmap, and retrieved for run-time operation.
- FIG. 4 illustrates a second dynamic object that is in the video scene and, upon detection, its corresponding suppression key according to a second exemplary embodiment of the present invention.
- the region of interest 252 may include a dynamic object 400 that is physically present within the video scene 102 in its entirety or within the region of interest 252 to provide an example.
- the dynamic object 400 represents an object such as one of the hockey players 114 to provide an example, which is capable of being seen by the actual viewers of the event.
- the video analysis module 204 may utilize automated techniques to detect object components of the physical object 400 , or portions thereof. These automated techniques may include torso detection 402 , helmet detection 404 , limb detection 406 , skate detection 408 , puck detection 410 , stick detection 412 , and/or any other suitable technique to detect an object component of the physical object 400 , or portions thereof, that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- the automated techniques for helmet detection 404 and limb detection 406 are well known in the art and will not be described in further detail.
- the automated techniques for torso detection 402 , puck detection 410 , and stick detection 412 are further described in U.S.
- Each of the object components of the physical object 400 may be represented using a spatial color model with appropriate size and shape characteristics dependent on a distance of these objects from a viewing camera.
- the color representation in a spatial color model may vary with spatial positioning.
- the color representation may be a histogram distribution according to color values.
- the video analysis module 204 may utilize a vision processing algorithm to train a collection of spatial color models to represent participants of the event, such as an organized team sport, such as hockey, to provide an example, before and/or during the event.
- the collection of spatial color models may be formulated in such a way that it varies spatially, such as a vertical direction, an inner region versus boundary, or any other suitable spatial relationship that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- an operator will initiate training of the collection of spatial color models by manually selecting the dynamic object 400 within the region of interest 252 through a method such as a rubber-band box to provide an example.
- foreground color training may be achieved by detecting foreground object in regions from the region of interest 252 that are expected to have constant background color.
- color segmentation algorithms may segment the region of interest 252 into multiple regions, which are then separated from pre-trained background colors from the collection of spatial color models to form an outline of the physical object 400 .
- These spatial color models may be considered spatial models.
- manual selection of colors at the boundary of the object may be combined with an edge detection scheme to foam the outline of the physical object 400 .
- the key generator module 206 provides the suppression key 256 upon detection of the dynamic object 400 within the region of interest 252 .
- the key generator module 206 applies the collection of spatial color models to color regions within the region of interest 252 during the event and detects the dynamic object 400 as being present within the region of interest 252 when a sufficient number and distribution or spatial extent are present.
- the suppression key 256 represents a suitable key or mask that allows the physical object 400 , such as the hockey player 114 to provide an example, to be extracted from the region of interest 252 and merged into another region of interest or another video scene.
- the suppression key 256 includes one or more regions from multiple detected components, such one or more of a player silhouette 422 , a helmet silhouette 424 , a puck silhouette 426 , and/or a stick silhouette 428 to provide some examples.
- the suppression key 256 may include a first value, typically a large value representative of approximately one hundred percent opacity, corresponding to the physical object 400 , namely the player silhouette 422 , the helmet silhouette 424 , the puck silhouette 426 , and the stick silhouette 428 .
- the area of suppression corresponds to this small value.
- the suppression key 256 may include a second value, typically a high value, corresponding to a remainder of the region of interest 252 not including the physical object 400 .
- the area without suppression corresponds to a high value.
- the video analysis module 204 may not be able to detect all of the pixels of the physical object 400 .
- the key generator module 206 uses a fills these missing pixels in the suppression key 256 using a mask fill process such as flood-fill, morphological operations, or any other suitable process that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the invention.
- the video analysis module 204 may optionally use an edge detector to reconstitute the boundary of the dynamic object 400 prior to the mask fill process.
- the video analysis module 204 may detect the boundary of the dynamic object 400 and track this boundary over time.
- the video analysis module 204 may detect the dynamic object 400 in a complex region, such as the complex background 110 to provide an example, using only a portion of the physical object 400 , such an upper torso and/or a head of the dynamic object 400 to provide an example.
- the video analysis module 204 may detect the dynamic object 400 in less complex regions, such as the simple background 112 to provide an example, using a combination of object modeling and conventional occlusion techniques.
- the region of interest 252 may be positioned high on the perimeter hockey glass in a broadcast of a hockey contest where a portion of the physical object 400 , such as tops of the helmets of the hockey participants, arms of the hockey participants, the sticks of the hockey participants to provide some examples, may occlude.
- criteria used by the video analysis module 204 for the dynamic object 400 may be made progressively stringent, for example toward a top of the region of interest 252 , based on a likelihood that the dynamic object 400 is present.
- FIG. 5 illustrates a block diagram of a video insertion system according to an exemplary embodiment of the present invention.
- a video insertion system 500 integrates realistic virtual graphics into a video feed 550 and/or overlays graphics onto the video feed 550 to provide a video scene 552 .
- the video feed 550 represents one or more video frames of an event, such as the hockey contest as illustrated in FIG. 1 , such as the video scene 102 to provide an example.
- the video insertion system 500 may be used in coordination with video production for television, internet, and mobile applications.
- the video insertion system includes a video tracking module 502 , an object tracking module 504 , a mask generator module 506 , a video delay module 508 , a video rendering module 510 , a key mixing module 512 , a video mixing module 514 , and an operational control module 516 .
- the video tracking module 502 analyzes the video feed 550 to determine the orientation of a camera recording the event, as well as transitions between multiple cameras recording the event, to provide camera tracking data 554 .
- the video feed 550 may be received from a video production such as a dedicated feed from the camera recording the event, or an isolated point of view of the camera, or a program feed which cuts among video from among the multiple cameras.
- the video tracking module 502 camera tracking data 554 may provide the camera tracking data 554 over a network such a wireless and/or wired network such as Ethernet to provide an example.
- the video tracking module 502 may additionally analyze the video feed 550 for landmarks to provide a representation of the pan, tilt, and zoom (PTZ) of the video feed 550 .
- the video tracking module 502 may provide the representation of the PTZ with respect to a location point in a scene. This representation of the PTZ may be combined with the determine the orientation of a camera recording the event, as well as the transitions between the multiple cameras recording the event, to provide a camera model representation as the camera tracking data 554 .
- the camera model may contain multiple parameters that relate to physical measurements of a tripod mounted camera such as pan, tilt, roll, image distance, x position, y position, z position. Other parameters, such as parameters for radial lens distortion for example, may be utilized.
- the camera model representation may be used to obtain information about the scene depicted in the video feed 550 .
- the camera model may be additionally used to establish a mapping between data gathered in a screen space of the broadcast video and view point representation.
- the camera or sensor model representations may include directional measurements.
- the sensor model representation may include information corresponding to objects or locations in an image or video.
- the video tracking module 502 may further update a camera position, or the position of the multiple cameras, over time through frame to frame analysis of features in the video, such as a texture analysis method as described in U.S. patent application Ser. No. 08/580,892, filed on Dec. 29, 1995, now U.S. Pat. No. 5,808,695, and U.S. patent application Ser. No. 09/308,949, filed on May 26, 1999, now U.S. Pat. No. 6,529,613, each of which is incorporated by reference herein in its entirety.
- the video tracking module 502 may rely on PTZ sensors on the camera or a combination of sensors and image stabilization as described in U.S. patent application Ser. No. 09/230,099, filed on Jan. 19, 1999, now U.S. Pat. No. 6,100,925, which is incorporated by reference herein in its entirety.
- the object tracking module 504 may detect and/or track virtual graphics and/or physical objects.
- the physical objects represent objects within the event that are capable of being viewed by the actual viewer, such as the dynamic object 400 to provide an example, that are depicted in the video feed 550 .
- the virtual graphics represents graphics within the event that are capable of being viewed by the remote viewer, such as the static graphic 104 , the dynamic graphic 106 and/or the realistic graphic 108 to provide some examples, which were previously overlaid into the video feed 550 .
- the object tracking module 504 provides object tracking information 556 indicative of whether the virtual graphics and/or the physical objects have been detected and, optionally, their location within the video feed 550 .
- the object tracking information 556 may indicate of a presence of the virtual graphics and/or the physical objects.
- the object tracking module 504 may detect and/or track static and/or dynamic objects, and/or portions thereof, such as the dynamic object 300 , the dynamic object 400 , and/or another other suitable dynamic object that will be apparent to those skilled in the relevant art(s) from the teachings herein without departing from the spirit and scope of the present invention.
- the object tracking module 504 may automatically detect and/or track static and/or dynamic objects, and/or portions thereof, in a simple background such as the simple background 112 to provide an example as described in U.S. patent application Ser. No. 12/403,857, filed on Mar. 13, 2009, which is incorporated by reference herein in its entirety, as well as static and/or dynamic objects, and/or portions thereof, in a complex background such as the complex background 110 .
- the mask generator module 506 creates a suppression key 558 that corresponds to the shape and location of the virtual graphics and/or the physical objects in response to the object tracking information 556 indicating their presence.
- the mask generator module 506 represents an exemplary embodiment of the suppression key generation system 200 as discussed above.
- the mask generator module 506 may include the video analysis module 204 , the key generator module 206 , and, optionally, the video reception module 202 .
- the video delay module 508 provides a pipe-line delay to the video feed 550 to compensate for the processing of the video tracking module 502 , the object tracking module 504 , mask generator module 506 , and the video rendering module 510 to provide a delayed video feed 560 .
- the video delay module 508 delays the video feed 550 for a sufficient duration to allow the video tracking module 502 , the object tracking module 504 , mask generator module 506 , and the video rendering module 510 to appropriately process the video feed 550 .
- the video rendering module 510 realistically formats virtual graphics such that these virtual graphics are adjusted to match their respective locations in the scene depicted on the video feed 550 based upon the camera tracking data 554 to provide a video key 562 and a video fill 564 .
- the video rendering module 510 may generate a graphical presentation, as the video fill 564 , by rendering a graphical insert into a region of interest of the video feed 550 in a position corresponding to a physical object in the video feed 550 .
- the graphical insert may appear to track with the physical object, such as a ring tracking a physical object, such as one of the hockey players 114 to provide an example, or an arrow pointing to the physical object where only the point of the arrow tracks the physical object to provide some examples.
- the graphical insert may appear to track with a background scene of the video feed 550 , such as the complex background 108 or the simple background 112 to provide some examples, and represent a path or a trail of the physical object moving across the background scene.
- the graphical insert may represent a three dimensional trajectory of the physical object, and appear locked in three dimensional space as the camera moves.
- the graphical insert may appear to be overlaid on the two dimensional surface of the display screen, and may appear to track with movements of the physical object.
- telestration graphics that diagrams the movements of the physical object are integrated into the background scene of the video feed 550 .
- the suppression key 558 suppresses foreground objects, namely those objects in front of a location of the telestration graphics, within the video feed 550 so that they appear to be in front of the graphic.
- the telestration graphics may be positioned and/or generated, in part, based on user commands captured using a manual interface, such as a touch screen, mouse, gaming device, tablet, and/or any other suitable device that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- a manual interface such as a touch screen, mouse, gaming device, tablet, and/or any other suitable device that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- the video key mixer 512 combines the suppression key 558 with the video key 562 to provide a suppressed video key 568 .
- the video mixer 514 uses the suppressed video key 568 to blend the video fill 564 with the delayed video 560 to create the video scene 552 .
- the operational control module 516 provides a control signal 566 to enable monitoring and/or correction of the operation of the video insertion system 500 .
- the operational control module 516 may provide feedback of the video tracking module 502 with diagnostics overlaid on a video window.
- the operator controller 518 may additionally allow monitoring of the tracking information 554 with an option to select information from an object of interest from the video feed 550 .
- the operational control module 516 may also enable the selection of graphics or statistics and the monitoring of results within the mask generator module 506 or the video rendering module 510 .
- the operational control module 516 may further notify the mask generator module 506 of a detection of the graphics and/or the objects within the video feed 550 by the video tracking module 502 .
- the operational control module 516 may control the video tracking module 502 , the object tracking module 504 , the mask generator module 506 , the video rendering module 510 , and the key mixing module 512 over a network such a wireless and/or wired network such as Ethernet to provide an example.
- the operational control module 516 may include an operator interface that allows an operator to manually determine a location of possible on-screen graphics before and/or during the event. For example, during the event, the operator may indicate the presence of a particular graphic through a manual interface such as a key pad, a mouse, a touch screen device, and/or any other suitable device that will be apparent to those skilled in the relevant art(s) without departing from the spirit and scope of the present invention.
- the video insertion system 500 may be positioned approximate to the event and/or away from the event allowing the broadcaster to remotely integrate realistic virtual graphics into the video depicting the event.
- FIG. 6 illustrates a block diagram of a video pipe-line according to an exemplary embodiment of the present invention.
- a video pipe-line 600 overlays virtual graphics onto a video depicting an event to provide a video scene, such as the video scene 102 to provide an example, to one or more remote viewers.
- the video pipe-line 600 includes a venue 602 , one or more cameras 604 , an on-site production facility 606 , a first insertion system 608 , a second insertion system 610 , a transmission medium 612 , a broadcast studio production facility 614 , a third insertion system 616 , and video distribution chain 618 .
- An event such as the hockey contest depicted in FIG. 1 , may take place at the venue 602 , whereby the event is recorded onto video by the one or more cameras 604 .
- the one or more cameras 604 may provide a corresponding video feed 650 depicting the event to the on-site production facility 606 and/or at least one of the one or more cameras 604 may, optionally, provide a dedicated video 652 to the first insertion system 608 .
- the on-site production facility 606 may overlay virtual graphics relating to the event, such as the dynamic object 300 to provide an example, onto one or more of the video feeds 650 to produce an event related video 654 .
- the first insertion system 608 may integrate other realistic virtual graphics into the dedicated video 652 to provide a dedicated video 656 which is then provided to the on-site production facility 606 to be overlaid with other virtual graphics.
- the on-site facility 606 may provide a video feed with or without overlaid virtual graphics to a second virtual insertion system 610 , which may integrate realistic virtual graphics and may return the composite feed back to the on-site facility 606 .
- the first or second virtual insertion systems may overlay other graphics for the on-site production facility 608 .
- the first insertion system 608 and/or the second insertion system 610 may be implemented as part of the on-site production facility 606 or external to the on-site production facility 606 .
- the event related video 654 may include one or more of the video feeds 650 and/or the dedicated video 656 , each of which has been overlaid with virtual graphics relating to the event.
- the on-site production facility 606 may provide camera data information, as discussed above, to insertion system 610 and/or the insertion system 612 that is extracted using instrumentation on the one or more cameras 604 or may be extracted directly by analyzing the one or more of the video feeds 650 , or a combination of video analysis and camera sensors. This camera data may additionally be provided to the broadcast studio production facility 614 for use by the third insertion system 616 .
- the event related video 654 is sent using a transmission medium 612 , such as a satellite to provide an example, to the broadcast studio production facility 614 .
- a transmission medium 612 such as a satellite to provide an example, to the broadcast studio production facility 614 .
- the transmission medium 612 is possible, such as other well known wired, such as coaxial cable or fiber optic to provide some examples, and/or wireless communications mediums without departing from the spirit and scope of the present invention.
- the broadcast studio production facility 614 may overlay virtual graphics relating to the other events and/or other virtual graphics onto one or more of the video feeds 650 to produce a video scene 656 .
- the video feed with or without virtual graphics may be provided to a third virtual insertion system 616 , which may integrate realistic graphics and may return the composite feed back to the studio production facility 614 .
- the third virtual insertion system may overlay other graphics for the studio production facility 614 .
- one or more of the first insertion system 608 , the second insertion system 610 , and the third insertion system 616 may represent one or more exemplary embodiments of the video insertion system 500 .
- each of the first insertion system 608 , the second insertion system 610 , and the third insertion system 616 is for exemplary purposes only, the video pipe-line 600 may include more or less insertion systems without departing from the spirit and scope of the present invention.
- the video scene 656 goes through a video distribution chain 618 , where it is provided to a television platform 624 , an internet platform 626 , and/or a mobile platform 628 .
- the video pipe-line 600 integrates virtual information into a video with an object based occlusion at a remote location from an on-site production using video analysis of the video.
- video analysis may include but not limited to a broadcast studio, regional cable head-end, local cable head-end, cable node, set-top box, computer system, mobile device, etc.
- this video analysis occurs at the on-site production facility 606 or at the broadcast studio production 616 and the information is propagated downstream in the distribution chain where the insertion is integrated (regional cable head-end, local cable head-end, cable node, set-top box).
- object detection information is sent to location remote from the event 602 to be used by the insertion system 620 integrate virtual graphics into video with occlusion.
- FIG. 7 is a schematic diagram of an exemplary computer system 700 used to integrate virtual graphics into a video depicting an event according to an exemplary embodiment of the present invention.
- the suppression key generation system 200 or portions thereof, may be implemented in the computer system 700 .
- the video insertion system 500 or portions thereof, may also be implemented in the computer system 700 .
- Portions of the video pipe-line 600 may also be implemented in the computer system 700 .
- Various embodiments of the invention are described in terms of this example computer system 700 . After reading this description, it will become apparent to a person skilled in the relevant art how to implement the invention using other computer systems and/or computer architectures.
- the computer system 700 includes one or more processors, such as processor 704 .
- the processor 704 may be a special purpose or a general-purpose processor.
- the processor 704 is connected to a communication infrastructure 706 , such as a bus or a network to provide some examples.
- the computer system 700 also includes a main memory 708 , preferably random access memory (RAM), and may also include a secondary memory 710 .
- the secondary memory 710 may include, for example, a hard disk drive 712 , a removable storage drive 714 , flash memory, a memory stick, and/or any similar non-volatile storage mechanism.
- the removable storage drive 714 may comprise a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, or the like.
- the removable storage drive 714 reads from and/or writes to a removable storage unit 718 in a well-known manner.
- the removable storage unit 718 may include a floppy disk, magnetic tape, optical disk to provide some examples which is read by and written to by the removable storage drive 714 .
- the secondary memory 710 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system 700 .
- Such means may include, for example, a removable storage unit 722 and an interface 720 .
- Examples of such means may include a program cartridge and cartridge interface, such as that found in video game devices, a removable memory chip, such as an EPROM, or PROM, and associated socket, and other removable storage units 722 and interfaces 720 which allow software and data to be transferred from the removable storage unit 722 to the computer system 700 .
- the computer system 700 may also include a communications interface 724 .
- the communications interface 724 allows software and data to be transferred between computer system 700 and external devices.
- the communications interface 724 may include a modem, a network interface, such as an Ethernet card to provide an example, a communications port, a PCMCIA slot and card, or the like.
- Software and data transferred via the communications interface 724 are in the form of signals, which may be electronic, electromagnetic, optical, or other signals capable of being received by communications interface 724 . These signals are provided to the communications interface 724 via a communications path 726 .
- the communications path 726 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, a radio frequency (RF) link or other communications channels.
- RF radio frequency
- Computer system 700 may additionally include computer display 730 .
- the computer display 730 may also be used as an interactive interface (not shown) displayed via device I/O 706 on client 702 .
- computer program medium “non-transitory computer readable medium,” and “computer usable medium” are used to generally refer to media such as the removable storage unit 718 , the removable storage unit 722 , and a hard disk installed in the hard disk drive 712 . Signals carried over communications path 726 may also embody the logic described herein. Computer program medium and computer usable medium may also refer to memories, such as the main memory 708 and the secondary memory 710 , which may be memory semiconductors, such as DRAMs to provide an example. These computer program products are means for providing software to the computer system 700 .
- Computer programs also referred to as computer control logic, are stored in the main memory 708 and/or the secondary memory 710 .
- the computer programs may also be received via the communications interface 724 .
- Such computer programs when executed, enable the computer system 700 to implement the present invention as discussed herein.
- the computer programs when executed, enable the processor 704 to implement the processes of the present invention as described above. Accordingly, such computer programs represent controllers of the computer system 700 .
- the software may be stored in a computer program product and loaded into the computer system 700 using the removable storage drive 714 , the interface 720 , the hard drive 712 , and/or the communications interface 724 .
- the invention is also directed to computer program products comprising software stored on any computer useable medium.
- Such software when executed in one or more data processing device, causes a data processing device(s) to operate as described herein.
- Embodiments of the invention employ any computer useable or readable medium, known now or in the future.
- Examples of computer useable mediums include, but are not limited to, primary storage devices, such as any type of random access memory to provide an example, secondary storage devices, such as hard drives, floppy disks, CD ROMS, ZIP disks, tapes, magnetic storage devices, optical storage devices, MEMS, and/or nanotechnological storage device to provide some examples, and communication mediums such as wired and wireless communications networks, local area networks, wide area networks, and/or intranets to provide some examples.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
Abstract
Description
Claims (30)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US12/909,508 US8922718B2 (en) | 2009-10-21 | 2010-10-21 | Key generation through spatial detection of dynamic objects |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US25374209P | 2009-10-21 | 2009-10-21 | |
US12/909,508 US8922718B2 (en) | 2009-10-21 | 2010-10-21 | Key generation through spatial detection of dynamic objects |
Publications (2)
Publication Number | Publication Date |
---|---|
US20110102678A1 US20110102678A1 (en) | 2011-05-05 |
US8922718B2 true US8922718B2 (en) | 2014-12-30 |
Family
ID=43925066
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US12/909,508 Active 2033-10-31 US8922718B2 (en) | 2009-10-21 | 2010-10-21 | Key generation through spatial detection of dynamic objects |
Country Status (1)
Country | Link |
---|---|
US (1) | US8922718B2 (en) |
Cited By (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120144304A1 (en) * | 2009-08-12 | 2012-06-07 | Ju Guo | System and method for reducing artifacts in images |
US20140320587A1 (en) * | 2013-04-26 | 2014-10-30 | Ozgur Oyman | Interactive zooming in video conferencing |
US20180048824A1 (en) * | 2012-11-21 | 2018-02-15 | Canon Kabushiki Kaisha | Transmission apparatus, setting apparatus, transmission method, reception method, and storage medium |
Families Citing this family (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9141859B2 (en) | 2008-11-17 | 2015-09-22 | Liveclips Llc | Method and system for segmenting and transmitting on-demand live-action video in real-time |
US9141860B2 (en) | 2008-11-17 | 2015-09-22 | Liveclips Llc | Method and system for segmenting and transmitting on-demand live-action video in real-time |
US20120198500A1 (en) * | 2011-01-31 | 2012-08-02 | Robin Sheeley | Touch screen video production and control system |
US9367745B2 (en) | 2012-04-24 | 2016-06-14 | Liveclips Llc | System for annotating media content for automatic content understanding |
US20130283143A1 (en) | 2012-04-24 | 2013-10-24 | Eric David Petajan | System for Annotating Media Content for Automatic Content Understanding |
US10474921B2 (en) * | 2013-06-14 | 2019-11-12 | Qualcomm Incorporated | Tracker assisted image capture |
WO2015126830A1 (en) * | 2014-02-21 | 2015-08-27 | Liveclips Llc | System for annotating media content for automatic content understanding |
US10290110B2 (en) * | 2016-07-05 | 2019-05-14 | Intel Corporation | Video overlay modification for enhanced readability |
US9754397B1 (en) * | 2017-04-07 | 2017-09-05 | Mirage Worlds, Inc. | Systems and methods for contextual augmented reality sharing and performance |
US10796723B2 (en) * | 2017-05-26 | 2020-10-06 | Immersive Licensing, Inc. | Spatialized rendering of real-time video data to 3D space |
EP3646610A4 (en) * | 2017-06-27 | 2020-11-11 | Pixellot Ltd. | Method and system for fusing user specific content into a video production |
US10504288B2 (en) | 2018-04-17 | 2019-12-10 | Patrick Piemonte & Ryan Staake | Systems and methods for shared creation of augmented reality |
US10932010B2 (en) | 2018-05-11 | 2021-02-23 | Sportsmedia Technology Corporation | Systems and methods for providing advertisements in live event broadcasting |
US11368743B2 (en) * | 2019-12-12 | 2022-06-21 | Sling Media Pvt Ltd | Telestration capture for a digital video production system |
US20220335452A1 (en) * | 2021-04-20 | 2022-10-20 | Walmart Apollo, Llc | Systems and methods for retail facilities |
Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5808695A (en) | 1995-06-16 | 1998-09-15 | Princeton Video Image, Inc. | Method of tracking scene motion for live video insertion systems |
US5953076A (en) | 1995-06-16 | 1999-09-14 | Princeton Video Image, Inc. | System and method of real time insertions into video using adaptive occlusion with a synthetic reference image |
US6100925A (en) | 1996-11-27 | 2000-08-08 | Princeton Video Image, Inc. | Image insertion in video streams using a combination of physical sensors and pattern recognition |
US6252632B1 (en) | 1997-01-17 | 2001-06-26 | Fox Sports Productions, Inc. | System for enhancing a video presentation |
US6529613B1 (en) | 1996-11-27 | 2003-03-04 | Princeton Video Image, Inc. | Motion tracking using image-texture templates |
US20030216922A1 (en) * | 2002-05-20 | 2003-11-20 | International Business Machines Corporation | Method and apparatus for performing real-time subtitles translation |
US20030222994A1 (en) * | 2002-05-28 | 2003-12-04 | Sony Electronics Inc. | Method and apparatus for synchronizing dynamic graphics |
US7015978B2 (en) | 1999-12-13 | 2006-03-21 | Princeton Video Image, Inc. | System and method for real time insertion into video with occlusion on areas containing multiple colors |
US20080303942A1 (en) * | 2001-12-06 | 2008-12-11 | Shih-Fu Chang | System and method for extracting text captions from video and generating video summaries |
US20100259676A1 (en) * | 2009-04-09 | 2010-10-14 | Ati Technologies Ulc | Detection and enhancement of in-video text |
US20110093882A1 (en) * | 2009-10-21 | 2011-04-21 | Candelore Brant L | Parental control through the HDMI interface |
-
2010
- 2010-10-21 US US12/909,508 patent/US8922718B2/en active Active
Patent Citations (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5808695A (en) | 1995-06-16 | 1998-09-15 | Princeton Video Image, Inc. | Method of tracking scene motion for live video insertion systems |
US5953076A (en) | 1995-06-16 | 1999-09-14 | Princeton Video Image, Inc. | System and method of real time insertions into video using adaptive occlusion with a synthetic reference image |
US6100925A (en) | 1996-11-27 | 2000-08-08 | Princeton Video Image, Inc. | Image insertion in video streams using a combination of physical sensors and pattern recognition |
US6529613B1 (en) | 1996-11-27 | 2003-03-04 | Princeton Video Image, Inc. | Motion tracking using image-texture templates |
US6252632B1 (en) | 1997-01-17 | 2001-06-26 | Fox Sports Productions, Inc. | System for enhancing a video presentation |
US7015978B2 (en) | 1999-12-13 | 2006-03-21 | Princeton Video Image, Inc. | System and method for real time insertion into video with occlusion on areas containing multiple colors |
US20080303942A1 (en) * | 2001-12-06 | 2008-12-11 | Shih-Fu Chang | System and method for extracting text captions from video and generating video summaries |
US20030216922A1 (en) * | 2002-05-20 | 2003-11-20 | International Business Machines Corporation | Method and apparatus for performing real-time subtitles translation |
US20030222994A1 (en) * | 2002-05-28 | 2003-12-04 | Sony Electronics Inc. | Method and apparatus for synchronizing dynamic graphics |
US20100259676A1 (en) * | 2009-04-09 | 2010-10-14 | Ati Technologies Ulc | Detection and enhancement of in-video text |
US20110093882A1 (en) * | 2009-10-21 | 2011-04-21 | Candelore Brant L | Parental control through the HDMI interface |
Non-Patent Citations (1)
Title |
---|
U.S. Appl. No. 12/403,857, filed Mar. 13, 2009 (pp. 1-44). |
Cited By (16)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20120144304A1 (en) * | 2009-08-12 | 2012-06-07 | Ju Guo | System and method for reducing artifacts in images |
US10715732B2 (en) * | 2012-11-21 | 2020-07-14 | Canon Kabushiki Kaisha | Transmission apparatus, setting apparatus, transmission method, reception method, and storage medium |
US10194087B2 (en) * | 2012-11-21 | 2019-01-29 | Canon Kabushiki Kaisha | Transmission apparatus, setting apparatus, transmission method, reception method, and storage medium |
US20180359422A1 (en) * | 2012-11-21 | 2018-12-13 | Canon Kabushiki Kaisha | Transmission apparatus, setting apparatus, transmission method, reception method, and storage medium |
US20180048824A1 (en) * | 2012-11-21 | 2018-02-15 | Canon Kabushiki Kaisha | Transmission apparatus, setting apparatus, transmission method, reception method, and storage medium |
US9414306B2 (en) | 2013-03-29 | 2016-08-09 | Intel IP Corporation | Device-to-device (D2D) preamble design |
US9307192B2 (en) * | 2013-04-26 | 2016-04-05 | Intel IP Corporation | Interactive zooming in video conferencing |
US9392539B2 (en) | 2013-04-26 | 2016-07-12 | Intel IP Corporation | User equipment and method for feedback of user equipment performance metrics during dynamic radio switching |
US9621845B2 (en) | 2013-04-26 | 2017-04-11 | Intel IP Corporation | Architecture for web-based real-time communications (WebRTC) to access internet protocol multimedia subsystem (IMS) |
US9743380B2 (en) | 2013-04-26 | 2017-08-22 | Intel IP Corporation | MTSI based UE configurable for video region-of-interest (ROI) signaling |
US9325937B2 (en) | 2013-04-26 | 2016-04-26 | Intel IP Corporation | Radio access technology information storage in a mobile network |
US9294714B2 (en) | 2013-04-26 | 2016-03-22 | Intel IP Corporation | User equipment and methods for adapting system parameters based on extended paging cycles |
US9288434B2 (en) | 2013-04-26 | 2016-03-15 | Intel IP Corporation | Apparatus and method for congestion control in wireless communication networks |
US10225817B2 (en) | 2013-04-26 | 2019-03-05 | Intel IP Corporation | MTSI based UE configurable for video region-of-interest (ROI) signaling |
US10420065B2 (en) | 2013-04-26 | 2019-09-17 | Intel IP Corporation | User equipment and methods for adapting system parameters based on extended paging cycles |
US20140320587A1 (en) * | 2013-04-26 | 2014-10-30 | Ozgur Oyman | Interactive zooming in video conferencing |
Also Published As
Publication number | Publication date |
---|---|
US20110102678A1 (en) | 2011-05-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US8922718B2 (en) | Key generation through spatial detection of dynamic objects | |
US11463678B2 (en) | System for and method of social interaction using user-selectable novel views | |
US10652519B2 (en) | Virtual insertions in 3D video | |
JP6894962B2 (en) | Image data capture method, device, and program for free-viewpoint video | |
US8451265B2 (en) | Virtual viewpoint animation | |
US10515471B2 (en) | Apparatus and method for generating best-view image centered on object of interest in multiple camera images | |
US20130278727A1 (en) | Method and system for creating three-dimensional viewable video from a single video stream | |
US20120013711A1 (en) | Method and system for creating three-dimensional viewable video from a single video stream | |
US20070279494A1 (en) | Automatic Event Videoing, Tracking And Content Generation | |
JP2018504814A (en) | System and method for tracking and tagging targets in broadcast | |
EP3295663B1 (en) | Digitally overlaying an image with another image | |
US9667887B2 (en) | Lens distortion method for broadcast video | |
US7091989B2 (en) | System and method for data assisted chroma-keying | |
Sabirin et al. | Toward real-time delivery of immersive sports content | |
JP6392739B2 (en) | Image processing apparatus, image processing method, and image processing program | |
JP6450306B2 (en) | Image processing apparatus, image processing method, and image processing program | |
KR101911528B1 (en) | Method and system for generating motion data of moving object in viedo | |
JP2005182402A (en) | Field area detection method, system therefor and program | |
JP6411991B2 (en) | Image processing apparatus, image processing method, and image processing program | |
JP2017102784A (en) | Image processing system, image processing method and image processing program | |
JP2005229451A (en) | Video display device, video display method, program for video display method, and recording medium recorded with program for video display method |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: ESPN TECHNOLOGY SERVICES, INC., CONNECTICUT Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PVI VIRTUAL MEDIA SERVICES, LLC;REEL/FRAME:026054/0053 Effective date: 20101210 |
|
AS | Assignment |
Owner name: DISNEY ENTERPRISES, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ESPN TECHNOLOGY SERVICES, INC.;REEL/FRAME:026061/0159 Effective date: 20110330 |
|
AS | Assignment |
Owner name: PVI VIRTUAL MEDIA SERVICE, LLC., NEW YORK Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:HOUSE, GREGORY;GONG, XIMIN;RAO, CEN;AND OTHERS;REEL/FRAME:026838/0783 Effective date: 20101119 |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551) Year of fee payment: 4 |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 8 |