NZ719619B2 - Selective capture and presentation of native image portions - Google Patents

Abstract

system and method for selective capture of and presentation of native image portions are provided. In a first embodiment a first image or video is captured at a first resolution, which resolution is greater than high definition and higher than a predetermined broadcast display resolution. A desired portion of the native, first image or video is then displayed at a second, lower resolution, which resolution is less than and closer to the predetermined broadcast display resolution. Accordingly, a selected portion of the captured image may be displayed at or near the predetermined broadcast display resolution. In other exemplary embodiments, video is captured at a first frame rate that is higher than a predetermined broadcast frame rate, and an extraction window extracts frames from a portion of the native, first video to provide improved broadcast quality of the portion(s) selected by the extraction window. d portion of the native, first image or video is then displayed at a second, lower resolution, which resolution is less than and closer to the predetermined broadcast display resolution. Accordingly, a selected portion of the captured image may be displayed at or near the predetermined broadcast display resolution. In other exemplary embodiments, video is captured at a first frame rate that is higher than a predetermined broadcast frame rate, and an extraction window extracts frames from a portion of the native, first video to provide improved broadcast quality of the portion(s) selected by the extraction window.

Description

IVE CAPTURE AND PRESENTATION OF NATIVE IMAGE PORTIONS CROSS REFERENCE TO RELATED APPLICATIONS This is a further application for a patent for subject matter disclosed in New Zealand patent application no. 620992. reference is also made to United States provisional patent applications nos. 61/515,549 and 61/563,126, the entire contents of which are specifically incorporated herein by reference.

BACKGROUND The present sure relates to systems and methods for selective capture of and presentation of native image portions, ularly for use in broadcast production.

Common image or video formats are typically referred to either in terms of vertical resolution or horizontal resolution. FIGURE 1 shows an example of relative pixel dimensions at a 2.39:1 aspect ratio, with 720p and 1080p formats being letterboxed.

Examples of vertical high resolution designators are 720p (1280 x 720 pixels), 1080i (utilizing an interlace of two fields of 1920 x 540 pixels for a total resolution of 1920 x 1080 ) or 1080p (representing a progressive scan of 1920 x 1080 pixels).

Examples of horizontal high resolution designators, which are more common to digital cinema terminology, e 2K (2048 pixels wide) and 4K (4096 pixels wide).

Overall tion would depend on the image aspect ratio, e. g. a 2K image with a rd or Academy ratio of 4:3 would have an overall ratio of 2048 x 1536 pixels, s an image with a Panavision ratio of 2.39:1 would have an overall ratio of 2048 x 856 pixels. PRIOR ART FIGURE 1 illustrates a comparison of relative pixel dimensions for 720p, 1080p, 2K and 4K captured images.

Currently, technologies exist for greater than high definition capture for l cinema, e.g. up to 2K, 4K and beyond. However, for consumer home viewing of the captured digital cinema, the captured image is compressed down at the buting studio to a version that is specific to traditional usable consumer high definition formats for broadcast or other distribution, e.g., at 720p, 1080i or 1080p.

Also, while digital cinema has utilized large resolution capture, traditional ast capture has not. This broadcast capture is performed at the desired consumer display resolution, e.g., 1080p, both due to limitations at the consumer display device as well as to bandwidth restrictions of broadcast carriers. Thus, in scenarios calling for magnification of the broadcast image, for example to better show line calls or to follow specific players on the field, the display resolution of the line calls is considerably less than the native image captured on the field. ingly, there is a need in the art for improved mechanisms for capturing and presenting image material for broadcasts or other image presentation.

SUMMARY In one broad form, the ion provides a method for selective capture and tation of native broadcast image portions, comprising: capturing a first broadcast image or video at a first resolution, which resolution is greater than high ion and higher than a predetermined second, output broadcast display resolution; selecting a first desired broadcast portion of the captured, native first broadcast image or video, wherein the first portion is at a resolution lower than that of the captured first broadcast image or video; wherein selecting a d first broadcast portion of the first image or video is ed by a graphical user interface having a selectable extraction window, and wherein the extraction window is configured to allow an operator to navigate within the captured broadcast image or video and select broadcast portions thereof for tation; and displaying the selected first ast portion at the second, output resolution.

In another broad form, the invention provides system for selective capture of and presentation of native image broadcast portions, comprising: a first broadcast camera configured to capture a first broadcast image or video at a first resolution, which resolution is greater than high definition and higher than a predetermined , broadcast output display resolution; a processor in communication with a graphical user interface, the interface configured to select a first desired broadcast portion of the , first broadcast image or video, wherein the first broadcast portion is at a resolution lower than that of the captured first image or video; n the graphical user interface has a selectable extraction , and wherein the extraction window is configured to allow an operator to navigate within the captured broadcast image or video and select ast portions thereof for presentation ; and an output mechanism configured to transport the selected first broadcast portion to a router, switcher or server at the second, broadcast output resolution.

The first resolution is preferably at or greater than 2K, and more preferably at or r than 4K.

In an embodiment, the second, output resolution is a high ion resolution.

The ed first broadcast portion of the first image or video may be reconstituted to the second, output resolution. The first broadcast resolution may be r than high definition resolution, and the second, output broadcast y resolution may be a high definition resolution.

The extraction window may be configured such that it can be adjusted by size and on within the captured image or video. The extraction window may also be configured to track or scan across moving images, such as to follow a t of interest over time.

A second desired portion of the first image or video may be ed which is at a resolution lower than that of the captured first image or video. The first and second broadcast ns may be selected via separate extraction windows. The separate extraction windows may concurrently select different broadcast portions of the first image or Video. The separate extraction windows may be provided via the same graphical user ace.

A second broadcast image or video may be captured at a third resolution which is higher than the predetermined, second output broadcast display resolution. A first desired broadcast portion of the second broadcast image or Video may be selected, wherein the first broadcast portion of the second broadcast image or video is at a tion lower than that of the captured second broadcast image or Video. The selected first broadcast portion of the second broadcast image or video may be displayed at the second, broadcast output resolution.

In an ment, the first broadcast resolution and the third broadcast resolution are the same, and the second broadcast image or video is provided via a separate camera from the first broadcast image or video and at a different point of view. Graphical user interfaces be provided for each camera, and at least one extraction window may be associated with each graphical user interface to facilitate selection of desired broadcast image or video portions.

In an embodiment, the capturing of a first ast image or video occurs at a desired broadcast venue, followed by transmission of the captured first broadcast image or video to an offsite operations base. The captured first broadcast image or video may be recorded at the operations base. The first d broadcast portion of the first broadcast image or video may be selected at the operations base, and the selected first broadcast portion may be output to a router, switcher or server at the second, broadcast output resolution.

In another embodiment, the capturing of a first broadcast image or video and recording of the captured first broadcast image or video occur at a desired ast venue, followed by outputting of the selected first broadcast n to a router, switcher or server at the second, broadcast output resolution.

The above discussed and other features and advantages of the present invention will be appreciated and tood by those skilled in the art from the ing detailed description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS Referring now to the drawings, n like ts are numbered alike in the following S: PRIOR ART FIGURE 1 is a diagram comparing relative pixel dimensions of high definition and greater than high definition images; FIGURE 2 is an exemplary graphical user interface of a 4K captured image with a 720p selectable extraction window; FIGURE 3 is an exemplary first system for capturing and transporting a 4K image to an offsite processor and graphical user interface; and FIGURE 4 is an exemplary second system for capturing and processing a 4K image onsite, followed by transport of a high definition image offsite.

DETAILED PTION As was noted above, the present disclosure relates to a system and method for ive capture of and presentation of native image portions.

In exemplary embodiments, a first image or video is captured at a first resolution, which resolution is greater than high ion and higher than a predetermined ast display resolution. A desired portion of the first image or video is then displayed at a second, lower resolution, which resolution is less than and closer to the predetermined broadcast display resolution. Accordingly, a selected portion of the captured image may be displayed at or near the predetermined broadcast display resolution (i.e., minimizing or eliminating loss of image detail relative to the predetermined broadcast display resolution).

An example of this is rated at FIGURE 2, which shows a screenshot of a full- raster 4K moving video image 10. A portion of the 4K image, rated as a 720p moving video selectable extraction window 12, is then selected for presentation. Thus, native image capture occurs at a greater than high definition resolution, and portions of that greater than high definition image are selected for presentation via the 720p extraction . While, FIGURE 2 specifically illustrates 4K capture and a 720p extraction window, it should be recognized that both or either of the captured image and extraction window may be provided at or sized to other resolutions.

Also, while one extraction window is illustrated in FIGURE 2, the present disclosure contemplates simultaneous multiple extraction s that may be applied to the same captured image.

In further exemplary embodiments, the selectable extraction window (12 in FIGURE 2) is provided at a graphical user interface (“GUI”) (14 in FIGURES 3 and 4) that is configured to allow an operator to navigate within a captured image and select portions of the captured image for presentation. In exemplary embodiments, the tion window is configured to allow the operator to adjust the size and position of the tion window. In other exemplary embodiments, the extraction window is configured to track or scan across moving images, e.g., to follow a play or subject of interest during a sporting event. In other exemplary embodiments, plural operators may extract from the same images via the same or Via plural GUIs.

Referring now to FIGURES 3 and 4, processing of the captured images may occur either offsite (FIGURE 3) or onsite (FIGURE 4). Referring to FIGURE 3, an exemplary system is illustrated wherein a camera 16 captures 4K images onsite, e. g., at a field (shown generally at 18) for a sporting event. A transport mechanism 20, e.g. a fiber capable of orting a full dth 4K video, transports the captured images to an operations base (“OB”) (shown generally at 22), e. g., a production truck away from the field 18.

An image recorder 24 records the captured images, e.g., as a data stream on a server, and is configured to allow an operator to go back in time relative to the recording and examine selected ns of the captured image as described above. Such control is provided to an or via the GUI 14 through a processor 26 interfacing with the GUI l4 and recorder 24. In exemplary embodiments, the recorder, processor and GUI are configured to allow the operator to go back instantaneously or near~instantaneously to select ns of the recorded image for presentation.

For example, with regard to FIGURE 2, an operator in a truck would use a GUI to navigate the full raster 4K image and maneuver the selective 16:9 extraction window, in a manner similar to a cursor, to select an area of interest. In exemplary embodiments, the GUI is red such that the extraction window may select an area of interest in one or both of live and recorded video. Also, as has been noted above, the present disclosure plates sizing and zooming lities for the extraction window. In other exemplary embodiments, the system is configured to mark keyframes and establish mapping for desired moves, e.g., pans and zooms, among others, around the image.

Referring again to FIGURE 3, in exemplary embodiments, the output 28 of the system (e.g., a 720p/59.94 output relative to a 4K capture) is provided to a router 30 that allows the output to be taken live to a switcher 32 or to be ingested at a server 34 (“EVS”) for later t. Also, in exemplary ments, a resulting image can be slowed down for replay or rendered as a still image, if desired, either at the server 34 or at the operator’s position (via processor 26).

FIGURE 4 es an alternate ary embodiment, wherein capture, transport and ing of the native image (in this example 4K images) occurs onsite, e.g., at the field 18 of a sporting event). An onsite processor 26 es or interfaces with an operator GUI 14 in an operations base 22 (e.g., a truck, though the GUI could be accessed from any convenient location) and provides a reference video 38 of the image to allow the or to navigate the image via the extraction window. The output 28 is then transported from the field to an offsite router 30.

In another embodiment, at least one GUI is accessed by a tablet controller as a navigation tool for the system. Such tablet controller may be wireless and portable to allow for e a primary or mental navigation tool.

In other exemplary embodiments, multiple cameras may be positioned to capture images from different points of view, and extraction windows may be provided relative to the multiple image captures in a system for selectively displaying portions of native images from different points of View.

Further exemplary embodiments provide real time or near real time tracking of subjects of interest (e.g., identified, selected or pre-tagged players of interest or automatic tracking of a ball in a game). onal exemplary embodiments also provide virtual directing of operated and automatically tracked subjects of interest for cutting into a full live broadcast, utilizing backend software and tracking technology to provide a virtual viewfinder that operates in manners similar to otherwise human camera operators. Such processes may also use artificial technology for simple tracking, e.g., of a single identified object, or for more complex operations approximating motions utilized by human camera operators, e.g., pan, tilt and zoom of the extraction window in a manner similar to human operators. For those examples using 4K (or the like) capture, camera e could utilize a cally designed 4K camera. A camera may also use wider lensing to capture more of the subject, with possible reconstituting or flattening in post production. Also, different lensing can be used specific to different applications.

Such ses may use the above—described multiple cameras and/or multiple extraction windows, or may run with specific regard to one camera and/or one extraction window. In such a way, an artificial intelligence can automatically capture, extract and display material for broadcast, utilizing the extraction window(s) as virtual viewfinders.

Additional exemplary ments also provide for virtual 3D extraction, e.g. via single camera at 4K or 8K with a two window output.

In other exemplary embodiments, an increased image capture frame rates relative to a broadcast frame rate along with or in lieu of an increased image capture tion, as has been discussed above.

In such embodiments, a first video is captured at a first frame rate, which frame rate is higher than a predetermined ast frame rate. A desired portion of the first video is then displayed at a second, lower frame rate, which frame rate is less than and closer to the predetermined broadcast frame rate. The desired portion of the first video is captured by an extraction window that extracts frames across the native captured video. In such a way, the extracted video provides smooth and clear video, without edgy or blurred frames. Such ed first video may be at any frame rate that is above the ermined broadcast frame rate.

In further exemplary embodiments, the first video is ed at a first frame rate that is in super motion or hyper . In traditional video, this equates to imately 180 (“supermotion”) frames per second or above (“hypermotion” or “ultramotion”) in a progressive frame rate. In exemplary embodiments, hypermotion is ed in discrete times sufficient to capture a triggered instance of an action of camera subject for playback. In other exemplary embodiments, the present system performs a full time record of a camera in hypermotion, e.g., of sufficient length for replay ck archiving, such as more than fifteen minutes, more than thirty minutes, more than an hour, more than an hour and a half, or more than two hours, among others.

In other exemplary embodiments, raw data from at least one camera is lated to adjust the image y (make it “paintable”) to broadcast specifications. In exemplary embodiments, broadcast es” may be integrated into the system to affect the raw data in a manner that is more germane to broadcast color temperatures, hues and gamma variables.

The present disclosure thus advantageously provides systems and methods for selective capture of and presentation of native image portions, for broadcast production or other applications. By providing exemplary embodiments using a selectable extraction window through a GUI, an operator has te control over portions within the native images that the operator desires for presentation. Also, by providing exemplary embodiments with image capture greater than high definition (e.g., 4K), desired portions of the image selected by an operator may be presented at or relatively near high definition quality (i.e., without relative degradation of image quality). Further, by providing exemplary embodiments with image capture frame rates greater than that of a predetermined broadcast frame rate, extracted Video rom provides smooth and clear Video, without edgy or blurred frames. Finally, various exemplary ments utilizing enhanced GUI features, such as automatic tracking of subjects of interests, plural GUIs or extraction windows for one or plural (for different points of view) captured images provide advantageous production flexibilities and ages.

It will be apparent to those skilled in the art that, while exemplary embodiments have been shown and described, s ations and variations can be made to the invention disclosed herein without departing from the spirit or scope of the invention. Also, the exemplary implementations described above should be read in a non—limiting fashion, both with regard to uction and methodology. Accordingly, it is to be tood that the various embodiments have been described by way of illustration and not limitation.

Claims (39)

1. A method for selective capture of and presentation of native broadcast image portions in a ast multi- camera system, comprising: with a first broadcast camera, capturing a first broadcast Video at a first native capture resolution, which native capture resolution is greater than high definition resolution and higher than a predetermined second, output broadcast display resolution; and ing a first desired broadcast portion of the captured, native first broadcast video, wherein said first portion is at a tion lower than that of the captured first broadcast video; wherein said first desired broadcast portion is selected Via an extraction window that ts frames across the native captured video with a second broadcast camera, capturing a second broadcast Video at said first native capture resolution, which native capture resolution is greater than high definition resolution and higher than the predetermined second output broadcast display tion; selecting a second desired broadcast portion of the captured, native second broadcast Video from the second broadcast camera, wherein said second portion is at a tion lower than that of the captured second broadcast Video, wherein said second desired broadcast portion is selected via the tion window that extracts frames across the native captured video; manipulating raw data from said first broadcast video of said first broadcast camera and of said second broadcast video of said second broadcast camera to make the image quality paintable to broadcast specifications; and broadcasting one of said selected first and second ast portion at said second, output resolution.

2. A method in accordance with claim 1, wherein said first resolution is at or r than 2K.

3. A method in accordance with claim 2, n said first resolution is at or greater than 4K.

4. A method in accordance with claim 2, wherein said second, output broadcast resolution is a high ion resolution.

5. A method in accordance with claim 1, further comprising reconstituting said selected first broadcast n of the first image or video to said second, output tion.

6. A method in accordance with claim 5, wherein said first broadcast resolution is a greater than high definition tion and wherein said second, output broadcast display resolution is a high definition resolution.

7. A method in ance with claim 1, wherein said ing of a desired first broadcast portion of the first image or video is provided by a graphical user interface having the extraction window.

8. A method in accordance with claim 7, wherein said extraction window is red to allow an operator to navigate within said ed broadcast image or video and select broadcast portions thereof for presentation.

9. A method in accordance with claim 7, wherein the extraction window is configured such that it can be adjusted by size and position within the captured broadcast image or video.

10. A method in accordance with claim 7, wherein the extraction window is configured to track or scan across moving broadcast images.

11. A method in ance with claim 10, wherein said extraction window is configured to follow a subject of interest over time.

12. A method in accordance with claim 1, wherein said first and second broadcast portions are selected via separate extraction windows.

13. A method in accordance with claim 12, wherein said separate extraction windows concurrently select different broadcast portions of said first image or video.

14. A method in accordance with claim 1, r comprising: capturing a third broadcast image or video at a third resolution, which resolution is higher than said predetermined, second output broadcast display resolution; selecting a first desired ast portion of the third broadcast image or video, wherein said first broadcast portion of said third broadcast image or video is at a resolution lower than that of the captured second broadcast image or Video; and displaying said selected first ast portion of said third broadcast image or video at said second, broadcast output resolution.

15. A method in accordance with claim 14, wherein said first broadcast tion and said third broadcast tion are the same, and wherein said third broadcast image or video is provided via a separate camera from said first broadcast image or video and at a different point of View.

16. A method in accordance with claim 15, wherein graphical user interfaces are provided for each camera, and wherein at least one extraction window is ated with each graphical user interface to facilitate selection of desired broadcast image or video portions.

17. A method in accordance with claim 1, wherein said capturing of a first broadcast image or video occurs at a desired ast venue, followed by ission of said captured first broadcast image or video to an offsite operations base.

18. A method in accordance with claim 17, further comprising: recording of said captured first broadcast image or video at said operations base; selecting said first desired broadcast portion of the first broadcast image or video at said operations base; and outputting said selected first broadcast portion to a , switcher or server at said second, broadcast output resolution.

19. A method in accordance with claim 1, wherein said capturing of a first broadcast image or video and ing of said captured first ast image or video occurs at a desired broadcast venue, followed by outputting said selected first broadcast portion to a router, switcher or server at said second, broadcast output resolution.

20. A system for selective capture of and presentation of native image broadcast portions in a broadcast multi-camera system, comprising: a first broadcast camera configured to capture a first broadcast image or video at a first resolution, which resolution is greater than high definition resolution and higher than a ermined second, broadcast output display resolution; a second broadcast camera configured to capture a second broadcast image or video at a first resolution, which resolution is greater than high definition re solution and higher than the predetermined , broadcast output display resolution; a processor in communication with a graphical user interface, said interface configured to select a first desired broadcast portion of the native, first broadcast image or video, wherein said first broadcast portion is at a resolution lower than that of the ed first broadcast image or Video and to key frame at least said selected ast portion by marking key frames, wherein said processor is also red to manipulate raw data from said first and second broadcast images or video to make the image y paintable to broadcast specifications, including configuring ast handles to affect the raw data to adjust the raw data to broadcast color temperatures, hues and gamma variables; and an output mechanism red to transport said selected first broadcast portion to a router, switcher or server at said second, broadcast output resolution.

21. A system in accordance with claim 20, wherein said first resolution is at or greater than 2K.

22. A system in accordance with claim 21, wherein said first resolution is at or greater than 4K.

23. A system in accordance with claim 21, wherein said , output broadcast resolution is a high definition resolution.

24. A system in accordance with claim 20, wherein said processor is configured to reconstitute said ed first broadcast portion of the first ast image or video to said second, broadcast output resolution.

25. A system in accordance with claim 24, wherein said first resolution is a greater than high definition tion and wherein said second, broadcast output resolution is a high ion resolution.

26. A system in accordance with claim 20, wherein said graphical user interface has an extraction window.

27. A system in accordance with claim 26, wherein said extraction window is configured to allow an operator to navigate within said captured broadcast image or video and select broadcast portions thereof for presentation.

28. A system in accordance with claim 26, wherein the extraction window is configured such that it can be adjusted by size and position within the captured image or video.

29. A system in accordance with claim 26, wherein the extraction window is configured to track or scan across moving broadcast images.

30. A system in accordance with claim 29, wherein said extraction window is configured to follow a subject of interest over time.

31. A system in accordance with claim 30, further comprising a second extraction window configured to select a second desired broadcast portion of the first image or video, wherein said second broadcast portion is at a resolution lower than that of the captured first broadcast image or video.

32. A system in accordance with claim 31, wherein said second extraction window is provided via the same graphical user interface.

33. A system in accordance with claim 31, wherein said second extraction window is configured to concurrently select different ast portions of said first image or video.

34. A system in accordance with claim 20, wherein said interface is configured to select a first desired broadcast portion of the second broadcast image or video, n said first broadcast portion of said second broadcast image or video is at a tion lower than that of the captured second broadcast image or video.

35. A system in ance with claim 34, wherein said first resolution and said third resolution are the same, and wherein said second image or video is ed via a separate broadcast camera from said first image or video and at a different point of View.

36. A system in accordance with claim 35, wherein graphical user aces are provided for each broadcast camera, and wherein at least one extraction window is associated with each graphical user interface to facilitate selection of desired broadcast image or video portions.

37. A system in accordance with claim 20, wherein said broadcast camera is configured to capture the first broadcast image or video at a d ast venue, and n a transmission mechanism is configured to transmit said captured first ast image or video to an offsite operations base.

38. A system in accordance with claim 37, further comprising: a recording mechanism at said ions base, said recording mechanism being configured to record said captured first broadcast image or video, wherein said graphical user interface is provided at said operations base, and wherein said output mechanism is red to transport said selected first broadcast portion to a separate router, er or SCI'VCEI‘.

39. A system in accordance with claim 20, wherein said camera and output mechanism are provided at said desired broadcast venue.