SELECTIVE CAPTURE AND PRESENTATION OF NATIVE IMAGE PORTIONS
CROSS REFERENCE TO D APPLICATIONS
This is a r application for a patent for subject matter disclosed in New
Zealand patent application no. . Cross-reference is also made to United States
provisional patent applications nos. 61/515,549 and 61/563,126, the entire contents of which
are specifically incorporated herein by reference.
BACKGROUND
The present disclosure relates to systems and methods for selective capture of and
presentation of native image portions, particularly for use in broadcast production.
Common image or video formats are typically referred to either in terms of al
resolution or horizontal resolution. FIGURE 1 shows an example of relative pixel dimensions
at a 2.39:1 aspect ratio, with 720p and 1080p formats being letterboxed.
Examples of vertical high resolution designators are 720p (1280 x 720 pixels),
1080i (utilizing an interlace of two fields of 1920 x 540 pixels for a total resolution of 1920 x
1080 pixels) or 1080p (representing a progressive scan of 1920 x 1080 pixels).
Examples of horizontal high resolution designators, which are more common to
digital cinema terminology, include 2K (2048 pixels wide) and 4K (4096 pixels wide).
Overall resolution would depend on the image aspect ratio, e.
g. a 2K image with a Standard or
Academy ratio of 4:3 would have an overall ratio of 2048 x 1536 pixels, whereas an image
with a Panavision ratio of 2.39:1 would have an overall ratio of 2048 x 856 pixels. PRIOR
ART FIGURE 1 rates a comparison of ve pixel dimensions for 720p, 1080p, 2K
and 4K captured images.
Currently, technologies exist for greater than high definition capture for digital
cinema, e.g. up to 2K, 4K and beyond. However, for er home viewing of the captured
l cinema, the ed image is ssed down at the distributing studio to a version
that is specific to traditional usable consumer high ion formats for broadcast or other
bution, e.g., at 720p, 1080i or 1080p.
Also, while digital cinema has utilized large resolution capture, traditional
broadcast capture has not. This broadcast capture is performed at the desired consumer
display resolution, e.g., 1080p, both due to limitations at the consumer display device as well
as to dth restrictions of broadcast carriers. Thus, in scenarios calling for magnification
of the broadcast image, for example to better show line calls or to follow specific players on
the field, the display resolution of the line calls is considerably less than the native image
captured on the field.
Accordingly, there is a need in the art for improved mechanisms for capturing and
ting image al for broadcasts or other image presentation.
SUMMARY
In one broad form, the invention es a method for selective capture and
presentation of native broadcast image portions, comprising: capturing a first broadcast image
or video at a first resolution, which resolution is greater than high definition and higher than a
predetermined second, output broadcast display resolution; selecting a first desired broadcast
portion of the captured, native first broadcast image or video, n the first portion is at a
tion lower than that of the captured first broadcast image or video; wherein selecting a
desired first broadcast portion of the first image or video is provided by a cal user
interface having a selectable extraction window, and wherein the extraction window is
configured to allow an operator to te within the captured broadcast image or video and
select broadcast portions thereof for presentation; and displaying the selected first broadcast
portion at the second, output resolution.
In another broad form, the invention provides system for selective capture of and
presentation of native image broadcast portions, sing: a first ast camera
configured to capture a first ast image or video at a first resolution, which resolution is
greater than high definition and higher than a predetermined second, broadcast output display
resolution; a processor in communication with a cal user interface, the interface
configured to select a first desired broadcast portion of the native, first broadcast image or
video, n the first broadcast portion is at a resolution lower than that of the ed first
image or video; wherein the graphical user interface has a able extraction window, and
wherein the extraction window is configured to allow an operator to navigate within the
captured broadcast image or video and select broadcast portions thereof for presentation ; and
an output mechanism configured to transport the selected first broadcast portion to a router,
switcher or server at the second, broadcast output resolution.
The first resolution is preferably at or greater than 2K, and more preferably at or
greater than 4K.
In an embodiment, the second, output resolution is a high ion resolution.
The selected first broadcast portion of the first image or video may be
reconstituted to the second, output resolution. The first broadcast resolution may be greater
than high definition resolution, and the second, output broadcast y resolution may be a
high definition resolution.
The extraction window may be configured such that it can be adjusted by size and
position within the captured image or video. The extraction window may also be configured
to track or scan across moving images, such as to follow a subject of interest over time.
A second d portion of the first image or video may be selected which is at a
resolution lower than that of the captured first image or video. The first and second broadcast
portions may be selected via te tion windows. The separate extraction windows
may concurrently select different ast portions of the first image or Video. The separate
extraction s may be provided via the same graphical user interface.
A second broadcast image or video may be ed at a third resolution which is
higher than the predetermined, second output broadcast display resolution. A first desired
broadcast portion of the second broadcast image or Video may be selected, wherein the first
broadcast portion of the second broadcast image or video is at a resolution lower than that of
the captured second broadcast image or Video. The ed first broadcast n of the
second broadcast image or video may be displayed at the second, broadcast output tion.
In an embodiment, the first broadcast resolution and the third broadcast resolution are the
same, and the second broadcast image or video is provided via a separate camera from the
first broadcast image or video and at a different point of view. Graphical user interfaces
be provided for each camera, and at least one extraction window may be associated with each
graphical user interface to facilitate selection of desired broadcast image or video portions.
In an embodiment, the capturing of a first broadcast image or video occurs at a
desired broadcast venue, followed by transmission of the captured first broadcast image or
video to an e operations base. The captured first broadcast image or video may be
recorded at the operations base. The first desired broadcast portion of the first broadcast
image or video may be selected at the operations base, and the selected first ast portion
may be output to a router, switcher or server at the second, broadcast output resolution.
In another embodiment, the capturing of a first broadcast image or video and
recording of the captured first broadcast image or video occur at a desired ast venue,
followed by ting of the selected first ast portion to a router, switcher or server at
the second, broadcast output resolution.
The above discussed and other features and advantages of the present invention
will be appreciated and understood by those skilled in the art from the following detailed
description and drawings.
BRIEF DESCRIPTION OF THE GS
Referring now to the drawings, wherein like elements are numbered alike in the
following FIGURES:
PRIOR ART FIGURE 1 is a diagram comparing relative pixel dimensions of high
definition and greater than high definition images;
FIGURE 2 is an exemplary graphical user interface of a 4K captured image with a
720p selectable extraction window;
FIGURE 3 is an exemplary first system for capturing and transporting a 4K image
to an offsite processor and graphical user interface; and
FIGURE 4 is an exemplary second system for capturing and processing a 4K
image onsite, followed by transport of a high definition image offsite.
DETAILED DESCRIPTION
As was noted above, the present disclosure relates to a system and method for
selective capture of and presentation of native image portions.
In exemplary embodiments, a first image or video is captured at a first tion,
which resolution is greater than high ion and higher than a predetermined broadcast
display resolution. A desired n of the first image or video is then displayed at a second,
lower resolution, which resolution is less than and closer to the predetermined broadcast
display resolution. Accordingly, a ed portion of the captured image may be yed at
or near the predetermined broadcast y resolution (i.e., minimizing or eliminating loss of
image detail relative to the predetermined broadcast display resolution).
An example of this is illustrated at FIGURE 2, which shows a screenshot of a full-
raster 4K moving video image 10. A portion of the 4K image, illustrated as a 720p moving
video selectable extraction window 12, is then selected for presentation. Thus, native image
e occurs at a greater than high definition resolution, and portions of that greater than
high ion image are selected for presentation via the 720p extraction window. While,
FIGURE 2 specifically illustrates 4K capture and a 720p extraction window, it should be
recognized that both or either of the captured image and tion window may be provided
at or sized to other resolutions.
Also, while one tion window is illustrated in FIGURE 2, the present
disclosure contemplates simultaneous multiple extraction windows that may be applied to the
same captured image.
In further exemplary ments, the selectable extraction window (12 in
FIGURE 2) is provided at a graphical user interface (“GUI”) (14 in S 3 and 4) that is
configured to allow an operator to navigate within a ed image and select portions of the
captured image for presentation. In exemplary ments, the extraction window is
configured to allow the operator to adjust the size and position of the extraction window. In
other ary embodiments, the extraction window is configured to track or scan across
moving images, e.g., to follow a play or subject of interest during a sporting event. In other
exemplary embodiments, plural operators may extract from the same images via the same or
Via plural GUIs.
Referring now to FIGURES 3 and 4, processing of the captured images may occur
either e (FIGURE 3) or onsite (FIGURE 4). Referring to FIGURE 3, an exemplary
system is illustrated wherein a camera 16 captures 4K images onsite, e. g., at a field (shown
generally at 18) for a sporting event. A transport mechanism 20, e.g. a fiber capable of
transporting a full bandwidth 4K video, transports the captured images to an operations base
(“OB”) (shown generally at 22), e. g., a production truck away from the field 18.
An image recorder 24 records the ed images, e.g., as a data stream on a
server, and is configured to allow an or to go back in time relative to the recording and
examine selected portions of the captured image as described above. Such control is provided
to an operator via the GUI 14 through a processor 26 interfacing with the GUI l4 and recorder
24. In exemplary embodiments, the recorder, processor and GUI are configured to allow the
operator to go back taneously or near~instantaneously to select portions of the recorded
image for presentation.
For example, with regard to FIGURE 2, an operator in a truck would use a GUI to
navigate the full raster 4K image and maneuver the selective 16:9 extraction , in a
manner similar to a cursor, to select an area of interest. In exemplary embodiments, the GUI
is configured such that the extraction window may select an area of interest in one or both of
live and recorded video. Also, as has been noted above, the present disclosure contemplates
sizing and zooming capabilities for the extraction . In other exemplary embodiments,
the system is configured to mark keyframes and establish mapping for desired moves, e.g.,
pans and zooms, among others, around the image.
Referring again to FIGURE 3, in exemplary embodiments, the output 28 of the
system (e.g., a 720p/59.94 output relative to a 4K capture) is provided to a router 30 that
allows the output to be taken live to a er 32 or to be ingested at a server 34 (“EVS”) for
later playout. Also, in exemplary embodiments, a resulting image can be slowed down for
replay or rendered as a still image, if desired, either at the server 34 or at the operator’s
position (via processor 26).
FIGURE 4 provides an alternate exemplary embodiment, wherein e,
transport and recording of the native image (in this example 4K images) occurs onsite, e.g., at
the field 18 of a sporting event). An onsite sor 26 es or interfaces with an
operator GUI 14 in an operations base 22 (e.g., a truck, though the GUI could be accessed
from any convenient location) and es a reference video 38 of the image to allow the
operator to te the image via the extraction window. The output 28 is then transported
from the field to an offsite router 30.
In another embodiment, at least one GUI is accessed by a tablet controller as a
navigation tool for the . Such tablet controller may be wireless and portable to allow
for flexible a primary or supplemental tion tool.
In other exemplary embodiments, multiple cameras may be positioned to capture
images from different points of view, and extraction windows may be provided relative to the
multiple image captures in a system for selectively displaying portions of native images from
different points of View.
Further ary embodiments provide real time or near real time tracking of
subjects of interest (e.g., identified, selected or gged players of interest or automatic
tracking of a ball in a game). Additional exemplary embodiments also provide virtual
directing of operated and automatically tracked subjects of interest for cutting into a full live
ast, utilizing backend software and tracking technology to provide a virtual viewfinder
that operates in manners similar to otherwise human camera operators. Such processes may
also use artificial technology for simple tracking, e.g., of a single identified object, or for
more complex ions approximating motions utilized by human camera operators, e.g.,
pan, tilt and zoom of the extraction window in a manner similar to human operators. For
those examples using 4K (or the like) capture, camera capture could utilize a specifically
designed 4K camera. A camera may also use wider lensing to capture more of the subject,
with possible reconstituting or flattening in post production. Also, different lensing can be
used specific to different applications.
Such processes may use the above—described multiple cameras and/or le
extraction windows, or may run with specific regard to one camera and/or one extraction
window. In such a way, an artificial intelligence can automatically capture, extract and
y al for broadcast, ing the extraction window(s) as virtual ders.
Additional ary embodiments also provide for virtual 3D extraction, e.g. via
single camera at 4K or 8K with a two window output.
In other ary embodiments, an increased image capture frame rates relative
to a broadcast frame rate along with or in lieu of an increased image capture resolution, as has
been discussed above.
In such embodiments, a first video is captured at a first frame rate, which frame
rate is higher than a predetermined broadcast frame rate. A desired portion of the first video
is then displayed at a second, lower frame rate, which frame rate is less than and closer to the
predetermined broadcast frame rate. The desired portion of the first video is captured by an
extraction window that ts frames across the native captured video. In such a way, the
extracted video provides smooth and clear video, without edgy or blurred frames. Such
captured first video may be at any frame rate that is above the predetermined ast frame
rate.
In further exemplary embodiments, the first video is captured at a first frame rate
that is in super motion or hyper motion. In traditional video, this equates to approximately
180 (“supermotion”) frames per second or above (“hypermotion” or “ultramotion”) in a
progressive frame rate. In ary embodiments, hypermotion is recorded in discrete times
sufficient to capture a triggered instance of an action of camera subject for playback. In other
exemplary embodiments, the present system performs a full time record of a camera in
hypermotion, e.g., of sufficient length for replay playback ing, such as more than fifteen
minutes, more than thirty minutes, more than an hour, more than an hour and a half, or more
than two hours, among .
In other exemplary embodiments, raw data from at least one camera is
manipulated to adjust the image quality (make it “paintable”) to ast specifications. In
exemplary embodiments, broadcast “handles” may be integrated into the system to affect the
raw data in a manner that is more germane to broadcast color temperatures, hues and gamma
variables.
The present disclosure thus advantageously provides systems and methods for
ive capture of and presentation of native image portions, for broadcast production or
other applications. By providing exemplary embodiments using a selectable extraction
window through a GUI, an operator has te control over ns within the native
images that the operator desires for presentation. Also, by providing exemplary embodiments
with image capture r than high definition (e.g., 4K), d portions of the image
selected by an operator may be presented at or relatively near high definition quality (i.e.,
without relative degradation of image quality). Further, by providing exemplary
embodiments with image capture frame rates greater than that of a predetermined ast
frame rate, extracted Video therefrom provides smooth and clear Video, without edgy or
blurred frames. Finally, various exemplary embodiments utilizing enhanced GUI features,
such as automatic tracking of subjects of sts, plural GUIs or extraction windows for one
or plural (for ent points of view) captured images provide advantageous production
flexibilities and advantages.
It will be apparent to those skilled in the art that, while exemplary embodiments
have been shown and described, various modifications and variations can be made to the
invention disclosed herein without departing from the spirit or scope of the invention. Also,
the exemplary implementations described above should be read in a miting fashion,
both with regard to construction and methodology. Accordingly, it is to be understood that
the various embodiments have been described by way of illustration and not limitation.