WO2023175114A1 - Procédé de production d'un flux vidéo - Google Patents

Procédé de production d'un flux vidéo Download PDF

Info

Publication number
WO2023175114A1
WO2023175114A1 PCT/EP2023/056826 EP2023056826W WO2023175114A1 WO 2023175114 A1 WO2023175114 A1 WO 2023175114A1 EP 2023056826 W EP2023056826 W EP 2023056826W WO 2023175114 A1 WO2023175114 A1 WO 2023175114A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
video
camera
image
scenery
Prior art date
Application number
PCT/EP2023/056826
Other languages
English (en)
Inventor
Max VON BRAUN
Original Assignee
Pucki Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from EP22177088.6A external-priority patent/EP4287630A1/fr
Application filed by Pucki Gmbh filed Critical Pucki Gmbh
Publication of WO2023175114A1 publication Critical patent/WO2023175114A1/fr

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41415—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance involving a public display, viewable by several users in a public space outside their home, e.g. movie theatre, information kiosk

Definitions

  • the present invention concerns a method for producing a video stream, especially a video stream of a scenery which can comprise fast moving objects.
  • a scenery is captured with a camera to provide a video of said scenery which has a predetermined standard video frame rate SVFR.
  • the standard video frame rate SVFR for broadcast is usually 50 frames per second (fps), i.e. 50 Hz.
  • fps frames per second
  • one second of a video stream comprises 50 frames (images), each having a duration of 20 ms.
  • the shutter time exposure time per frame
  • a video frame where the shutter time (exposure time) corresponds to the maximum value 1/SVFR is denoted “unshuttered” while a video frame where the shutter time (exposure time) is lower than the duration corresponding to the standard video frame rate 1/SVFR is denoted “shuttered”.
  • video cameras are employed which are able to capture a sequence of frames (camera images) of a scenery at a camera frame rate CFR which is higher than the standard frame rate SVFR.
  • a camera operating at a camera frame rate CFR 200 fps (200 Hz) can be employed.
  • four frames (camera images) are recorded. If the sequence of recorded images is then displayed at the standard video frame rate SVFR, a 4x slow motion effect results.
  • video streams of sceneries have to be captured where fast moving objects are present. Especially, if these objects are small, it can be difficult to visualize the trajectory of such objects for the video or TV viewer.
  • Typical events where small fast moving objects can be part of a scenery are sports events such as a tennis event or a golf event.
  • Typical examples where shuttered video images are recorded are known from multiple advertisement broadcasting, typically at sports events.
  • the term “multiple advertisement” relates to different techniques which allows to broadcast/video stream an event where a scenery includes advertisements displays (signboards), typically LED displays, on which a certain type of advertisement is shown for the direct viewers present at the event while different advertisement content on the displays is visible for certain sub-groups of broadcast viewers/video viewers.
  • One technique known as virtual advertisement uses image replacement technologies in video post-production to identify the location of the advertisement display in a video and to selectively replace the image content visible on the LED display with alternative image content.
  • DCM dynamic content multiplication
  • the technical problem underlying the present invention is to provide a method for producing a video stream which allows an improved representation of fast moving objects in videos of a scenery using shuttered cameras, i.e. where the shutter time (exposure time) is shorter than the standard video frame rate time interval.
  • the method of the invention should also improve the image quality of the resulting video stream.
  • the present invention suggests to provide a different video processing for different parts, ie different frame areas, of each video image (video frame) captured by the camera.
  • the corresponding frame areas of the at least first and second captured camera images are combined.
  • a second frame area of the video image frame only one corresponding frame area of the at least first and second captured camera images is used. Consequently, when any fast moving object in the scenery is present, it will be depicted in the first frame area multiple times, i.e. corresponding to the number of at least first and second captured camera images which are combined to form the respective area of the video image frame.
  • the fast moving object is in the second frame area of the video image frame, only one copy of the object is visible in each final video image frame. Accordingly, when a fast moving object is in the first frame area, the trajectory can easily be followed by a TV or video viewer because each video frame contains multiple copies of the object. In the second frame area, only one copy of a fast moving object in this area is visible per video frame.
  • the method of the present invention is particularly useful when the second frame area is only a small portion of the total frame area of a video image frame because in most parts of the video image frame, tracing a fast moving object is easily possible and only a small portion of the video image frame is still beset with the problems of prior art. Overall, the method of the present invention results in an improved perception of small fast moving objects in a video stream nonetheless.
  • the camera frame rate CFR is an integer multiple n of said standard video frame rate SVFR, with the integer multiple n being greater than one, for instance 2 or 3 or larger.
  • the integer multiple n is at least 4, so that in areas of a video frame where all camera images captured within a standard video frame rate time interval are combined, n representations, for instance at least 4 representations, of the fast moving object along its trajectory are visible.
  • the integer multiple n is at least 6, more preferably at least 8, i.e. the camera is operated at a frame rate which is at least 6 or 8 time higher that the standard video frame rate SVFR.
  • the camera is operated in an unshuttered mode to maximize the exposure time so that fast moving objects, i.e. even the multiple copies of an object in the first frame area, appear somewhat motion blurred in order to facilitate tracking of the object further.
  • one video image frame per standard video frame rate time interval is generated with the method of the present invention.
  • the method of the present invention can also be generalized for producing slow motion videos if more than one video image frame is generated per standard video frame rate time interval but the actual video is replayed/broadcast at the standard video frame rate SVFR.
  • the camera frame rate CFR has to be chosen to be at least the above mentioned integer multiple n of said standard video frame rate SVFR (n greater than 1 ) further multiplied with a desired slow motion factor which is also an integer greater than 1 .
  • the first and second frame areas defined in step b) can be static in subsequent video image frames. Consequently, “defining at least a first frame area and a second frame area” in step b) can also mean that the first and second frame areas of a particular video image frame correspond to the respective frame areas of the previous video image frame so that no changes have to be effected. This can, for instance, be effected using a vision mixer which receives the sequence of first camera images and the sequence of second camera images as different video inputs.
  • a “vision mixer” is a device that allows switching video inputs and outputs and perform image processing like fade-overs, include text overlays or graphics overlays, etc.
  • Vision mixers are used in typical sport or studio broadcast situation where video streams from various cameras need to be received and handled in order create a final video stream to be delivered to the broadcast viewers.
  • conventional vision mixers can be used which, in this case, receive different inputs from signals obtained from a single camera rather than multiple cameras.
  • the vision mixer can still receive additional inputs from additional cameras.
  • the operator can define the respective frame areas for the final video stream and indicate which video feed or which combination of video feeds shall be shown in the respective frame areas.
  • Many modern vision mixers comprise features like “zoning” which allows defining different static or dynamic zones in a video frame in which image processing operations can be applies which differ from zone to zone.
  • the present invention can be implemented using standard broadcasting technology.
  • SDI standard broadcasting technology
  • modern video production often employs image resolutions higher than the standard full HD resolution (1920 x 1080 pixel), for instance, an ultra-HD or4K resolution (3840 x2160 pixel at 16:9 image ratio or 4096 x2160 pixel at 19:10 image ratio). If videos are captured at UHD or 4K image resolution and/or in slow motion, the required bandwidth usually exceeds the bandwidth provided by the SDI standard. Therefore, camera manufacturers provide their cameras with multiple SDI outputs allowing transmission of parts of an image and/or sequences of images at different outputs where each output meets the SDI bandwidth requirements.
  • Such outputs can be provided already at the camera head or at a downstream camera control unit CCU or baseband processing unit BPU.
  • the present invention can also be implemented using modern internet-protocol (IP) based broadcasting technology, for instance technology implementing SMPTE 2110 standards defined by the Society of Motion Picture and Television Engineers (SMPTE).
  • IP internet-protocol
  • the first and second frame areas are dynamic in subsequent video image frames.
  • the first and second frame areas in the final video image frame will have to be adapted as well.
  • Different techniques for dynamically adapting the frame areas defined in each video image frame can be employed. For instance, it is possible to define initial frame areas and to obtain information regarding movement, panning, tilting and zooming of the camera via appropriate sensors and/or control units and to calculate the effect on the appearance of the initially defined frame areas in subsequent video image frames.
  • the at least first and second frame areas will correspond to distinct features in the scenery captured by the camera. It is therefore possible to use image processing technology to identify the at least first and second frame areas within the image by exploiting, for instance, structural, spectral or other features of the recorded image. For instance, in virtual advertisement technologies, it is important to differentiate background and foreground of a captured camera image. Such techniques have, for instance, been described in US 6,490,006 (Daisho Denki), WO 2012/038009 A1 (Fraunhofer) or WO 2020/021067 A1 (Appario Global Solutions). In one embodiment, the image processing is based on brightness-offset information shown in said scenery.
  • the term “brightness-offset” means that the scenery captured by the camera is modulated differently with respect to brightness at locations which shall correspond to the first areas and at locations which shall correspond to the second areas in the captured video frames. Accordingly, it is possible to modulate one of the frame areas between subsequent video image frames, e.g. in terms of brightness and to calculate a mask corresponding to the respective frame area from a differential image of the subsequent video image frames. Such a modulation can easily be accomplished when the respective frame area corresponds to an active display, such as an LED display.
  • said at least first and second frame areas are defined by image processing, for instance by using chromakey information shown in the scenery captured by the camera.
  • a given frame area in the video image frame can be defined via the appearance of certain chromakey features in the scenery.
  • the pre-defined chromakey can be detected and the associated frame areas can be defined accordingly.
  • Features within the scenery can also be used to identify camera motion such as movement, panning, tilting and zooming, for instance by using so-called “tracking patterns”.
  • the image processing to identify the at least first and second frame areas can be based on complementary pattern information shown in the scenery as described in the above-mentioned international patent application WO 2020/021067 A1 .
  • a pattern and a complementary pattern is shown alternatively in subsequent video frames by synchronizing the presentation of the pattern/complementary pattern in the scenery with the image capturing of the camera. If a differential image is calculated from such subsequent video frames, the differential image will show essentially black areas in areas which are unchanged from one video frame to the next, and the differential image will show bright/white areas which have a certain pattern in one video frame but a complementary pattern in the next image frame.
  • the generated differential image is a mask which can be used to identify the first and second frame areas.
  • Typical video cameras employ a sensor technology which is either based on a rolling shutter technology or a global shutter technology.
  • Rolling shutter and global shutter technologies relate to two distinct sequences through which the image may be read off an sCMOS sensor.
  • rolling shutter mode different lines of the array are exposed at different times as the read out signal sweeps through the sensor, whereas in global shutter mode each pixel in the sensor begins and ends the exposure simultaneously.
  • Using such cameras requires using the above defined image processing technology in order to define the first and second frame areas.
  • camera sensors available which allow a pixel-based exposure and readout.
  • Such censors can be employed to define the first and second frame areas via suitable camera firmware or software directly on the sensor.
  • the at least first and second frame areas are defined directly on a sensor of a camera recording the scenery.
  • the first and second frame areas are read-out sequentially.
  • each of the correspondingly defined first and second frame areas can be read read-out in mode corresponding to a conventional global shutter mode or in a mode corresponding to a conventional rolling shutter mode, but restricted to the respective area, i.e. to a sub-frame of the whole sensor frame.
  • the first and second frame areas can vary in subsequent camera images.
  • one of said at least first and second captured camera images is synchronized with such additional features required for dynamically adapting the first and second frame areas.
  • the method of the present invention is particularly useful when the scenery captured by the camera comprises moving objects, especially fast moving small objects such as tennis balls or golf balls.
  • the method of the present invention involves a scenery which comprises, at least temporarily, moving objects, in particular, small fast moving objects such as tennis balls or golf balls. Consequently, the scenery is often a sports event such as a tennis match or a golf tournament.
  • the present invention can also be used in a scenery which comprises larger moving objects such as ski jumpers of racing cars.
  • the scenery can comprise physical displays showing alternative image content in a time-sliced multiplexed manner where typically one main type of image content is intended to be noticeably visible for direct viewers who are part of the scenery or who are close to the scenery and other image content is intended to be only captured by the camera(s) recording the scenery.
  • the latter type of image content is typically presented only for such a brief period of time that the direct viewers are not able to noticeably view the image content. It can also be presented as a sequence of images and inverse/complementary images which are presented within a short time interval, typically within a time interval shorter than 10 ms, preferably within a time interval not exceeding 3.3 ms.
  • the physical displays can be active displays such as LED displays or OLED displays which allow for the required rapid change of different image content in a time-sliced multiplexed manner.
  • Fig. 1 shows a typical broadcast scheme of a tennis event according to prior art
  • Fig. 2 shows a schematic overview of image-capturing and video stream generation according to prior art
  • FIG. 3 shows a video image frame of a video stream obtained with the prior art technique of Fig. 2;
  • Fig. 4 shows a schematic overview of image-capturing and video stream generation according to prior art in the context of multi-content advertisement;
  • Fig. 5 shows a video image frame of a video stream obtained with the prior art technique of Fig. 4;
  • Fig. 6 shows a broadcast scheme of a tennis event according to the present invention in the context of multi-content advertisement
  • Fig. 7 shows the definition of different frame areas according to the method of the present invention.
  • Fig. 8 shows a resulting video image frame where a fast moving object is in a second frame area
  • Fig. 9 shows an image corresponding to Fig. 6 where a fast moving object is in a first frame area.
  • FIG 1 shows a typical broadcast scheme of a tennis event according to prior art.
  • a tennis court 10 For the sake of simplicity, in the present example only one camera 11 having a camera head 12 and a camera lens 13 mounted to the camera head is shown. Camera 11 is placed behind and above a baseline 14 of tennis court 10.
  • sports events which are broadcast include various kinds of advertisement and marketing elements.
  • several LED signboards 15 are shown surrounding the tennis court. In a conventional setting, the LED signboards 15 show a single type of advertisement, namely an advertisement which can be seen by direct viewers present at the event and also by broadcast viewers because the same type of advertisement is captured by camera 11 .
  • Camera 11 is operating at a standard video frame rate SVFR, for instance at a standard video frame rate of 24, 25, 30, 50 or 60 fps (frames per second).
  • Any fast moving object such as a tennis ball consequently appears as an extended element blurred along its trajectory over the exposure time of 20 ms. This allows broadcast viewers to more easily perceive the movement of the ball because “jumps” of the ball between consecutive video image frames are avoided.
  • a data transmission line 16 connects camera 11 with a camera control unit (CCU) 17 installed in an OB van 18.
  • a raw video output line 19 connects the camera control unit 17 with a vision mixer 20 which has multiple inputs 21 a - 21 e for video data so that graphic, text or video inserts or overlays can be added to the raw video captured by camera 11 .
  • the final video stream generated by the vision mixer (and optional other video processing equipment not shown in Fig.1 ) is then provided at a video output 22 of the vision mixer as an output video stream which can, for instance, be broadcast at the standard video frame rate via an output transmission line 23 and a satellite dish 24 installed on top of the OB van 18 to the broadcast viewers.
  • wire-based transmission using, e.g. a fiber line, or transmission via a broad band cellular network, e.g. a 5G network are also possible.
  • an OB van is only a typical example for an existing broadcasting technology.
  • remote broadcast technology can be employed as well where rather than transmitting final video feeds from an OB van to a rebroadcast center, raw camera video feeds are transmitted to a remote location where they are processed/assembled into the final video signal.
  • Fig. 2 shows a time line of a typical image capturing scheme employed in the state of the art system of Fig. 1 for generating the video stream of a sports event where advertisements are shown on LED signboards.
  • Fig. 2a shows the sequence of images (image frame) presented on the LED signboards 15.
  • AT in the present example: 20 ms
  • F1 n intended for all viewers (i.e. direct viewers present at the event and broadcast viewers) is presented.
  • the sub-index n denotes the respective image frame shown during the 20 ms time interval.
  • the next image frame F n +i is shown, etc.
  • the images on the LED signboards can remain unchanged, for instance if a static advertisement video sequence shown, or can be different, e.g. sequences of an advertisement video shown on the signboard.
  • Fig. 2b shows the video capturing process of camera 11 .
  • the camera operates unshuttered, i.e. the shutter time corresponds to the standard video frame rate time interval AT. Consequently, one image frame C1 n of the video generated by camera 11 is captured per standard video frame rate time interval. The next image frame C1 n +i is captured in the following standard video frame rate time interval.
  • Fig. 3 shows a typical video image frame of the resulting video shown to broadcast viewers when captured with the scheme of Fig.2.
  • the video image frame has an outer image border 30 and an inner image region 31 corresponding to the camera image captured during a standard video frame rate time interval AT.
  • camera 11 of Fig. 1 captures a scenery which comprises the tennis court 10.
  • one tennis player 32 is in the process of serving while the second player 33 is expecting the serve.
  • LED signboards 15 which show advertisements, (denoted “AD1” in Fig. 3) intended to be seen by the direct viewers (spectators) 34 are installed around tennis court 10 and are therefore part of the scenery shown in the image frame of Fig. 3.
  • the same advertisement is shown to all broadcast viewers.
  • FIG. 3 shows a typical video image frame of the resulting video shown to broadcast viewers when captured with the scheme of Fig.2.
  • the video image frame has an outer image border 30 and an inner image region 31 corresponding to the camera image captured during a standard video frame rate time interval AT.
  • the shutter time of camera 11 corresponds to the standard frame rate time interval of 20 ms so that the fast moving tennis ball 35 is no longer a sharp object but motion-blurred along a part of its trajectory. This effect is intended in conventional recording of tennis matches to increase visibility of the tennis ball for the broadcast viewers because it avoids “jumps” of the tennis ball 35 from one video image frame to the next.
  • FIG 4 shows a time line of a typical image capturing scheme employed in a state of the art system for generating video streams of a sports event where advertisements are shown on LED signboards 15 in the context of multi-content advertisement.
  • Multi-content advertisement in the sense of the present invention means that on one and the same advertisement signboard which is part of a scenery captured by the camera, different types of image content are shown for different sub-sets of viewers, for instance the direct viewers present at the event and one or more different sets of broadcast viewers. Therefore, the dedicated image content (e.g. different advertisements) can be presented simultaneously throughout the event to the direct viewers and different sets of broadcast viewers, for instance to broadcast viewers in different countries where each can see an advertisement in their native language.
  • different technologies have been developed in the past.
  • technologies denoted “virtual advertisement” use image processing technology to identify the location of the physical advertisement signboards in a scenery of the video stream and selectively replace the area of the detected signboards by alternative image content for different sub-sets of broadcast viewers resulting in different final videos being produced of the event where each shows the same scenery but the advertisement boards show different advertisements.
  • This technique still faces problems with properly identifying background and foreground (for instance, players moving before the signboards), etc.
  • dynamic content multiplication described in international patent application WO 2018/138366 A1 , every type of advertisement of the multi-content advertisement is actually shown on the physical display but only one type long enough so that it can be perceived by the direct viewers present at the event (the so-called venue feed).
  • Additional alternative advertisements are shown in a time-sliced multiplexed manner on the physical display for only brief periods of time too short to be noticed by the direct viewers.
  • international patent application WO 2018/138366 A1 suggests to present the additional image content (the so-called “parallel feeds”) not only as a sequence of the respective images but as a sequence of images and corresponding inverse images so that the resulting inserted image is a grey image which reduces the disturbance of the direct viewers.
  • a camera is employed which captures images synchronized to the particular image content feeds shown on the LED signboards but do not capture the inverse images shown on the LED signboards.
  • alternative image content in the sense of the present invention can be actual advertisements shown on the physical displays which are captured by synchronized cameras or image content which is only intended to identify the location of the signboards in the captured camera images for replacing the image content of the signboards by alternative image content in video post-production.
  • the alternative image contents shown on the physical displays can in this case be for instance monochromatic chromakey images or pattern images.
  • the time line of Figure 4a shows the sequence of images presented on the LED signboards 15.
  • AT standard frame rate time interval
  • the standard frame rate time interval corresponds to 20 ms
  • four different advertisement feeds are shown, namely a venue feed F1 n intended to be seen by the direct viewers and any set of broadcast viewers who view a video/TV broadcast of a video stream using camera images synchronized to the presentation of the venue feed.
  • the sub-index n denotes the respective image frames shown during the 20 ms time interval and the sub-index n+1 denotes the image frames of the subsequent frame rate time interval.
  • each parallel feed is presented as a combination of the actual parallel feed image and its corresponding inverse/complementary image I2 n , I3 n , I4 n respectively.
  • Fig. 4b shows the video capturing process of the camera.
  • a camera operating at a higher frame rate than the standard frame rate is employed.
  • a camera operating at four times the standard frame rate is employed, i.e. a camera image is captured every 5 ms.
  • the camera is synchronized such that the venue feed advertisement F1 n and each of the additional advertisement feeds intended for broadcast viewers only F2 n , F3 n and F4 n are captured by the camera.
  • a first video stream for the venue feed advertisement comprising camera images C1 n , C1 n +i, C1 n +2, etc.
  • a second video stream comprising the scenery when the second advertisement F2 intended for broadcast viewers only is shown on the physical displays (signboards 15), comprising camera images C2 n , C2 n +i, C2 n +2, etc.
  • third and fourth video streams are formed on the basis of camera images C3 and C4, respectively. Due to the shorter shutter times, the image noise is higher compared to the prior art technology of Figures 1-3 where only one advertisement is shown on the signboards 15.
  • Fig. 5 shows a typical video image frame of the resulting video which is based on camera images C2 captured with the scheme of Fig. 4.
  • the signboards 15 depict “AD2” as a representation of the advertisement feed based on images F2 n shown on the signboards.
  • AD2 advertisement content
  • Fig. 3 it is noted that due to the much shorter shutter time of the camera, the fast moving ball 35 appears as a sharp image.
  • the present invention is based on the observations exemplified in Fig. 5, namely that in the context of multi-content advertisement broadcasts, the image noise is higher compared to traditional videos of sports events and small fast moving objects are harder to follow by the broadcast viewers.
  • the present invention suggests to overlay multiple camera images in areas where the short shutter times of Fig. 4 are not essential and to use images of only one camera image captured with the short shutter time of Fig. 5 only in frame areas where the short shutter time is actually essential.
  • the short shutter times are only essential in areas which correspond to the LED signboards 15 in the scenery in order to clearly discriminate between advertisements intended for one set of broadcast viewers from advertisements intended for other types of broadcast viewers.
  • FIG. 6 shows a setup for broadcasting a tennis event according to the method of the present invention.
  • Features and elements of Fig. 6 which correspond to features and elements already described in connection with Fig. 1 are denoted by the same reference number and related features are designated by the same reference number augmented by 100.
  • the tennis court 10 corresponds to the tennis court of Fig. 1 and, although shown only partially, also has the same signboards 15 which, in the present case, do not only show one type of advertisement but four advertisement feeds in a time- sliced multiplexed manner as described in connection with Fig. 4.
  • a high-speed camera for example a commercially available slow motion camera, capable of recording at a camera frame rate which is, in the present example, at least four times the standard frame rate. As indicated above, a minimum of two times the standard frame rate is required in order to practice the invention.
  • the camera again has a camera body 112 and a camera lens 113 and is connected via a high bandwidth data transmission line 116 to camera control unit 117 installed in OB van 18.
  • the camera control unit 117 provides four raw output lines 19a, 19b, 19c, 19d which correspond to four different phase-shifted sequences of camera images.
  • the camera captures an image every 5 ms, i.e. four images during each video frame rate time interval of 20 ms.
  • the first camera images captured during each video frame rate time interval form the first video feed 19a
  • the second camera images form the second video feed 19b, etc. Consequently, lines 19a- 19d show essentially the same video stream but video data on lines 19b, 19c and 19d are phase-shifted by 5 ms, 10 ms, 15 ms with respect to the video signal on line 19a, respectively.
  • the signal captured by camera 111 is used for slow motion video and the SDI outputs 19a-19d are provided to accommodate legacy low-bandwidth data connections which are often already installed at sports events.
  • these data lines 19a-19d are typically fed to a slow motion server which re-combines the different feeds to generate a slow motion video at the standard video frame rate.
  • Camera control units 117 also provide a live feed output line 119 which already corresponds to the combined signal on lines 19a-19d as a 50hz video signal output and which therefore already comprises all captured camera images. It is noted that camera control unit 117 can also be a baseband processing unit typically installed in the OB van 18. Alternatively, the functionality of the camera control unit 117 can essentially already be implemented in the camera head 112 of camera 111 so that in other embodiments, lines 19a-19d and 119 are already provided at the camera head 112.
  • the individual phase-shifted outputs 19a-19d as well as the live output 119 are fed to a vision mixer 20 typically also installed in the OB van 18.
  • the vision mixer 20 allows defining the individual frame areas according to the method of the present invention which will be described in more detail.
  • the final video generated by the vision mixer and optional other video processing equipment not shown here is then provided as an output video stream at a standard video frame rate (in the present example 50 fps) via output line 23 which can, for instance, be broadcast via satellite dish 24 installed on top of the OB van 18.
  • Fig. 7 shows a mask for defining frame areas in the example of Fig. 5.
  • the first frame area 40 covers the scenery outside the LED displays 15 in Fig. 5, while the second frame area 41 covers the area of the LED displays 15 in Fig. 5 where different advertisement content is shown.
  • the first frame area 40 is shown in white, while the second frame area 41 is shown in black.
  • the mask defined by frame areas of Fig. 7 corresponds to different video post-processing the different areas by vision mixer 20 or any other suitable video processing unit.
  • the first frame area 40 is formed from the corresponding frame areas of camera images C1 n , C2 n , C3 n and C4 n
  • the second frame area 41 is formed by the corresponding frame area of only one of the camera images C1 n , C2 n , C3 n or C4 n , in the context of the example shown in the figures specifically of camera images C2 n .
  • Figures 8 and 9 show two typical examples of image frames resulting from the corresponding video stream.
  • the reference signs correspond to the reference signs used in connection with Figure 5.
  • the video image frames are from a video intended to be broadcast or otherwise presented to a set of broadcast viewers who shall see the advertisement “AD2” shown in the advertisement feed F2 n on the LED signboards 15.
  • the corresponding advertisements are shown with the corresponding synchronized camera images C2, (see Fig. 4).
  • tennis ball 35 is within the second frame area (frame area 41 in Fig. 7) where only one image (here image C2 n ) is used to form the frame area. Therefore, tennis ball 35 appears as a single object.
  • the image noise is still relatively high, corresponding to the image noise of Figure 5.
  • the image noise is considerably reduced because all four corresponding areas of the camera images C1 n , C2 n , C3 n and C4 n are overlayed. Overall, especially when the second frame area 41 is small compared to the first frame area 40, the image quality of the resulting video is considerably improved.
  • Fig. 9 shows a video frame corresponding to Fig. 8 but captured shortly after the frame of Fig. 8 has been captured, i.e. at a time when the tennis ball is in the first frame area. Accordingly, as four images captured by camera 111 are overlayed, each image captured with the short shutter time of the scheme of Figure 4, the video frame image shows four copies 35a, 35b, 35c, 35d of the tennis ball at different positions of its trajectory corresponding to the image capturing times of camera images C1 n , C2 n , C3 n and C4 n in the scheme of Figure 4. Consequently, the “jumps” of fast moving objects such as tennis ball 35 in consecutive video image frames are considerably reduced and the broadcast viewer can more easily follow the ball along its trajectory.
  • the tracing of the trajectory of the tennis ball can be even more facilitated if the representation of the tennis ball 35, 35a, 35b, 35c, 35d is not as sharp as depicted in Figures 8 and 9 but slightly motion blurred along the trajectory. This effect will depend on the speed of the tennis ball and can be enhanced if the camera is operated at maximum permitted exposure/shutter time.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Business, Economics & Management (AREA)
  • Marketing (AREA)
  • Studio Devices (AREA)

Abstract

La présente invention concerne un procédé de production d'un flux vidéo comprenant une séquence d'images vidéo à une fréquence d'images vidéo standard SVFR comprenant les étapes suivantes : a) capturer une séquence d'images de caméra d'un décor à une fréquence d'images de caméra CFR qui est supérieure à ladite fréquence d'images vidéo standard SVFR, ladite séquence d'images de caméra comprenant pendant chaque intervalle de temps de fréquence d'images vidéo standard (ΔT = 1/SVFR) au moins une première image de caméra et une deuxième image de caméra ; et, pour chaque trame d'image vidéo par intervalle de temps de fréquence d'images vidéo : b) définir au moins une première zone d'image et une deuxième zone d'image dans ladite trame d'image vidéo ; c) former ladite première zone d'image de ladite trame d'image vidéo en superposant des zones d'image correspondantes desdites au moins première et deuxième images de caméra capturées ; et d) former ladite deuxième zone d'image de ladite trame d'image vidéo à partir d'une zone d'image correspondante de l'une desdites au moins première et deuxième images de caméra capturées.
PCT/EP2023/056826 2022-03-16 2023-03-16 Procédé de production d'un flux vidéo WO2023175114A1 (fr)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
EP22162475 2022-03-16
EP22162475.2 2022-03-16
EP22177088.6A EP4287630A1 (fr) 2022-06-02 2022-06-02 Procédé de production d'un flux vidéo
EP22177088.6 2022-06-02

Publications (1)

Publication Number Publication Date
WO2023175114A1 true WO2023175114A1 (fr) 2023-09-21

Family

ID=85706803

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2023/056826 WO2023175114A1 (fr) 2022-03-16 2023-03-16 Procédé de production d'un flux vidéo

Country Status (1)

Country Link
WO (1) WO2023175114A1 (fr)

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6490006B1 (en) 1998-08-22 2002-12-03 Daisho Denki Inc. Chroma key system for discriminating between a subject and a background of the subject based on a changing background color
WO2012038009A1 (fr) 2010-09-20 2012-03-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Procédé de distinction entre l'arrière-plan et le premier plan d'une scène et procédé de remplacement d'un arrière-plan dans les images d'une scène
US20170257571A1 (en) * 2014-03-27 2017-09-07 Facebook, Inc. Stabilization of low-light video
WO2018138366A1 (fr) 2017-01-27 2018-08-02 Lane Gmbh Procédé et système de transmission d'un contenu d'images alternatif d'un affichage physique à différents spectateurs
WO2020021067A1 (fr) 2018-07-27 2020-01-30 Appario Global Solutions (AGS) AG Procédé et système de remplacement de contenu d'image dynamique dans un flux vidéo
US20210297717A1 (en) * 2018-07-27 2021-09-23 Appario Global Solutions (AGS) AG Method and system for transmitting alternative image content of a physical display to different viewers
US20210321032A1 (en) * 2018-09-13 2021-10-14 Appario Global Solutions (AGS) AG Method and device for synchronizing a digital photography camera with alternative image content shown on a physical display

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6490006B1 (en) 1998-08-22 2002-12-03 Daisho Denki Inc. Chroma key system for discriminating between a subject and a background of the subject based on a changing background color
WO2012038009A1 (fr) 2010-09-20 2012-03-29 Fraunhofer-Gesellschaft zur Förderung der angewandten Forschung e.V. Procédé de distinction entre l'arrière-plan et le premier plan d'une scène et procédé de remplacement d'un arrière-plan dans les images d'une scène
US20170257571A1 (en) * 2014-03-27 2017-09-07 Facebook, Inc. Stabilization of low-light video
WO2018138366A1 (fr) 2017-01-27 2018-08-02 Lane Gmbh Procédé et système de transmission d'un contenu d'images alternatif d'un affichage physique à différents spectateurs
US20190394504A1 (en) * 2017-01-27 2019-12-26 Appario Global Solutions (AGS) AG Method and system for transmitting alternative image content of a physical display to different viewers
WO2020021067A1 (fr) 2018-07-27 2020-01-30 Appario Global Solutions (AGS) AG Procédé et système de remplacement de contenu d'image dynamique dans un flux vidéo
US20210227270A1 (en) * 2018-07-27 2021-07-22 Appario Global Solutions (AGS) AG Method and system for dynamic image content replacement in a video stream
US20210297717A1 (en) * 2018-07-27 2021-09-23 Appario Global Solutions (AGS) AG Method and system for transmitting alternative image content of a physical display to different viewers
US20210321032A1 (en) * 2018-09-13 2021-10-14 Appario Global Solutions (AGS) AG Method and device for synchronizing a digital photography camera with alternative image content shown on a physical display

Similar Documents

Publication Publication Date Title
US11825137B2 (en) Method and system for transmitting alternative image content of a physical display to different viewers
US11582493B2 (en) Method and system for dynamic image content replacement in a video stream
JP7391936B2 (ja) 物理ディスプレイの代替画像コンテンツを異なる視聴者へ送信するための方法およびシステム
US8457350B2 (en) System and method for data assisted chrom-keying
Armstrong et al. High frame-rate television
EP4287630A1 (fr) Procédé de production d'un flux vidéo
WO2023175114A1 (fr) Procédé de production d'un flux vidéo
EP4252409A2 (fr) Procédé et système de capture d'images
EP4080866A1 (fr) Procédé et système de capture d'images
JP2012019434A (ja) 広告の放送又は送信方法
Moore et al. 10 Video Production Switchers and Special Effects Systems

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23712220

Country of ref document: EP

Kind code of ref document: A1