EP4104164A1

EP4104164A1 - Method and device for reducing motion blur on displays

Info

Publication number: EP4104164A1
Application number: EP21703485.9A
Authority: EP
Inventors: Max VON BRAUN
Original assignee: Appario Global Solutions AGS AG
Current assignee: Appario Global Solutions AGS AG
Priority date: 2020-02-10
Filing date: 2021-02-10
Publication date: 2022-12-21
Also published as: WO2021160703A1

Abstract

The present invention concerns a method and a device for reducing motion blur when presenting media data on a physical display such as an LED display, comprising: sending a first sequence of images (/,) from a computer device (11) to a display (16) at a frame rate f, displaying each image (/,) of said first sequence of images for a time interval ΔΤ₁ corresponding to a fraction of the frame time interval 1/f (ΔΤ₁ = p. 1/f), where p less than 0.5, and after displaying each image (/,), presenting a neutral insert (N) on said display for a time interval ΔΤ₂ corresponding to the remainder of the frame time interval (AT₂ = (1 - p). 1/f).

Description

Method and device for reducing motion blur on displays Description The present invention concerns a method for reducing motion blur when presenting media data, especially video data, on a display, for instance LCD displays but especially on a display comprising active pixels, such as a LED display or an OLED display. Media data, such as video data consist of a sequence of individual images which are usually presented on modern displays in a progressive mode, i.e. each individual image is presented for a frame time interval Po the viewer which corresponds to the inverse of the frame rate f(T = 1/f). The frame time or frame rate is usually determined by the incoming source material zone shown on the display, e.g. when video data are concerned, the frame rate determines the rate, at which different images are presented. In contrast to frame time/frame rate, each display has a certain intrinsic performance determining the rate at which images can be presented which is denoted by the refresh rate rwhich is at least as high, but usually much higher than the frame rate f. When the refresh rate is higher than the frame rate, the same image can be presented multiple times within a given frame time interval T. It is known that moving content in search video data may exhibit a so-called “motion blur”. Motion blur stems from certain physiological and psycho-physiological characteristics of the human eye. One characteristic effecting motion blur is known as “persistence of vision” or “flicker fusion” describing the effect that the human eye has a certain threshold frequency above which intermittent light stimuli can no longer be differentiated from each other. Flicker fusion is beneficial when presenting video data on a display, because, if a rapid succession of static images is presented on a display at a frame rate higher than the flicker fusion threshold, individual images can no longer be discriminated from each other and a smooth playback of the video is perceived. The flicker fusion threshold of a person depends on individual factors such as age and external factors such as image brightness. The persistence of the vision can, however, result in motion blur if two successive (different) images of a video screen show objects, which change their location from one frame to another. Motion blur is particularly visible if the human eye follows the motion of an object on the display screen. It has already been suggested that such motion blur can be reduced by introducing short pixel-off periods between each individual image. However, the motion blur reduction achieved so far is still not sufficient.

It is therefore an object of the present invention to provide a method and a device which considerably improves the motion blur reduction of prior art.

This object is achieved by the method and the device as defined in the appended claims.

Accordingly, the present invention is directed to a method for reducing motion blur when presenting media data, such as video data, on a display, comprising the steps of: sending a first sequence of images (I,) from a computer device (11) to a display (16) at a frame rate f, displaying each image (I,) of said first sequence of images for a time interval DTi corresponding to a fraction of the frame time interval 1/f (AT_t = p 1/f), where p less than 0.5, and after displaying each image (/,), presenting a neutral insert (N) on said display for a time interval DT2 corresponding to the remainder of the frame time interval (AT₂ = (1 - P) 1 //)

Accordingly, the present invention suggests that the time interval in which a neutral insert is interleaved between the actual images of the video stream is larger than each time interval in which each single image (frame) of the video stream is presented. It has surprisingly been found that with the method of the present invention, motion blur can be significantly reduced. Without wishing to bind the present invention to a particular theory, it can be assumed that when the human eye follows the motion of an object on the display screen, presenting an image only a fraction of the frame time interval reduces motion blur because the human brain will correctly interpolate the trajectory of an object between images successive frames. In contrast to prior art, where no or only short dark periods are inserted between successive frames, the longer neutral inserts according to the present invention (more than 50% of a frame time interval) reduce the ambiguities in the reconstruction of the trajectory of a moving object by the human brain which in turn reduces the perception of motion blur. A “neutral insert” (N) in the sense of the present invention is any state of the display within a frame time interval which does not or not significantly disturb the perception of the sequence of images (/,) of the media data presented on the display to the direct viewers. “Direct viewers” are viewers are viewers which are present at the location where the display is operated.

In a preferred embodiment the fraction p is at least 0.1 and lower than 0.5 (0.1 £ p < 0.5), more preferably in the range from 0.25 to 0.45 (0.25 £ p < 0.45). Various embodiments of a “neutral insert” can be employed to reduce or eliminate motion blur to occur in the perception of the sequence of images (/,) of the media data presented on the display for the direct viewers. For instance, in one embodiment, the neutral insert can be a grey image, preferably a dark-grey image having a lower intensity than the images of the sequence of images (/,) of the media data. On an active display, image intensity is usually controlled via pulse width modulation techniques (PWM), i.e. the active elements such as LEDs are fed with a given current and the on/off-times of each individual LED are controlled in order to achieve the desired perceived intensity (brightness). If the neutral insert is a grey image, the only effect on the sequence of images (/,), as perceived by the direct viewers is a moderate increase in the black levels, i.e. due to the persistence of vision of the human eye, the direct viewer will ultimately perceive a combination of the sequence of images (/,) and the interleaved neutral inserts ( N ).

In a preferred embodiment of the invention, each neutral insert ( /) is an individual image, for instance a dark-grey image. In one embodiment, the dark-grey image is a black image ( B ). In this case, no specific image data for the neutral insert has to be transferred to the display, but the display controller can be configured to actually switch off the individual pixels of an active display such as a LED or a OLED display or, for instance in case of a LCD display, switch off the backlight of the display.

In another preferred embodiment, the neutral inserts can comprise more than one image. The images of the neutral insert are selected such that their combined effect on the perception of the sequence of images (/,) of the media data results in a perceived addition of a neutral/grey image. For instance, the neutral insert can comprise an additional video feed which is configured such that it is not consciously perceived by the direct viewers of the display. In order to avoid perception of the additional video feed interleaved within the sequence of images (/,), the additional video feed or “parallel video feed PF consists of images (PF) and their respective complementary/inverse images ( CF, ). An image ( PF) and its associated complementary/inverse image (CF) are presented within the same frame time interval which is shorter than the persistence of vision of the human eye such that the combined effect is a structureless homogeneous grey image which is not consciously perceived at all but can again, as in the case of an actual grey image, only be noticed (if at all) as a minor increase of the black level of the main video feed, i.e. of the first sequence of images (/,). The additional video data of the interleaved parallel feed ( PF) can be recorded by an appropriately synchronized camera and transmitted as an alternative content of the physical display to broadcast viewers as described, for instance, in applicant’s International Patent Application WO 2018/138366 A1. As a matter of cause, more than one parallel feed ( PF) and its corresponding complementary/inverse feed ( CF) can be included within each neutral insert.

A variety of data content can be included in the so-configured parallel feeds (PF). For instance, as described above, a parallel feed can comprise actual alternative image content, for instance video data, to be transmitted to broadcast viewers. In addition or alternatively, one dedicated parallel feed can comprise chromakey data or other data which can be captured by a synchronized camera and which can be used to identify the location of the physical display within a captured video screen in order to replace the image content of the physical display by an alternative image content in video post- production (a scheme which is, for instance, employed in so-called virtual advertising techniques). In these cases, the complementary/inverse images would have the complementary color of the chromakey employed. Likewise, in addition to or alternatively to chromakeys, certain image patterns shown on the display can be employed which could help identifying the physical display within a captured video for image content replacement, for instance a checkerboard pattern and a complementary checkerboard pattern as described in International Patent Application WO 2020/021067 A1. In this case, the complementary checkerboard pattern can be employed as the complementary/inverse image feed (CF,). However, while usually the complementary/inverse images are not captured by a synchronized camera, in the latter case described in WO 2020/021067 A1 , the synchronized camera would capture both, the parallel feed {PF) and the complementary/inverse feed ( CF) in order to allow identifying the location of the display within the captured video feed in post-production. In another embodiment, the neutral insert can, alternatively or in addition to the embodiment described above, comprise tracking patterns as a parallel feed ( PF) and complementary tracking patterns as complementary/inverse feed {CF). Such tracking patterns can be employed to identify the exact location and orientation/attitude of a camera with respect to the display in order to increase the precision of content replacement in post-production, e.g. by taking into account parallax distortions and lens distortions.

In a preferred embodiment of the method of the present invention a, said frame rate 7is at least 24 Hz, preferably at least 50 Hz. 24 Hz is a frame rate used in analog movie production for decades. Is represents the lower threshold for perceiving a relatively smooth video. In Europe, 50 Hz is the standard frame rate for television broadcasts. A corresponding frame rate in the US would be 60 Hz.

However, with these standard frame rates, flickering of the display can occur. Therefore, in a more preferred embodiment of the present invention, said frame rate f is at least 100 Hz or 120 Hz.

When using higher frame rates of 100 Hz or more, typical sending cards used for operating displays such as larger fine-pitch LED displays exhibit a bandwidth which can be insufficient for transferring the amount of data required to operate these displays at these higher frequencies. In one embodiment of the present invention, these bandwidth restrictions can be taken into account by employing a method which further comprises the steps of jointly displaying pairs of two consecutive images (/,) and (/,_+?) with reduced resolution on a control monitor of said computer device (11), jointly grabbing each pair of two consecutive images by a frame grabber (13) of said computer device (14), jointly sending each pair of two consecutive images from a sending card (14) of said computer device to a receiving card (15) of said display and displaying each image (/,) and (/,·_+/) separately on said display in two consecutive frame intervals. In a preferred embodiment of the method of the present invention, said display (16) is a LED display, specifically a fine-pitch LED display.

The above described scheme for motion blur reduction can be employed in various applications where human viewers are present which directly view media content presented on physical displays installed in a scenery. Typical applications include physical displays used as computer screens, television and video screens and the like. Other applications include larger arrays of physical displays such as LED displays which are used as advertising signboards/billboards at various kinds of events, such as sport events, for instance in a soccer stadium where the direct viewers are represented by the audience actually present at the event. Other applications include LED display array installations in movie and film studios which became popular in recent years as a replacement for actual physical backgrounds or which are used to present concurrently video background and chromakeys for synchronized cameras. The method of the present invention helps to reduce motion blur in such applications.

The present invention is also concerned with a device for reducing motion blur when presenting media data on a display comprising: a computer device (10) for generating a sequence of images (I,), said computer device comprising at least one sending card (14) for sending said sequence of images (I,) from said computer device (10) to a display (16), said display (16) comprising at least one receiving card (15) for storing and receiving said sequence of images received from said sending card (14); means for displaying each image (I,) for a time interval DTi corresponding to a fraction of the frame interval 1/f (AT_t = p 1 //), where p less than 0.5, and after displaying each image (/,), for presenting a dark image (B) on said display for a time interval DT2 corresponding to the remainder of the frame interval (AT₂ = (1 - p) 1/f).

In a preferred embodiment further comprising a control monitor (12) and a frame grabber (13) for grabbing images shown on said control monitor and transferring said images to said sending card (14).

The present invention will now be described in connection with a preferred embodiment depicted in the annexed drawings. In the drawings, Fig. 1 shows a schematic embodiment of the device of the present invention for carrying out the method of the invention;

Fig. 2 shows a first scheme of a time sequence of a sequence individual images presented at a frame rate of 50 Hz where the neutral insert consists of a black image;

Fig. 3 shows an alternative time sequence where the neutral insert consists of an additional sequence of media data images and complementary/inverse images; and

Fig. 4 shows a variant of the embodiment of Fig. 3.

In the embodiment of Fig. 1, a computer 11 presents to consecutive images and h simultaneously on a control monitor 12 where a frame grabber 13 grabs the images, sends them to a sending card 14 which transmits the image data to the receiving card 15 of the display 16, which in the embodiment shown in Fig. 1 is a LED display consisting of individual LEDs 17.

Fig. 2 shows a first scheme of a time sequence (x-axis denotes time, y-axis image intensity) of a sequence individual images presented at a frame rate of 100 Hz where the neutral insert N consists of a black image. As can be taken from Fig. 2, each image /,· is only presented for a fraction of the frame interval which is smaller than 50% of the frame interval. In the present example, each image is presented for a fraction p corresponding to 20% of the frame interval {p = 0.2). In the remaining 80% of the frame interval ( 1-p ), a black image £is shown by switching the individual LEDs off. Fig. 3 shows an alternative time sequence according to the method of the present invention where the neutral insert comprises an additional sequence of media image data, for instance an additional/parallel video feed comprising media data images (PFi) and corresponding complementary/inverse images ( CF, ). The complementary/inverse images are generated on the basis of the additional media data images (PFi) such that their combined perception results in a neutral/grey image due to the persistence of vision of the human eye. Again, as in the example of Fig. 2, the example of Fig. 3 assumes a first sequence of image data (main video feed) which provides the media data to be perceived by the direct viewers of the display with a frame rate of 100 Hz, i.e. each time interval allocated for a frame of the video feed at a duration of 1/100 Hz = 10 ms. It is further assumed that the display itself has a refresh rate rof at least 500 Hz, i.e. individual images can be presented every 2 ms so that there are five image slots within each 10 ms frame time interval. In the example of Fig. 3, images are presented for fraction p corresponding to 20% of the frame time interval (p = 0.2). In the remaining 80% of the time interval ( /-/?), corresponding to 8 ms, a second sequence of media data (a second video feed) (PF is presented in such a manner that the resulting 80% of the frame time interval corresponds to a neutral insert N which is not consciously perceived by the direct viewer. To this effect, each neutral insert N comprises the actual image data of the second video feed (PF presented in a 2 ms slot and correspondingly configured complementary/inverse images (CF), also presented in a 2 ms slot, which result in a non consciously perceived grey image which is effectively overlaid to the images of the first video feed. In order to delimit the effect on the perception of the video feed //^'even further, the images 4 can be presented at a higher intensity than the images and complementary images ( PFi) and {CF), respectively, so that the effect of neutral insert on the perception of the first video feed is minimized. In the present example, the neutral insert {N) further includes two black image slots {B) also having a duration of 2 ms, each, during which the LED pixels are effectively turned off. When using such a scheme, a camera filming a scenery including the physical display can be synchronized to the presentation of the second sequence of image data ( PF resulting in a video feed of the second media data which can be broadcast to viewers not present at the event or which can be used in video post production for image content replacement. Accordingly, as described in applicant’s International Patent Applications WO 2020/021067 A1 and WO 2020/021067 A1 mentioned above, different content for different sub-sets of viewers (especially direct viewers present at an event and broadcast viewers not present at an event) can be presented on the physical display without having a significant effect on the perception of the display by the direct viewers. As the time interval between individual images ( ) of the first sequence of image data, i.e. the media data presented to the direct viewers, is maximized, motion blur effects for the direct viewers are considerably reduced.

The embodiment of Fig. 4 essentially corresponds to the embodiment of Fig. 3, except that the parallel feed images ( PF of the additional media content are presented in two 2 ms slots in each frame time interval, i.e. for a total duration of 4 ms in each time interval. The corresponding complementary/inverse images {CF,) are presented within one 2 ms slot in each frame time interval but their intensity is increased as compared to the parallel feed images (PF) so that the combination of the two slots (PF) and one slot (OF,·) still result in an homogeneous grey image. In the remaining fifths slot of duration 2 ms, the neutral insert ( /) further includes a black image slot ( B) during which the LED pixels are effectively turned off. Such a scheme facilitates capturing the images (PF) with a synchronized camera because these images are presented for a longer time period on the physical display as compared to the embodiment of Fig. 3. The effect of the neutral insert ( /) on the perception of the main feed (/) is still kept low. The scheme described in Figures 2 to 4 can be adapted to particular applications as desired. For instance, with higher display refresh rates, each frame can be subdivided in more slots than depicted in the examples.

Claims

1. Method for reducing motion blur when presenting media data on a display, comprising: sending a first sequence of images (/,) from a computer device (11) to a display (16) at a frame rate f, displaying each image (/,) of said first sequence of images for a time interval DTi corresponding to a fraction of the frame time interval 1/f (AT_t = p 1 //), where /?less than 0.5, and after displaying each image (/,), presenting a neutral insert ( N) on said display for a time interval DT2 corresponding to the remainder of the frame time interval (AT₂ = (1 - p) 1/f).

2. Method according to claim 1 , wherein p is at least 0.1 and lower than 0.5 (0.1 £ p < 0.5).

3. Method according to one of claims 1 or 2, wherein said neutral insert (N) comprises a grey image or a black image ( B ).

4. Method according to one of claims 1 to 3, wherein said neutral insert ( N) comprises at least one additional sequence of images (PFJ) and complementary images (CFJ).

5. Method according to claim 4, wherein said at least one additional sequence of images (PFJ) and complementary images (CFJ) comprises second media data images and complementary images of said second media data images.

6. Method according to one of claims 4 or 5, wherein said at least one additional sequence of images (PFJ) and complementary images (CFJ) comprises chroma- key images and complementary chromakey images.

7. Method according to one of claims 4 to 6, wherein said at least one additional sequence of images (PFJ) and complementary images (CFJ) comprises tracking pattern images and complementary tracking pattern images.

8. Method according to one of claims 1 to 7, wherein said frame rate is at least 24 Hz, preferably at least 50 Hz.

9. Method according to claim 8, wherein said frame rate is at least 100 Hz.

10. Method according to claim 9, further comprising the steps of jointly displaying pairs of two consecutive images (/,) and (/,_+?) on a control monitor of said computer device (11), jointly grabbing each pair of two consecutive images by a frame grabber (13) of said computer device (14), jointly sending each pair of two consecutive images from a sending card (14) of said computer device to a receiving card (15) of said display and displaying each image (/,) and (/,_+?) separately on said display in two consecutive frame intervals.

11. The method of one of claims 1 to 10, wherein said display (16) is an active matrix display, preferably a LED display.

12. A device for reducing motion blur when presenting media data on a display comprising: a computer device (10) for generating a first sequence of images (/,), said computer device comprising at least one sending card (14) for sending said first sequence of images (/,) from said computer device (10) to a display (16), said display (16) comprising at least one receiving card (15) for storing and receiving said first sequence of images received from said sending card (14); means for displaying displaying each image (/,) for a time interval DTi corresponding to a fraction of the frame time interval 1/f (AT_t = p 1 //), where p less than 0.5, and after displaying each image (/,), for presenting a dark image (B) on said display for a time interval DT2 corresponding to the remainder of the frame interval (AT₂ = (1 - p) 1/f).

13. Device according to claim 12, further comprising a control monitor (12) and a frame grabber (13) for grabbing images shown on said control monitor and transferring said images to said sending card (14).