GB2526148A - Seamless display of a video sequence with increased frame rate - Google Patents

Seamless display of a video sequence with increased frame rate Download PDF

Info

Publication number
GB2526148A
GB2526148A GB1408731.6A GB201408731A GB2526148A GB 2526148 A GB2526148 A GB 2526148A GB 201408731 A GB201408731 A GB 201408731A GB 2526148 A GB2526148 A GB 2526148A
Authority
GB
United Kingdom
Prior art keywords
subdivision
pixel
image
video sequence
partitions
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
GB1408731.6A
Other versions
GB201408731D0 (en
GB2526148B (en
Inventor
Falk Tannhauser
Arnaud Closset
Tristan Halna Du Fretay
Brice Le Houerou
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon Inc
Original Assignee
Canon Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon Inc filed Critical Canon Inc
Priority to GB1408731.6A priority Critical patent/GB2526148B/en
Publication of GB201408731D0 publication Critical patent/GB201408731D0/en
Publication of GB2526148A publication Critical patent/GB2526148A/en
Application granted granted Critical
Publication of GB2526148B publication Critical patent/GB2526148B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/01Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level
    • H04N7/0127Conversion of standards, e.g. involving analogue television standards or digital television standards processed at pixel level by changing the field or frame frequency of the incoming video signal, e.g. frame rate converter
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136Incoming video signal characteristics or properties
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234381Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the temporal resolution, e.g. decreasing the frame rate by frame skipping

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

A method of processing image data of a video sequence having a first frame rate, for display at a second frame rate, comprising: successively receiving, within a transmission period corresponding to the first frame rate, a plurality of partitions of each image of the video sequence. Upon receipt of each partition, a current image of the video sequence to be displayed is updated, according to the second frame rate. The updating comprises updating at least one subdivision of the current image, based on a set of at least one pixel of said each partition, the set of at least one pixel originating from a corresponding subdivision of said each image. Independent claims to the partitioning of an image, and successive transmitting of the partitions within a transmission period corresponding to the first frame rate, are also disclosed. The method make it possible to take advantage of the capability of display devices to render video sequences at a high frame rate, thereby obtaining a smooth and seamless visual rendering, despite the video sequences are initially at a low frame rate.

Description

Intellectual Property Office Application No. GB1408731.6 RTTVT Date:10 November 2014 The following terms are registered trade marks and should be read as such wherever they occur in this document: Blu-ray Intellectual Property Office is an operating name of the Patent Office www.ipo.govuk
TITLE OF THE INVENTION
Seamless display of a video sequence with increased frame rate
FIELD OF THE INVENTION
The present invention relates to video data transmission from a source device to a display device. The present invention also relates to the control of the display of the video data.
BACKGROUND OF THE INVENTION
Video display devices, such as display screens, video projectors, multi-projection systems, etc. may display video sequences with a wide range of formats in terms of resolution, colour depth and frame rate, e.g. up to a 1920 x 1080 resolution at 24 bits/pixel and 240 frames/s.
Video source devices, such as cameras, DVD or Blu-ray players, personal computers, Set-top Boxes, etc. may deliver video pictures at a given format (given resolution, colour depth, etc.), but only at a considerably lower frame rate, e.g. 60 or 15 frames/s.
Furthermore, the video display device may be connected to the video source device through a network link (wire or wireless) offering only a limited bandwidth. For example, in order to transport HD video streams at a 1920 x 1080 resolution with 24 bits/pixel, corresponding to a 4:4:4 chroma subsampling, a frame rate of 60 or 15 frames/s only is usually used.
The above examples require a network bandwidth of 3 Gbit/s or 0.75 Gbit/s, this excluding the possible overhead for transmission protocol (frame headers) and error detection/correction.
Document US 2010/0265392 discloses a method of transmitting uncompressed video information over a wireless communication medium from a wireless sending device to a wireless receiving device. Progressive transmission data rate adaptation is performed on original uncompressed video pixel information. The progressive transmission data rate adaptation is performed on selected pixel information in order to obtain a rate-reduced pixel information. The rate-reduced pixel information requires a transmission data rate a lower than the data rate of the original frame. The rate-reduced pixel information is transmitted over the wireless communication medium to the wireless receiving device. At the receiver, a reverse operation is performed to recover the original uncompressed video pixel information.
This document discloses the creation of pixel partitions wherein neighbouring pixels are part of several partitions. It also discloses how to perform data rate adaptation to uncompressed video data according to the bandwidth of a wireless communication medium.
However, the transmission order of the adapted video data is not taken into account. Therefore, the target display device cannot perform a seamless display of the video sequence by creating and displaying supplementary interpolated video frames.
Document US 6,014,694 discloses a system for adaptively transporting video over networks, wherein the available bandwidth varies with time. The system comprises a video/audio codec configured to compress, code, decode and decompress video streams that are transmitted over networks having available bandwidths that vary with time and location. Depending on the channel bandwidth, the system adjusts the compression ratio to accommodate a plurality of bandwidths ranging from 20 Kbps for POTS to several Mbps for switched LAN and ATM environments. Bandwidth adjustability is provided by offering a trade-off between video resolution, frame rate and individual frame quality. The system generates a video data stream comprised of Key, P and B frames from a raw source of video. Each frame type is further comprised of multiple levels of data representing varying degrees of quality. In addition, several video server platforms can be utilized in tandem to transmit video/audio information with each video server platform transmitting information for a single compression/resolution level.
This document discloses how to adjust the video data compression ratio (adjusting resolution, frame rate, individual frame quality) according to the bandwidth of the communication system. However, it doesn't consider the transmission order of the adapted video data and hence it doesn't help the target display device in increasing the fluidity of the displayed video sequence by creating and displaying supplementary interpolated video frames.
Furthermore, it concerns only sources delivering compressed video.
Thus, there is still a need for improvements in video data transmission between a source device working at a given frame rate and a display device working at a higher frame rate.
The invention lies within this context.
SUMMARY OF THE INVENTION
According to a first aspect of the invention there is provided a method of processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein: each image of said video sequence is subdivided according to a subdivision grid into subdivisions, each subdivision comprising sets of at least one pixel, a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the method comprising the following steps, for each image of said video sequence: successively receiving, within a transmission period corresponding to the first frame rate, said plurality of partitions of said each image, upon receipt of each partition, updating, according to said second frame rate, a current image of the video sequence to be displayed, said current image being subdivided according to said subdivision grid, said updating comprising updating at least one subdivision of said current image, based on a set of at least one pixel of said each partition, said set of at least one pixel originating from a corresponding subdivision of said each image.
A method according to the first aspect makes it possible to take advantage of the capability of display devices to render video sequences at a high frame rate, thereby obtaining a smooth and seamless visual rendering, despite the video sequences are initially at a low frame rate.
A method according to the first aspect makes it possible to display video sequence at a high frame rate whereas the display device communicates with the video source device at a low frame rate.
Even though the video sequence is received at a low frame rate (due to the video source device or the network link for instance) the video sequence may be displayed at a high frame rate.
The apparent video frame rate is increased with a reduced visual deterioration of the video resolution and of the visual quality.
A method according to the first aspect can be easily implemented in terms of complexity, e.g. memory use and processing time.
According to embodiments, 15 or 60 frames/s video sequences may be rendered at a 240 frames/s display rate, while smoothing the visual appearance without requiring excessive additional buffer memory or costly supplemental processing by the display device or the video source device.
In addition, a method according to the first aspect makes it possible to facilitate the connection of a low-resolution display device (for example a control display) to a same video transmission network as a main display device (with higher resolution). Thus, the two devices may receive the same video data and the downscaled low-resolution version of the video sequence can be easily extracted by the low-resolution device display.
For example, said updating of said at least one subdivision is based on at least two partitions of a same image of said video sequence.
For example, said updating of said at least one subdivision is based on at least two partitions of at least two respective consecutive images of said video sequence.
According to embodiments, each set of at least one pixel of each subdivision of said current image is updated, based on sets of at least one pixel from respective partitions.
The method may further comprise determining said subdivision grid based on a regular subdivision of the images of the video sequence.
The method may further comprise determining said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
The method may further comprise assigning indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a source device from which originates said video sequence.
The method may further comprise receiving a definition of said subdivision grid from a source device from which originates said video sequence.
For example, said updating comprises determining a pixel parameter associated with a set of at least one pixel in said each partition, and applying said parameter to at least one set of at least one pixel of the subdivision corresponding to the subdivision from which originates the set of at least one pixel of said each partition.
For example, upon receipt of a first partition for an image of the video sequence, said parameter is applied to all the sets of at least one pixel in the subdivision.
According to embodiments, upon receipt of a further partition for said image of the video sequence, parameters applied to a first part of the sets of at least one pixel in the subdivision are maintained, while a parameter associated to a set of at least one pixel in said further partition is applied to a second part of the sets of at least one pixel in the subdivision.
For example, the sets of at least one pixel are updated within each subdivision of the current image from a center of the subdivision to the edges of the subdivision.
For example, the number of partitions received for each image corresponds to a ratio between the second frame rate and the first frame rate.
According to embodiments, each set of pixels of a partition has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
According to a second aspect of the invention, there is provided a method of processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein -each image of said video sequence is subdivided according to a subdivision grid, into subdivisions, each subdivision comprising sets of at least one pixel, -a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the method comprising the following steps, for each image of said video sequence: -partitioning said each image of said video sequence according to said subdivision grid, and -successively transmitting, within a transmission period corresponding to the first frame rate, said plurality of partitions of said each image.
The method may further comprise determining said subdivision grid based on a regular subdivision of the images of the video sequence.
The method may further comprise comprising determining said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
The method may further comprise assigning indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a display device receiving said plurality of partitions for display of said each image.
The method may further comprise transmitting a definition of said subdivision grid to said display device.
For example, the number of partitions transmitted for each image corresponds to a ratio between the second frame rate and the first frame rate.
For example, each set of pixels has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
According to a third aspect of the invention there are provided computer programs and computer program products comprising instructions for implementing methods according to the first, second and/or third aspect(s) of the invention, when loaded and executed on computer means of a programmable apparatus.
According to a fourth aspect of the invention, there is provided a device configured for implementing methods according to the first and/or second aspect(s).
According to a fifth aspect of the invention, there is provided a system comprising devices according to the fourth aspect.
BRIEF DESCRIPTION OF THE DRAWINGS
Other features and advantages of the invention will become apparent from the following description of non-limiting exemplary embodiments, with reference to the appended drawings, in which: -Figure 1 illustrates an exemplary system according to embodiments, -Figure 2A-2B illustrate a partitioning of a video image into 2 x 2 macro-blocks and the corresponding transmission time line and display update according to embodiments, -Figure 3A-3B illustrate a partitioning of a video image into 4 x 4 macro-blocks and the corresponding transmission time line and display update according to embodiments, -Figure 4A-4B illustrate a partitioning of a video image into 2 x 2 macro-blocks and the corresponding transmission time line and
B
display update according to embodiments, wherein data from the preceding original video frame is used, -Figure 5 illustrates an exemplary (pseudo) random partitioning of a video image data according to embodiments, -Figure 6 is a functional diagram of a video display device according to embodiments, -Figure 7 is a functional diagram of a video source device according to embodiments, -Figures 8A-8B are flowcharts of steps performed according to embodiments.
DETAILED DESCRIPTION OF THE INVENTION
In what follows, embodiments of the invention are described.
Figure 1 illustrates an exemplary video display system according to embodiments. A video source device 100 is connected to a video display device through a network 120.
The video source device maybe a camera, a DVD or Blu-ray player, a hard-disk based video storage, a personal computer, a Set-top Box, etc. The video source device delivers a video sequence to the video display device.
The video display device may be a screen, a video projector, a multi-projection system, etc. The video display device is configured to display the video sequence received from the video source device.
The network may be wired or wireless. Also, it may comprise a point-to-point link connecting the source 100 to the target 110, or comprise a network interconnecting said nodes as well as other nodes (not represented) sharing the medium among different connections.
The video source delivers video images at a given resolution (e.g. 1920 x 1080 pixels) and colour depth (e.g. 24 bits/pixel) that the display device is configured to process.
The display device supports a frame rate which is higher than the frame rate of the source device and/or the frame rate enabled by the available bandwidth of the network for the given resolution and colour depth. For example, the frame rate ratio is 1:4 (e.g. 15 vs. 60 or 60 vs. 240 frames/s) or 1:16 (e.g. 15 vs. 240 frames/s).
The video source device may perform a partitioning of video images into 2 x 2 macro-blocks as illustrated in Figure 2A.
Figure 2B illustrates the corresponding transmission time line for the video data transmitted through the network 120. Figure 2B also illustrates the display update which may be performed by the video display device 110. In the example presented in Figure 2B, the update is based only on the data from the same original video frame.
In Figure 2A there is shown a 6 x 4 extraction of a video frame belonging to the video sequence to be transmitted by the video source device 100. For example, the whole frame has a resolution of 1920 x 1080 pixels. Also, the original duration of a video frame is, for instance, 1/15 s. The video source device is configured to create an exemplary number of four partitions (or sub-frames"), numbered ito 4 in Figure 2A. Hence, the original frame is partitioned into macro-blocks of 2 x 2 pixels each and the pixels are assigned to their partitions according to their respective number (1 to 4). In the present example, the partitioning is performed according to a regular square grid. Each one of the four partitions has a size of 960 x 540 pixels and represents a down-sampled version of the original image. The combination of the four partitions represents the full original pixel information.
In Figure 2B, there is shown the transmission sequence of the partitions corresponding to consecutive original video frames over network 120.
For example, the duration of each partition is 1/60 s. The video display device, which receives the data, updates its 1920 x 1080 display of the video sequence as follows.
Upon receipt of the first partition (partition n° 1 in Figure 2B) of a given original video frame, the colour value of each received pixel is replicated in each of the 2 x 2 pixels of the corresponding macro-block. This results in fast display of an up-scaled low-resolution version of the original video frame.
Upon receipt of the second partition (partition n° 2 in Figure 2B) of the original video frame, the colour value of each received pixel is replicated in the lower 2 x 1 sub-block of the corresponding 2 x 2 macro-block, while the colour value from partition n° 1 is retained in the respective upper 2 x 1 sub-block. This results in visible improvement of the vertical resolution of the displayed image.
Upon receipt of the third and fourth partitions (partitions n° 1 and n° 2 in Figure 2B), the colour value of each received pixel is assigned to the respective position of said pixel in its original macro-block, finally resulting in displaying of the fully-resolved original image.
Upon receipt of the first partition (partition n° 1 in Figure 2B) of the subsequent original video frame, the above procedure is repeated.
In Figure 2B, pixels updated upon receipt of each frame partition are marked in bold font and pixels that have reached their final value while displaying a given original video frame are preceded by an asterisk (*) in Figure 2B.
Figure 3A illustrates a partitioning a video image data into 4 x 4 macro-blocks which may be performed by the video source device.
Figure 3B illustrates the corresponding transmission time line for the video data transmitted through the network. Figure 3B also illustrates the display update which may be performed by the display device.
In Figure 3A there is shown a 8 x 4 extraction of a video frame belonging to the video sequence to be transmitted by the video source device.
For example, the whole frame has a resolution of 1920 x 1080 pixels. The original duration of a video frame is, for instance, 1/15 s. The video source device may create sixteen partitions (or "sub-frames"), numbered 1 to 16 in Figure 3A. The original frame is thus partitioned into macro-blocks of 4 x 4 pixels each and the pixels are assigned to their partitions according to their respective number (1 to 16). In the present example, the image is partitioned according to a regular square grid. Consequently, each one of the four partitions has a size of 480 x 270 pixels and represents a down-sampled version of the original image. The combination of all the sixteen partitions represents the full original pixel information.
In Figure 3B there is shown the transmission sequence of the partitions corresponding to an original video frame as these data are transmitted over the network. The duration of each partition is 1/240 s. The video display device that receives the data may update the 1920 x 1080 display as follows.
Upon receipt of the first partition (partition n° 1 in Figure 3B) of a given original video frame, the colour value of each received pixel is replicated in each of the 4 x 4 pixels of the corresponding macro-block. This results in a fast display of an up-scaled low-resolution version of the original video frame.
Upon receipt of the second, third and fourth partitions (partitions n° 1, n° 2 an n° 3 in Figure 3B), the colour value of each received pixel is replicated in the respective 2 x 2 sub-block of the corresponding 4 x 4 macro-block, while the colour values from the preceding partitions are retained outside of said 2 x 2 sub-block. This results in visible improvement of the horizontal and vertical resolution of the displayed image.
Upon receipt of the fifth to eighth partitions (partitions n° 5 to n° 8 in Figure 3B), the colour value of each received pixel is replicated in the respective 2 x 1 sub-block of the corresponding 4 x 4 macro-block, while the colour values from the preceding partitions are retained outside of said 2 x 1 sub-block. This results in further visible improvement of the vertical resolution of the displayed image.
Upon receipt of the ninth to sixteenth partitions (partitions n° 9 to n° 16 in Figure 3B), the colour value of each received pixel is assigned to the respective position of said pixel in its original macro-block, finally resulting in displaying of the fully-resolved original image.
Upon receipt of the first partition (partition n° 1 in Figure 3B) of the subsequent original video frame, the above procedure is repeated.
In Figure 3B, pixels updated upon receipt of each frame partition are marked in bold font and pixels that have reached their final value while displaying a given original video frame are preceded by an asterisk (*).
While different transmission orders of the pixel partitions are possible, it may be advantageous to start by transmitting the partitions containing the pixels situated in the centre of the macro-blocks first (partitions 1 to 4), in order to reduce apparent movement induced by the successive refinement of the initial low-resolution version of the image.
Figure 4A illustrates another partitioning a video image data into 2 x 2 macro-blocks which may be performed by the video source device.
Figure 4B illustrates the corresponding transmission time line for the video data transmitted over the network. Figure 4B also illustrates the display update which may be performed by the video display device. In the example of Figure 4B, both the data from the same original video frame and from the preceding one are used.
The manner the display of the image is performed in the example of Figures 4A-4B is different from the update in the example of Figures 2A-2B in that upon receipt of each partition, the colour value of each received pixel is assigned to the respective position of said pixel in the original macro-block, while the remaining pixels of each macro-block retain their value (either from preceding partitions of the same original video frame, or from the preceding video frame). Thus, by having pixels retaining their values, displaying of low-resolution image versions exhibiting visually disturbing pixelization artefacts is avoided.
According to embodiments, partitioning of the images may be performed randomly (or pseudo-randomly) as illustrated in Figure 5.
The original video image may be partitioned into four partitions by the video source device 100 prior to transmission over network 120. The target video display device 110 may update the display of the image incrementally upon receipt of each partition, while the pixels not being part of said partitions retain their value either from their respective preceding partition corresponding to the same original video frame, or from the preceding original video frame (in a manner similar to the display update described with reference to Figures 4A-4B).
The video source device 100 and the display 110 may agree on the assignment of the pixels to the partitions, for example, by deriving the partitioning from a pseudorandom number generator (e.g. a linear congruential generator) using common, predefined, parameters and seed values. While Figure 5 depicts an 8 x 8 image, the principle applies to any image size as e.g. 1920 x 1080. In the example of Figure 5, the image has a total of 64 pixels with 16 pixels per partition, wherein the pixels numbered 1 to 16 belong to partition n° 1, the pixels numbered 17 to 32 belong to partition n° 2, the pixels numbered 33 to 48 belong to partition n° 3 and the pixels numbered 49 to 64 belong to partition n° 4).
One advantage of a using (pseudo) random pattern instead of a regular grid of macro-blocks may be the avoidance of visible regular pattern artifacts.
A video display device 110 according to embodiment is illustrated in Figure 6.
The device comprises a processor 601 which is configured to control the configuration tasks, and to execute several algorithms as described hereinabove.
The processor is connected to several functional blocks by an interconnection bus 603. The functional blocks may be configured by the processor through bus 603. The device comprises a random access memory (RAM) 600 for storing program instructions and data processed by the processor.
Video data are received from the network 120 through a network controller 608. The video data are then transferred to a video buffer controller 607 which controls the access to a video buffer 606, implemented in a RAM (Random Access Memory), and serving as temporary storage for the received video partitions.
The device further comprises a network controller 608 which provides synchronization signals to the synchronization controller 609. The synchronization controller 609 is configured to generate the video synchronization signals for the management of the video rendering. The video buffer controller 607 is also configured to read video data out from the video buffer 606 and to provide it to a local display controller 610. The local display controller is configured to perform a display update according to embodiments, as described herein above.
With reference to Figure 7, there is described a video source device according to embodiments.
The device comprises a processing unit 701 (CPU), a random access memory (RAM) 700 and a connection bus 703. The processing unit is configured to perform image partitioning as described herein above.
The device 100 comprises a video source module 711 for generating video data or for reproducing stored video data. A video source interface (IF) module 704 is configured to receive video data and synchronization information from the video source. The interface 704 outputs the video data to a video buffer controller 707 and the synchronization signals to a synchronization controller 709. The video buffer controller 707 is configured to control the access to the video buffer 706, implemented in a RAM (Random Access Memory). It writes data from the video source interface 704 into the video buffer 706. It also reads video data out from the video buffer 706 (performing partitioning as described hereinabove) and provides it to the network controller 708.
Figure 8A is a flowchart of steps performed for transmitting a video stream according to embodiments. The steps may be performed by the video source device 100. A video transmission request is received during step 800.
Next, the source device transmits through the network 120 to the display device 110, step 801, the relevant video parameters, such as the horizontal and vertical resolution, the pixel depth in bits, the colour coding (e.g. RGB or YCbCr), the chroma subsampling (if applicable) the frame rate, or the like.
It is then awaited, during step 805, receipt of a message containing the partitioning scheme (together with a corresponding transmission sequence) that is requested by the display device 110. The scheme may correspond, for example, to one of the schemes presented in Figure 2, 3, 4 or 5. The parameters contained in this scheme are presented below with reference to Figure SB (step 853).
Next, the video stream transmission comprises two nested loops, wherein in the inner loop (comprising steps 806 and 808) the video data generated from a single original video frame are transmitted. During step 806, a video frame partition is extracted from the original video data according to the scheme received in step 805 and transmitted through the network 120 to the display device 110 (original video frames may be dropped according to the partitioning scheme if necessary in order to accommodate for low available network bandwidth). Step 808 is a test by which it is checked whether or not all partitions belonging to the current original video frame were transmitted. In case they are not, the process continues with step 806, in case they are, with step 809.
During step 809, it is tested whether or not there are any further original video frames to be transmitted. In case there are, the process continues with step 806, in case there are not it stops with step 810.
Figure 8B is a flowchart of steps performed for transmitting a video stream according to embodiments. The steps may be performed by the video display device 110.
After an initialization step 850, it is awaited during step 851 for the receipt of a message containing video parameters as discussed hereinabove for step 801. Next, during step 852, the display device determines the bandwidth available on network 120 for video transmission. During step 853, the device then determines from said video parameters and the network bandwidth: * The maximal possible rate at which original video frames can be transmitted from source 100 to display 110 over network 120 (source 100 may need to drop some original frames, effectively decreasing its original video frame rate, if the bandwidth would be insufficient otherwise); * The number of partitions (equal to the ratio of the rate at which the display 110 receives original video frames and the rate at which it updates its display); * The actual partitioning scheme -either the size and form of macro-blocks together with the transmission order of the individual pixels (in case of schemes conforming e.g. to Figure 2, 3 or 4), or the parameters for pseudo-random partitioning (e.g. conforming to figure 5).
Next, during step 855, the aforementioned data are transmitted to the video source 100 (in the message awaited in step 805 described with reference to Figure 8A).
During step 856, it is awaited for receipt of a video frame partition (as transmitted by the source 100 during step 806). Next, during step 857, the display is incrementally updated using the data from the previously received video transmission, as explained e.g. with reference to Figures 2, 3, 4 or 5 (note that during processing of the first partition of the first original video frame in the stream, the display is initially assumed to be in a predetermined state e.g. completely black). During step 858, it is tested whether or not there are more video partitions to be received. In case there are, the process continues with step 856, otherwise it stops with step 860.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive, the invention being not restricted to the disclosed embodiment. Other variations to the disclosed embodiment can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure and the appended claims.
In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfil the functions of several items recited in the claims. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be advantageously used. Any reference signs in the claims should not be construed as limiting the scope of the invention.

Claims (47)

  1. CLAIMS1. A method of processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein -each image of said video sequence is subdivided according to a subdivision grid into subdivisions, each subdivision comprising sets of at least one pixel, -a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the method comprising the following steps, for each image of said video sequence: -successively receiving, within a transmission period corresponding to the first frame rate, said plurality of partitions of said each image, -upon receipt of each partition, updating, according to said second frame rate, a current image of the video sequence to be displayed, said current image being subdivided according to said subdivision grid, said updating comprising updating at least one subdivision of said current image, based on a set of at least one pixel of said each partition, said set of at least one pixel originating from a corresponding subdivision of said each image.
  2. 2. A method according to claim 1, wherein said updating of said at least one subdivision is based on at least two partitions of a same image of said video sequence.
  3. 3. A method according to claim 1, wherein said updating of said at least one subdivision is based on at least two partitions of at least two respective consecutive images of said video sequence.
  4. 4. A method according to any one of the preceding claims, wherein each set of at least one pixel of each subdivision of said current image is updated, based on sets of at least one pixel from respective partitions.
  5. 5. A method according to any one of the preceding claims, further comprising determining said subdivision grid based on a regular subdivision of the images of the video sequence.
  6. 6. A method according to any one of claims 1 to 4, further comprising determining said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
  7. 7. A method according to claim 6, further comprising assigning indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a source device from which originates said video sequence.
  8. 8. A method according to any one of claims 6 and 7, further comprising receiving a definition of said subdivision grid from a source device from which originates said video sequence.
  9. 9. A method according to any one of the preceding claims, wherein said updating comprises -determining a pixel parameter associated with a set of at least one pixel in said each partition, and -applying said parameter to at least one set of at least one pixel of the subdivision corresponding to the subdivision from which originates the set of at least one pixel of said each partition.
  10. 10. A method according to claim 9, wherein, upon receipt of a first partition for an image of the video sequence, said parameter is applied to all the sets of at least one pixel in the subdivision.
  11. 11. A method according to claim 10, wherein, upon receipt of a further partition for said image of the video sequence, parameters applied to a first part of the sets of at least one pixel in the subdivision are maintained, while a parameter associated to a set of at least one pixel in said further partition is applied to a second part of the sets of at least one pixel in the subdivision.
  12. 12. A method according to any one of the preceding claims, wherein the sets of at least one pixel are updated within each subdivision of the current image from a center of the subdivision to the edges of the subdivision.
  13. 13. A method according to any one of the preceding claims, wherein the number of partitions received for each image corresponds to a ratio between the second frame rate and the first frame rate.
  14. 14. A method according to any one of the preceding claims, wherein each set of pixels of a partition has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
  15. 15. A method of processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein -each image of said video sequence is subdivided according to a subdivision grid, into subdivisions, each subdivision comprising sets of at least one pixel, -a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the method comprising the following steps, for each image of said video sequence: -partitioning said each image of said video sequence according to said subdivision grid, and -successively transmitting, within a transmission period corresponding to the first frame rate, said plurality of partitions of said each image.
  16. 16. A method according to claim 15, further comprising determining said subdivision grid based on a regular subdivision of the images of the video sequence.
  17. 17. A method according to any one of claims 15 to 16, further comprising determining said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
  18. 18. A method according to claim 17 further comprising assigning indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a display device receiving said plurality of partitions for display of said each image.
  19. 19. A method according to any one of claims 17 and 18, further comprising transmitting a definition of said subdivision grid to said display device.
  20. 20. A method according to any one of claims 15 to 19, wherein the number of partitions transmitted for each image corresponds to a ratio between the second frame rate and the first frame rate.
  21. 21. A method according to any one of claims 15 to 20, wherein each set of pixels has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
  22. 22. A computer program product comprising instructions for implementing a method according to any one of claims 1 to 21 when the program is loaded and executed by a programmable apparatus.
  23. 23. A non-transitory information storage means readable by a computer or a microprocessor storing instructions of a computer program, for implementing a method according to any one of claims 1 to 21, when the program is loaded and executed by the computer or microprocessor.
  24. 24. A device for processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein -each image of said video sequence is subdivided according to a subdivision grid into subdivisions, each subdivision comprising sets of at least one pixel, -a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the device comprising: -a communication unit configured to successively receive, within a transmission period corresponding to the first frame rate, said plurality of partitions of each image of said video sequence, and -a processing unit configured to update, upon receipt of each partition, according to said second frame rate, a current image of the video sequence to be displayed, said current image being subdivided according to said subdivision grid, said updating comprising updating at least one subdivision of said current image, based on a set of at least one pixel of said each partition, said set of at least one pixel originating from a corresponding subdivision of said each image.
  25. 25. A device according to claim 24, wherein said updating of said at least one subdivision is based on at least two partitions of a same image of said video sequence.
  26. 26. A method according to claim 24, wherein said updating of said at least one subdivision is based on at least two partitions of at least two respective consecutive images of said video sequence.
  27. 27. A device according to any one of claims 24 to 26, wherein each set of at least one pixel of each subdivision of said current image is updated, based on sets of at least one pixel from respective partitions.
  28. 28. A device according to any one of claims 24 to 27, wherein said processing unit is further configured to determine said subdivision grid based on a regular subdivision of the images of the video sequence.
  29. 29. A device according to any one of claims 24 to 27, wherein said processing unit is further configured to determine said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
  30. 30. A device according to claim 29, wherein said processing unit is further configured to assign indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a source device from which originates said video sequence.
  31. 31. A device according to any one of claims 29 and 30, wherein said communication unit is further configured to receive a definition of said subdivision grid from a source device from which originates said video sequence.
  32. 32. A device according to any one of claims 24 to 31, wherein said updating comprises: -determining a pixel parameter associated with a set of at least one pixel in said each partition, and -applying said parameter to at least one set of at least one pixel of the subdivision corresponding to the subdivision from which originates the set of at least one pixel of said each partition.
  33. 33. A device according to claim 32, wherein, upon receipt of a first partition for an image of the video sequence, said parameter is applied to all the sets of at least one pixel in the subdivision.
  34. 34. A device according to claim 33, wherein, upon receipt of a further partition for said image of the video sequence, parameters applied to a first part of the sets of at least one pixel in the subdivision are maintained, while a parameter associated to a set of at least one pixel in said further partition is applied to a second part of the sets of at least one pixel in the subdivision.
  35. 35. A device according to any one of claims 24 to 34, wherein the sets of at least one pixel are updated within each subdivision of the current image from a center of the subdivision to the edges of the subdivision.
  36. 36. A device according to any one of claims 24 to 35, wherein the number of partitions received for each image corresponds to a ratio between the second frame rate and the first frame rate.
  37. 37. A device according to any one of claims 24 to 36, wherein each set of pixels of a partition has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
  38. 38. A device for processing image data of a video sequence having a first frame rate, for display at a second frame rate, wherein -each image of said video sequence is subdivided according to a subdivision grid, into subdivisions, each subdivision comprising sets of at least one pixel, -a plurality of partitions are defined for each image of said video sequence, each partition comprising one set of at least one pixel of each subdivision of said each image, the device comprising: -a processing unit configured to partition each image of said video sequence according to said subdivision grid, and -a communication unit configured to successively transmit, within a transmission period corresponding to the first frame rate, said plurality of partitions of said each image.
  39. 39. A device according to claim 38, wherein said processing unit is further configured to determine said subdivision grid based on a regular subdivision of the images of the video sequence.
  40. 40. A device according to any one of claims 38 to 39, wherein said processing unit is further configured to determine said subdivision grid based on a pseudo random subdivision of the images of the video sequence.
  41. 41. A device according to claim 40, wherein said processing unit is further configured to assign indexes to the sets of at least one pixel, based on pseudo randomly generated numbers generated based on at least one common seed parameter shared with a display device receiving said plurality of partitions for display of said each image.
  42. 42. A device according to any one of claims 40 and 41, wherein said communication unit is further configured to transmit a definition of said subdivision grid to said display device.
  43. 43. A device according to any one of claims 38 to 42, wherein the number of partitions transmitted for each image corresponds to a ratio between the second frame rate and the first frame rate.
  44. 44. A device according to any one of claims 38 to 43, wherein each set of pixels has a same position within each respective subdivision, the sets of at least one pixel having respective positions within the partitions corresponding to the position of their respective subdivision in the subdivision grid.
  45. 45. A system comprising: -at least one device according to any one of claims 38 to 44, and -at least one device according to any one of claims 24 to 44.
  46. 46. A device substantially as hereinbefore described with reference to, and as shown in, Figures 6 and 7 of the accompanying drawings.
  47. 47. A method substantially as hereinbefore described with reference to, and as shown in, Figures 8A and SB of the accompanying drawings.
GB1408731.6A 2014-05-16 2014-05-16 Seamless display of a video sequence with increased frame rate Active GB2526148B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
GB1408731.6A GB2526148B (en) 2014-05-16 2014-05-16 Seamless display of a video sequence with increased frame rate

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1408731.6A GB2526148B (en) 2014-05-16 2014-05-16 Seamless display of a video sequence with increased frame rate

Publications (3)

Publication Number Publication Date
GB201408731D0 GB201408731D0 (en) 2014-07-02
GB2526148A true GB2526148A (en) 2015-11-18
GB2526148B GB2526148B (en) 2016-06-08

Family

ID=51134981

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1408731.6A Active GB2526148B (en) 2014-05-16 2014-05-16 Seamless display of a video sequence with increased frame rate

Country Status (1)

Country Link
GB (1) GB2526148B (en)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105609014A (en) * 2016-04-01 2016-05-25 北京小鸟科技发展有限责任公司 Implementation method for seamless splicing point-to-point display of liquid crystal screens
WO2019001723A1 (en) * 2017-06-30 2019-01-03 Nokia Solutions And Networks Oy Real time video
US11132758B2 (en) * 2016-09-14 2021-09-28 Inscape Data, Inc. Embedding data in video without visible impairments

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110141350A1 (en) * 2009-12-16 2011-06-16 Canon Kabushiki Kaisha Image processing apparatus, control method thereof, and program
US20140002739A1 (en) * 2012-06-28 2014-01-02 Seh Kwa Method and apparatus for reducing power usage during video presentation on a display

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110141350A1 (en) * 2009-12-16 2011-06-16 Canon Kabushiki Kaisha Image processing apparatus, control method thereof, and program
US20140002739A1 (en) * 2012-06-28 2014-01-02 Seh Kwa Method and apparatus for reducing power usage during video presentation on a display

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105609014A (en) * 2016-04-01 2016-05-25 北京小鸟科技发展有限责任公司 Implementation method for seamless splicing point-to-point display of liquid crystal screens
US11132758B2 (en) * 2016-09-14 2021-09-28 Inscape Data, Inc. Embedding data in video without visible impairments
WO2019001723A1 (en) * 2017-06-30 2019-01-03 Nokia Solutions And Networks Oy Real time video
CN111034199A (en) * 2017-06-30 2020-04-17 诺基亚通信公司 Real-time video

Also Published As

Publication number Publication date
GB201408731D0 (en) 2014-07-02
GB2526148B (en) 2016-06-08

Similar Documents

Publication Publication Date Title
US10817987B2 (en) Method and apparatus for streaming data
JP6934908B2 (en) Video content distribution method and video processing equipment
US9906765B2 (en) Transmitting display management metadata over HDMI
US9967599B2 (en) Transmitting display management metadata over HDMI
US9177359B2 (en) Information processor, cloud platform, information processing method, and computer program product thereof
KR101668858B1 (en) Method for transmitting multi-channel video stream, and surveillance system using the same method
JP7262877B2 (en) Adaptive High Dynamic Range Tonemapping with Overlay Directives
EP3685575B1 (en) Display apparatus, method for controlling the same and image providing apparatus
JP5969597B2 (en) Data transmission method, data reception method, wireless transmitter, and wireless receiver
US10389908B2 (en) Image processing device, image processing method, and program with reduction and enlargement scaling of image data
JP2016046778A (en) Video compression apparatus and video reproducing device
US10181296B2 (en) Display device and display method
CN109429085B (en) Display device and image processing method thereof
CN114631315A (en) Image encoding method and apparatus, and image decoding method and apparatus
GB2526148A (en) Seamless display of a video sequence with increased frame rate
JP5941000B2 (en) Video distribution apparatus and video distribution method
JP2012220671A (en) Display system and control method thereof
US8994789B2 (en) Digital video signal, a method for encoding of a digital video signal and a digital video signal encoder
WO2011134373A1 (en) Method, device and system for synchronous transmmision of multi-channel videos
KR101967819B1 (en) Imaging processing device for playing multiple videos based on tiled image and method for constructing the tiled image thereof
US11659223B2 (en) System, device and method for displaying display-dependent media files
KR102411911B1 (en) Apparatus and method for frame rate conversion
KR102213423B1 (en) Apparatus and method for distributing of ultra high definition videos using scalers
US20230196505A1 (en) Artificial intelligence-based image providing apparatus and method, and artificial intelligence-based display apparatus and method
US20230086916A1 (en) Image processing apparatus and image processing method