US9270916B2 - Method and system of improving quality of video beaming - Google Patents

Method and system of improving quality of video beaming Download PDF

Info

Publication number
US9270916B2
US9270916B2 US13/762,960 US201313762960A US9270916B2 US 9270916 B2 US9270916 B2 US 9270916B2 US 201313762960 A US201313762960 A US 201313762960A US 9270916 B2 US9270916 B2 US 9270916B2
Authority
US
United States
Prior art keywords
beaming
video
frames
application
change
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US13/762,960
Other versions
US20130208789A1 (en
Inventor
Matan SHAPIRA
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intel Corp
Original Assignee
Screenovate Technologies Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Screenovate Technologies Ltd filed Critical Screenovate Technologies Ltd
Priority to US13/762,960 priority Critical patent/US9270916B2/en
Publication of US20130208789A1 publication Critical patent/US20130208789A1/en
Assigned to SCREENOVATE TECHNOLOGIES LTD. reassignment SCREENOVATE TECHNOLOGIES LTD. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SHAPIRA, MATAN
Application granted granted Critical
Publication of US9270916B2 publication Critical patent/US9270916B2/en
Assigned to INTEL CORPORATION reassignment INTEL CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SCREENOVATE TECHNOLOGIES LTD.
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/38—Transmitter circuitry for the transmission of television signals according to analogue transmission standards
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • H04N19/00169—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/162—User input
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
    • H04N21/41407—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance embedded in a portable device, e.g. video client on a mobile phone, PDA, laptop
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/44004—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving video buffer management, e.g. video decoder buffer or video display buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
    • H04N21/4402—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display
    • H04N21/440281—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving reformatting operations of video signals for household redistribution, storage or real-time display by altering the temporal resolution, e.g. by frame skipping
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
    • H04N21/845—Structuring of content, e.g. decomposing content into time segments
    • H04N21/8455—Structuring of content, e.g. decomposing content into time segments involving pointers to the content, e.g. pointers to the I-frames of the video stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/12—Picture reproducers
    • H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
    • H04N9/3179—Video signal processing therefor
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2340/00—Aspects of display data processing
    • G09G2340/02—Handling of images in compressed format, e.g. JPEG, MPEG
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G2350/00—Solving problems of bandwidth in display systems
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4782—Web browsing, e.g. WebTV

Definitions

  • the present invention generally relates to the field of beaming data of a Graphical User Interface (GUI) screen of an application on a remote screen by using video beaming technology.
  • GUI Graphical User Interface
  • Some video encoders when configured to operate below a maximum output bit rate, increase the compression of video frames to maintain frame rate (i.e. not dropping frames) which results in decrease in the quality of images, especially when the rate of the input frame is high.
  • the encoded application video beaming i.e. images
  • the encoded application video beaming may be perceived by a user as in a decreased quality in some “key periods”.
  • a method for improving quality of video beaming of any content by a beaming application that is running on a computerized mobile device is provided herein.
  • the method is comprising the steps of: (i) selecting content for beaming; (ii) beaming selected content by utilizing a beaming application; (iii) identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application which starts a set of critical frames; (iv) performing quality improvement of the video beaming of the set of critical frames based on the identified pattern changes; and (v) identifying in real time a second pattern change in display video to a lower delivered data rate which ends the set of critical frames, wherein the identifying and the performing is running on at least one processing unit.
  • higher delivered data rate is compressed data.
  • higher delivered data rate is high bandwidth usage.
  • the identifying is achieved by analyzing characteristics of the video frames.
  • the performing of quality improvement of the video beaming is activated by selecting frames of the beaming video in low quality for quality improvement and applying quality improvement on the selected frames.
  • the performing of quality improvement is achieved by generating an additional frame with a higher quality and adding the additional frame to the video beaming.
  • the identifying of the change which signifies transition in content displaying pattern is performed according to information that is received from a User interface (UI) by attaching metadata tags which includes the information related to the beaming video frames.
  • UI User interface
  • the UI is of an Operating System (OS).
  • OS Operating System
  • the UI is of the application that is running on the computerized mobile device.
  • the identifying of the change which signifies pattern change in displayed video is performed according to information that is received from an Operation System (OS) or a User interface (UI) by sending metadata tags and identification (ID) of the critical frames.
  • OS Operation System
  • UI User interface
  • the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
  • the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames.
  • a beaming application for improving quality of video beaming by a processing unit that is running on a computerized mobile device.
  • the beaming application comprising: (i) a detector module for identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application; and (ii) a video encoding module for performing quality improvement of the beaming video after the identified a pattern change in beamed video.
  • the detector module is comprising of a frame analyzing module for monitoring bit rate of frames of the beaming video and detecting a change which signifies transition in usage of the application which starts a set of critical frames.
  • the detector module is part of a UI framework module for identifying critical frames based on information that is received from UI components; and a surface composition module for receiving information regarding critical frames from OS is part of the UI framework.
  • the UI framework module is further creating metadata tags to the identified critical frames and attaching the metadata tags to the critical frames.
  • the UI framework module is further creating metadata tags to the identified critical frames and identification (ID) of the critical frames and sends is separately from the critical frames.
  • the surface composition module is further attaching metadata tags which includes the information related to the beaming video frames.
  • the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
  • the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames.
  • the detector module identifies beginning of a set of critical frames and an end of the set of critical frames.
  • FIG. 1A is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is separate to a UI framework, according to some embodiments of the invention
  • FIG. 1B is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is part of a UI framework, according to some embodiments of the invention
  • FIG. 2 is a flowchart illustrating a process of video encoding module, according to some embodiments of the invention
  • FIG. 3 is a flowchart illustrating a process of frames analyzing module, according to some embodiments of the invention.
  • FIG. 4 is a flowchart illustrating a process of User Interface (UI) framework, according to some embodiments of the invention.
  • FIG. 5 is a block diagram illustrating a process of Surface composition module, according to some embodiments of the invention.
  • GUI Screen Graphical User Interface
  • key period as used herein in this application is defined as a period in which there is a transition in content displaying pattern of an application which starts a set of critical frames, For example, when a user focuses in a specific area in a screen to perceive the details of the content that is appearing on the screen after an action of the user such as scrolling. For example, when the user is reading a text or watching an image or reviewing a webpage or increase in content changes which appear on the screen in a non limiting example, when the user is moving the mouse.
  • the key period is as opposed to a period when scrolling a webpage, or while loading an application on a computerized mobile device.
  • That period is quite a passive period in which the user is not focused on the screen, and not trying to perceive the details of the content that is displayed on the screen.
  • a video movie may be comprised of parts with higher level of compression of data and other parts with a lower level of compression.
  • a key period may be a period when there is an increase in content changes in a specific frame.
  • beaming as used in this application is defined as the communication of screen content encoded data between a computerized mobile device and a target display device such as iPhone airplay mirroring.
  • delivered data is defined as the delivered data before compression or before high bandwidth usage.
  • I-frame intra-coded pictures
  • P-frames Predicted picture
  • B-frames Bi-predicted picture
  • video beaming as used in this application is defined as the action of projecting content of an application that is running on a computerized mobile device on a remote screen by utilizing video format technology and video transmission protocols.
  • the present invention provides a system and method for improving the user's experience by increasing the image quality in key periods in which an output data of a video encoder is displayed in low quality.
  • a user may beam content that is displayed on a screen of a computerized mobile device.
  • the content may be beamed by a beaming application that is utilizing video technology.
  • the method according to the present invention improves user's experience by increasing the quality of a selected frame at the key period.
  • the frame is selected by identifying the last frame of an action which is known to generate a high frame rate which results in low quality video output. Examples for such actions, which generate a high frame rate, are scrolling, rotating, switching between application screens or animations.
  • the identifying may be achieved by analyzing characteristics of the video frames in different techniques as will be further explained below.
  • an improvement in the quality of the frame during the key period is a result of a first set of low quality frames before the key period (during the action period) and a frame with a higher quality at the beginning of the key period.
  • the higher quality frame is configured as an I-frame.
  • the user's experience is improved by adding an additional high quality frame (instead of increasing a quality of a frame), at the beginning of the key period.
  • the added frame is identical to the last low quality frame, having less compressed rate and hence in higher quality.
  • the added frame is configured as an I-frame.
  • a detection process for identifying in real time, a critical frame in key periods for providing higher quality frame in this period can be implemented using two techniques.
  • the two techniques of detecting the critical frames may exist at the same implementation, and complete the operation of each other.
  • one technique is using metadata tags that maybe attached to frames based on information that is arriving from various Operating Systems (OS)s and User Interface (UI) components.
  • the metadata tags may be separate from the frames and may be sent with frames ID.
  • the metadata tags indicate for each frame if it's the critical frame in the timeline at the beginning of a key period. For example, attaching this metadata tags to a frame according to information from a scrolling UI component indicating that this rendered frame is the last frame of a key period in a non limiting example, a scroll of website action.
  • second technique is implemented by analyzing the output bit rate of the encoder in real time for detecting beginning of a key period.
  • the key period is characterized by a known pattern of bit rate.
  • the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames.
  • FIG. 1A is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is separate to a UI framework, according to some embodiments of the invention.
  • the computerized mobile device is comprised of: (i) surface composition module 500 (which is part of the OS of the computerized mobile device) that is enabling to integration of a surface of multiple applications into a single frame, to be displayed on a target display such as a television (TV) screen; (ii) a beaming module 600 to display on a target screen; (iii) a video encoding manager module 200 for managing and sending instructions related to the compression of the beamed video to a video encoder 700 ; (iv) a frame analyzing module 300 for analyzing the bit rate which is embedded in detector module 450 ; and (v) a UI framework code 400 (which is part of the computerized mobile device OS) for identifying critical frames.
  • surface composition module 500 which is part of the OS of the computerized mobile device
  • FIG. 1B is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module 450 is part of a UI framework, according to some embodiments of the invention.
  • FIG. 2 is a flowchart illustrating at least part of the process of Video encoding module 200 , according to some embodiments of the present invention.
  • the process of Video encoding module 200 monitors video frames (stage 212 ) and receives indication of critical frames (stage 214 ). Next, according to received indication frames are selected for quality improvement (stage 216 ). Next, the qualities of the selected frames in the key period are improved (stage 218 ), providing improved experience to the user/viewer. The improvement may be achieved by changing the frames quality or adding new identical frames in better/higher quality.
  • FIG. 3 is a flowchart illustrating at least part of the process of the frames analyzing module 300 according to some embodiments of the present invention.
  • the process of the frames analyzing module 300 monitors the bit rate or the change of content (stage 312 ), for detecting a predefined pattern of bit rate period such as key period in which a high bit rate period with minimal length is followed by a low bit period or a period with no input frames or for identifying increase of change of content (stage 314 ).
  • identifying in real time a pattern change in displayed video wherein the pattern change signifies transition from a rate of a predefined size of chunk of delivered data to a higher delivered data rate in content displaying pattern by the application which starts a set of critical frames.
  • the detected frames within the key period are marked and reported to the encoding module (stage 316 ).
  • FIG. 4 is a flowchart schematically illustrating at least part of one of the UI framework processes, according to some embodiments of the present invention.
  • the code identifies change which signifies transition in content displaying pattern is performed according to information that is received from a User interface (UI), such as end of scrolling of a webpage (stage 412 ) and creates metadata tags which include indication information of frames which are part of a key period (optionally) (stage 414 ).
  • the metadata tags are attached to the relevant frames (optionally) (stage 416 ).
  • sending the metadata tags separate from the relevant frames using frame IDs.
  • the tagged frames are enhanced by the encoding video module 300 for improving the quality.
  • the UI is of an Operating System (OS) or of the application that is running on the computerized mobile device.
  • OS Operating System
  • FIG. 5 is a flowchart schematically illustrating at least part of the process of surface composition module according to some embodiments of the present invention.
  • the modules receives surfaces (screen images) from various applications running on the computerized mobile device and system UI components with their metadata tags (stage 512 ) and compose the visible surfaces to a single frame which is composition of all visible application surfaces.
  • the module merges the metadata of the invalidated (dirty) areas (modified area of the application) (stage 514 ). Finally, the composed frame and the merged metadata tags (stage 516 ) are posted.
  • the beamed content is different from the displayed content on the mobile device screen.
  • the surface composition module may be a detector module.
  • Methods of the present invention may be implemented by performing or completing manually, automatically, or a combination thereof, selected steps or tasks.
  • the term “method” may refer to manners, means, techniques and procedures for accomplishing a given task including, but not limited to, those manners, means, techniques and procedures either known to, or readily developed from known manners, means, techniques and procedures by practitioners of the art to which the invention belongs.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Abstract

A method for improving quality of video beaming of any content by a beaming application that is running on a computerized mobile device is provided herein. The method comprising the steps of: (iii) selecting content for beaming; (ii) beaming by utilizing a beaming application; (iii) identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application which starts a set of critical frames; (iv) performing quality improvement of the video beaming of the set of critical frames based on the identified pattern changes; and (v) identifying in real time a second pattern change in display video to a lower delivered data rate which ends the set of critical frames.

Description

BACKGROUND
1. Technical Field
The present invention generally relates to the field of beaming data of a Graphical User Interface (GUI) screen of an application on a remote screen by using video beaming technology.
2. Discussion of Related Art
Some video encoders, when configured to operate below a maximum output bit rate, increase the compression of video frames to maintain frame rate (i.e. not dropping frames) which results in decrease in the quality of images, especially when the rate of the input frame is high. When such video encoder is being used to encode a main display of a computing/media device, the encoded application video beaming (i.e. images) may be perceived by a user as in a decreased quality in some “key periods”.
An example for such “key periods” with decreased quality of images occurs immediately after a scrolling operation in a browser. The scrolling operation creates a high frame rate, as a result the encoder increases the compression of video frames and the quality of the images decreases.
In a “key period” which follows the end of the scrolling, when the user wants to read a text in the point where it was scrolled to in the webpage, the captured video of the beamed browser is displayed in a low quality to the user, which results in poor experience for the user of reading the text and watching pictures which are embedded in the text. After the user scrolled the webpage and reached a point in the text where the user is interested in i.e. once the user stopped scrolling the webpage and input frame rate is decreased or stopped or increase in content changes which appear on the screen (when the user is moving the mouse), the encoder starts outputting standard quality frame or gradually increase the frame quality. However, from the moment the user started scrolling and until the user reached a point of interest in the text, the user is provided with a low quality image of the video beaming.
BRIEF SUMMARY
According to some embodiments of the invention, a method for improving quality of video beaming of any content by a beaming application that is running on a computerized mobile device is provided herein. The method is comprising the steps of: (i) selecting content for beaming; (ii) beaming selected content by utilizing a beaming application; (iii) identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application which starts a set of critical frames; (iv) performing quality improvement of the video beaming of the set of critical frames based on the identified pattern changes; and (v) identifying in real time a second pattern change in display video to a lower delivered data rate which ends the set of critical frames, wherein the identifying and the performing is running on at least one processing unit.
According to some embodiments of the invention, higher delivered data rate is compressed data.
According to some embodiments of the invention, higher delivered data rate is high bandwidth usage.
According to some embodiments of the invention, the identifying is achieved by analyzing characteristics of the video frames.
According to some embodiments of the invention, the performing of quality improvement of the video beaming is activated by selecting frames of the beaming video in low quality for quality improvement and applying quality improvement on the selected frames.
According to some embodiments of the invention, the performing of quality improvement is achieved by generating an additional frame with a higher quality and adding the additional frame to the video beaming.
According to some embodiments of the invention, the identifying of the change which signifies transition in content displaying pattern is performed according to information that is received from a User interface (UI) by attaching metadata tags which includes the information related to the beaming video frames.
According to some embodiments of the invention, the UI is of an Operating System (OS).
According to some embodiments of the invention, the UI is of the application that is running on the computerized mobile device.
According to some embodiments of the invention, the identifying of the change which signifies pattern change in displayed video is performed according to information that is received from an Operation System (OS) or a User interface (UI) by sending metadata tags and identification (ID) of the critical frames.
According to some embodiments of the invention, the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
According to some embodiments of the invention, the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames. (i) According to some embodiments of the invention, a beaming application for improving quality of video beaming by a processing unit that is running on a computerized mobile device is provided herein. The beaming application comprising: (i) a detector module for identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application; and (ii) a video encoding module for performing quality improvement of the beaming video after the identified a pattern change in beamed video.
According to some embodiments of the invention, the detector module is comprising of a frame analyzing module for monitoring bit rate of frames of the beaming video and detecting a change which signifies transition in usage of the application which starts a set of critical frames.
According to some embodiments of the invention, the detector module is part of a UI framework module for identifying critical frames based on information that is received from UI components; and a surface composition module for receiving information regarding critical frames from OS is part of the UI framework.
According to some embodiments of the invention, the UI framework module is further creating metadata tags to the identified critical frames and attaching the metadata tags to the critical frames.
According to some embodiments of the invention, the UI framework module is further creating metadata tags to the identified critical frames and identification (ID) of the critical frames and sends is separately from the critical frames.
According to some embodiments of the invention, the surface composition module is further attaching metadata tags which includes the information related to the beaming video frames.
According to some embodiments of the invention, the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
According to some embodiments of the invention, the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames.
According to some embodiments of the invention, the detector module identifies beginning of a set of critical frames and an end of the set of critical frames.
BRIEF DESCRIPTION OF THE DRAWINGS
The present invention will be more readily understood from the detailed description of embodiments thereof made in conjunction with the accompanying drawings of which:
FIG. 1A is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is separate to a UI framework, according to some embodiments of the invention;
FIG. 1B is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is part of a UI framework, according to some embodiments of the invention;
FIG. 2 is a flowchart illustrating a process of video encoding module, according to some embodiments of the invention;
FIG. 3 is a flowchart illustrating a process of frames analyzing module, according to some embodiments of the invention;
FIG. 4 is a flowchart illustrating a process of User Interface (UI) framework, according to some embodiments of the invention; and
FIG. 5 is a block diagram illustrating a process of Surface composition module, according to some embodiments of the invention.
 DETAILED DESCRIPTION
Before explaining at least one embodiment of the invention in details, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of the components set forth in the following description or illustrated in the drawings. The invention is applicable to other embodiments and/or may be practiced or carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein is for the purpose of description and should not be regarded as limiting.
For a better understanding of the invention, the usages of the following terms in the present disclosure are defined in a non-limiting manner:
The term “screen Graphical User Interface (GUI)” as used herein in this application is define as data and appearance of application screens such as a video screen or messaging application screen.
The term “key period” as used herein in this application is defined as a period in which there is a transition in content displaying pattern of an application which starts a set of critical frames, For example, when a user focuses in a specific area in a screen to perceive the details of the content that is appearing on the screen after an action of the user such as scrolling. For example, when the user is reading a text or watching an image or reviewing a webpage or increase in content changes which appear on the screen in a non limiting example, when the user is moving the mouse. The key period is as opposed to a period when scrolling a webpage, or while loading an application on a computerized mobile device. That period is quite a passive period in which the user is not focused on the screen, and not trying to perceive the details of the content that is displayed on the screen. In another example, a video movie may be comprised of parts with higher level of compression of data and other parts with a lower level of compression. Also, a key period may be a period when there is an increase in content changes in a specific frame.
The term “beaming” as used in this application is defined as the communication of screen content encoded data between a computerized mobile device and a target display device such as iPhone airplay mirroring.
The term “delivered data” as used in this application is defined as the delivered data before compression or before high bandwidth usage.
The term “intra-coded pictures (I-frame)” as used in this application is defined as a fully specified picture (like a conventional static image file) in process of video compression stream. There are two types of inter frames: Predicted picture (P-frames) and Bi-predicted picture (B-frames). P-frames and B-frames hold only part of the image information, so they need less space for storage than an I-frame, and thus improve video compression rates.
The term “video beaming” as used in this application is defined as the action of projecting content of an application that is running on a computerized mobile device on a remote screen by utilizing video format technology and video transmission protocols.
The present invention provides a system and method for improving the user's experience by increasing the image quality in key periods in which an output data of a video encoder is displayed in low quality. A user may beam content that is displayed on a screen of a computerized mobile device. The content may be beamed by a beaming application that is utilizing video technology.
The method according to the present invention, improves user's experience by increasing the quality of a selected frame at the key period. The frame is selected by identifying the last frame of an action which is known to generate a high frame rate which results in low quality video output. Examples for such actions, which generate a high frame rate, are scrolling, rotating, switching between application screens or animations. The identifying may be achieved by analyzing characteristics of the video frames in different techniques as will be further explained below.
According to some embodiments of the invention, an improvement in the quality of the frame during the key period is a result of a first set of low quality frames before the key period (during the action period) and a frame with a higher quality at the beginning of the key period. The higher quality frame is configured as an I-frame.
According to some embodiments of the present invention the user's experience is improved by adding an additional high quality frame (instead of increasing a quality of a frame), at the beginning of the key period. The added frame is identical to the last low quality frame, having less compressed rate and hence in higher quality. The added frame is configured as an I-frame.
According to some embodiments of the present invention, a detection process for identifying in real time, a critical frame in key periods for providing higher quality frame in this period. The identification can be implemented using two techniques. The two techniques of detecting the critical frames may exist at the same implementation, and complete the operation of each other.
According to some embodiments of the present invention, one technique is using metadata tags that maybe attached to frames based on information that is arriving from various Operating Systems (OS)s and User Interface (UI) components. Alternatively, the metadata tags may be separate from the frames and may be sent with frames ID. The metadata tags indicate for each frame if it's the critical frame in the timeline at the beginning of a key period. For example, attaching this metadata tags to a frame according to information from a scrolling UI component indicating that this rendered frame is the last frame of a key period in a non limiting example, a scroll of website action.
According to some embodiments of the present invention, second technique is implemented by analyzing the output bit rate of the encoder in real time for detecting beginning of a key period. The key period is characterized by a known pattern of bit rate. The known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames.
FIG. 1A is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module is separate to a UI framework, according to some embodiments of the invention. The computerized mobile device is comprised of: (i) surface composition module 500 (which is part of the OS of the computerized mobile device) that is enabling to integration of a surface of multiple applications into a single frame, to be displayed on a target display such as a television (TV) screen; (ii) a beaming module 600 to display on a target screen; (iii) a video encoding manager module 200 for managing and sending instructions related to the compression of the beamed video to a video encoder 700; (iv) a frame analyzing module 300 for analyzing the bit rate which is embedded in detector module 450; and (v) a UI framework code 400 (which is part of the computerized mobile device OS) for identifying critical frames.
FIG. 1B is a block diagram, illustrating the modules of a beaming application in a computerized mobile device, wherein a detector module 450 is part of a UI framework, according to some embodiments of the invention.
FIG. 2 is a flowchart illustrating at least part of the process of Video encoding module 200, according to some embodiments of the present invention. The process of Video encoding module 200 monitors video frames (stage 212) and receives indication of critical frames (stage 214). Next, according to received indication frames are selected for quality improvement (stage 216). Next, the qualities of the selected frames in the key period are improved (stage 218), providing improved experience to the user/viewer. The improvement may be achieved by changing the frames quality or adding new identical frames in better/higher quality.
FIG. 3 is a flowchart illustrating at least part of the process of the frames analyzing module 300 according to some embodiments of the present invention. According to this embodiment the process of the frames analyzing module 300 monitors the bit rate or the change of content (stage 312), for detecting a predefined pattern of bit rate period such as key period in which a high bit rate period with minimal length is followed by a low bit period or a period with no input frames or for identifying increase of change of content (stage 314). Meaning, identifying in real time a pattern change in displayed video, wherein the pattern change signifies transition from a rate of a predefined size of chunk of delivered data to a higher delivered data rate in content displaying pattern by the application which starts a set of critical frames. The detected frames within the key period are marked and reported to the encoding module (stage 316).
FIG. 4 is a flowchart schematically illustrating at least part of one of the UI framework processes, according to some embodiments of the present invention. According to this embodiment, the code identifies change which signifies transition in content displaying pattern is performed according to information that is received from a User interface (UI), such as end of scrolling of a webpage (stage 412) and creates metadata tags which include indication information of frames which are part of a key period (optionally) (stage 414). Next, the metadata tags are attached to the relevant frames (optionally) (stage 416). Optionally, sending the metadata tags separate from the relevant frames using frame IDs.
The tagged frames are enhanced by the encoding video module 300 for improving the quality.
According to some embodiments of the invention, the UI is of an Operating System (OS) or of the application that is running on the computerized mobile device.
FIG. 5 is a flowchart schematically illustrating at least part of the process of surface composition module according to some embodiments of the present invention. The modules receives surfaces (screen images) from various applications running on the computerized mobile device and system UI components with their metadata tags (stage 512) and compose the visible surfaces to a single frame which is composition of all visible application surfaces. When composing the single frame the module merges the metadata of the invalidated (dirty) areas (modified area of the application) (stage 514). Finally, the composed frame and the merged metadata tags (stage 516) are posted.
Optionally, the beamed content is different from the displayed content on the mobile device screen.
According to some embodiments of the invention, the surface composition module may be a detector module.
It is to be understood that the phraseology and terminology employed herein is not to be construed as limiting and are for descriptive purpose only.
The principles and uses of the teachings of the present invention may be better understood with reference to the accompanying description, figures and examples. It is to be understood that the details set forth herein do not construe a limitation to an application of the invention.
Furthermore, it is to be understood that the invention can be carried out or practiced in various ways and that the invention can be implemented in embodiments other than the ones outlined in the description above. It is to be understood that the terms “including”, “comprising”, “consisting” and grammatical variants thereof do not preclude the addition of one or more components, features, steps, or integers or groups thereof and that the terms are to be construed as specifying components, features, steps or integers.
If the specification or claims refer to “an additional” element, that does not preclude there being more than one of the additional element. It is to be understood that where the claims or specification refer to “a” or “an” element, such reference is not to be construed that there is only one of that element.
It is to be understood that where the specification states that a component, feature, structure, or characteristic “may”, “might”, “can” or “could” be included, that particular component, feature, structure, or characteristic is not required to be included. Where applicable, although state diagrams, flow diagrams or both may be used to describe embodiments, the invention is not limited to those diagrams or to the corresponding descriptions. For example, flow need not move through each illustrated box or state, or in exactly the same order as illustrated and described.
Methods of the present invention may be implemented by performing or completing manually, automatically, or a combination thereof, selected steps or tasks. The term “method” may refer to manners, means, techniques and procedures for accomplishing a given task including, but not limited to, those manners, means, techniques and procedures either known to, or readily developed from known manners, means, techniques and procedures by practitioners of the art to which the invention belongs.
The descriptions, examples, methods and materials presented in the claims and the specification are not to be construed as limiting but rather as illustrative only. Meanings of technical and scientific terms used herein are to be commonly understood as by one of ordinary skill in the art to which the invention belongs, unless otherwise defined. The present invention may be implemented in the testing or practice with methods and materials equivalent or similar to those described herein.
Any publications, including patents, patent applications and articles, referenced or mentioned in this specification are herein incorporated in their entirety into the specification, to the same extent as if each individual publication was specifically and individually indicated to be incorporated herein. In addition, citation or identification of any reference in the description of some embodiments of the invention shall not be construed as an admission that such reference is available as prior art to the present invention.
While the invention has been described with respect to a limited number of embodiments, these should not be construed as limitations on the scope of the invention, but rather as exemplifications of some of the preferred embodiments. Other possible variations, modifications, and applications are also within the scope of the invention. Accordingly, the scope of the invention should not be limited by what has thus far been described, but by the appended claims and their legal equivalent.

Claims (17)

What is claimed is:
1. A method for improving quality of video beaming of any content by a beaming application that is running on a computerized mobile device, said method comprising the steps of:
selecting content for beaming;
beaming selected content by utilizing a beaming application;
identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application which starts a set of critical frames;
performing quality improvement of the video beaming of the set of critical frames based on the identified pattern changes; and
identifying in real time a second pattern change in display video to a lower delivered data rate which ends the set of critical frames,
wherein the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames; and
wherein the identifying and the performing is running on at least one processing unit;
wherein the performing of quality improvement of the video beaming is activated by selecting frames of the beaming video in low quality for quality improvement and applying quality improvement on the selected frames.
2. The method of claim 1, wherein higher delivered data rate is compressed data.
3. The method of claim 1, wherein higher delivered data rate is high bandwidth usage.
4. The method of claim 1, wherein the identifying is achieved by analyzing characteristics of the video frames.
5. The method of claim 1, wherein the performing of quality improvement is achieved by generating an additional frame with a higher quality and adding the additional frame to the video beaming.
6. The method of claim 1, wherein the identifying of the change which signifies transition in content displaying pattern is performed according to information that is received from a User interface (UI) by attaching metadata tags which includes the information related to the beaming video frames.
7. The method of claim 1, wherein the UI is of an Operating System (OS).
8. The method of claim 1, wherein the UI is of the application that is running on the computerized mobile device.
9. The method of claim 1, wherein the identifying of the change which signifies pattern change in displayed video is performed according to information that is received from an Operation System (OS) or a User interface (UI) by sending metadata tags and identification (ID) of the critical frames.
10. The method of claim 1, wherein the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
11. A beaming application for improving quality of video beaming by a processing unit that is running on a computerized mobile device, said beaming application comprising:
a detector module for identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application;
a video encoding module for performing quality improvement of the beaming video after the identified a pattern change in beamed video; and
wherein the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames
wherein the detector module is part of a UI framework module for identifying critical frames based on information that is received from UI components; and a surface composition module for receiving information regarding critical frames from OS is part of the UI framework;
wherein the UI framework module is further creating metadata tags to the identified critical frames and attaching the metadata tags to the critical frames.
12. The beaming application of claim 11, wherein the detector module is comprising of a frame analyzing module for monitoring bit rate of frames of the beaming video and detecting a change which signifies transition in usage of the application which starts a set of critical frames.
13. The beaming application of claim 11, wherein the UI framework module is further creating metadata tags to the identified critical frames and identification (ID) of the critical frames and sends is separately from the critical frames.
14. The beaming application of claim 11, wherein the surface composition module is for composing the single frame by merging the metadata tags of modified areas of the application.
15. The beaming application of claim 11, wherein the identifying of the change which signifies transition in usage of the application is performed by analyzing an output bitrate of an encoder for detecting the change, wherein the change is characterized by a known pattern of bit rate.
16. The beaming application of claim 11, wherein the detector module identifies beginning of a set of critical frames and an end of the set of critical frames.
17. A beaming application for improving quality of video beaming by a processing unit that is running on a computerized mobile device, said beaming application comprising:
a detector module for identifying in real time a pattern change in beamed video, wherein the pattern change signifies transition from a previous rate of a predefined size of chunk of delivered data to a higher rate of currently delivered data in content displaying pattern by the application;
a video encoding module for performing quality improvement of the beaming video after the identified a pattern change in beamed video; and
wherein the known pattern includes a high bit rate period with minimal length followed by a low bit rate period or a period with no input frames
wherein the detector module is part of a UI framework module for identifying critical frames based on information that is received from UI components; and a surface composition module for receiving information regarding critical frames from OS is part of the UI framework;
wherein the UI framework module is further creating metadata tags to the identified critical frames and identification (ID) of the critical frames and sends is separately from the critical frames.
US13/762,960 2012-02-09 2013-02-08 Method and system of improving quality of video beaming Expired - Fee Related US9270916B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US13/762,960 US9270916B2 (en) 2012-02-09 2013-02-08 Method and system of improving quality of video beaming

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261596757P 2012-02-09 2012-02-09
US13/762,960 US9270916B2 (en) 2012-02-09 2013-02-08 Method and system of improving quality of video beaming

Publications (2)

Publication Number Publication Date
US20130208789A1 US20130208789A1 (en) 2013-08-15
US9270916B2 true US9270916B2 (en) 2016-02-23

Family

ID=48945522

Family Applications (1)

Application Number Title Priority Date Filing Date
US13/762,960 Expired - Fee Related US9270916B2 (en) 2012-02-09 2013-02-08 Method and system of improving quality of video beaming

Country Status (1)

Country Link
US (1) US9270916B2 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10484288B2 (en) * 2016-02-23 2019-11-19 Level 3 Communications, Llc Network flow control

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102088384B1 (en) * 2013-09-04 2020-05-28 삼성전자주식회사 Method for controlling a display apparatus, sink apparatus thereof, mirroring system thereof
US10410398B2 (en) * 2015-02-20 2019-09-10 Qualcomm Incorporated Systems and methods for reducing memory bandwidth using low quality tiles

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5956681A (en) * 1996-12-27 1999-09-21 Casio Computer Co., Ltd. Apparatus for generating text data on the basis of speech data input from terminal
US20020011980A1 (en) * 2000-06-09 2002-01-31 Tetsuya Aoyama Display device
US20100121977A1 (en) * 2008-11-10 2010-05-13 Nokia Corporation Predictive Bit-Rate Modification of Content Delivery in a Wireless Network
US20100295866A1 (en) * 2009-05-21 2010-11-25 Spatial Photonics, Inc. Reducing image artifacts in a color sequential display system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5956681A (en) * 1996-12-27 1999-09-21 Casio Computer Co., Ltd. Apparatus for generating text data on the basis of speech data input from terminal
US20020011980A1 (en) * 2000-06-09 2002-01-31 Tetsuya Aoyama Display device
US20100121977A1 (en) * 2008-11-10 2010-05-13 Nokia Corporation Predictive Bit-Rate Modification of Content Delivery in a Wireless Network
US20100295866A1 (en) * 2009-05-21 2010-11-25 Spatial Photonics, Inc. Reducing image artifacts in a color sequential display system

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10484288B2 (en) * 2016-02-23 2019-11-19 Level 3 Communications, Llc Network flow control
US11063875B2 (en) * 2016-02-23 2021-07-13 Level 3 Communications, Llc Network flow control
US11601372B2 (en) * 2016-02-23 2023-03-07 Level 3 Communications, Llc Network flow control
US20230208768A1 (en) * 2016-02-23 2023-06-29 Level 3 Communications, Llc Network flow control

Also Published As

Publication number Publication date
US20130208789A1 (en) 2013-08-15

Similar Documents

Publication Publication Date Title
US10003842B2 (en) Auto video preview within a digital magazine
US9064393B2 (en) Video playback method, terminal, and system
US20190245907A1 (en) Video Production System With DVE Feature
US20240388745A1 (en) Optimized Content Delivery
US9426476B2 (en) Video stream
SG188630A1 (en) Video bit stream transmission system
CA2828205A1 (en) Facilitating placeshifting using matrix code
US20160065625A1 (en) Notification of Change in Online Conferencing
US20120304077A1 (en) Accessing window pixel data for application sharing
US10021433B1 (en) Video-production system with social-media features
US10219036B2 (en) Dynamically adjusting video merchandising to reflect user preferences
CN113965777A (en) Method and system for combining digital video content
CN105165009A (en) System, apparatus and method for sharing a screen having multiple visual components
US9053526B2 (en) Method and apparatus for encoding cloud display screen by using application programming interface information
US9270916B2 (en) Method and system of improving quality of video beaming
US7403566B2 (en) System, computer program product, and method for transmitting compressed screen images from one computer to another or many computers
US20110271307A1 (en) Video data stream evaluation systems and methods
US11323701B2 (en) Systems and methods for group of pictures encoding
US20190141366A1 (en) System and method for insertion of an asset into a source dynamic media
US20120254933A1 (en) Network video server and video control method thereof
US20240338165A1 (en) Client-Side Composite Video Stream Processing
US12184930B1 (en) In-frame video markers
WO2016032865A1 (en) Thumbnail generation
CN112911299B (en) A video code rate control method, device, electronic equipment and storage medium
JP2012204878A (en) Video data stream evaluation method and system

Legal Events

Date Code Title Description
AS Assignment

Owner name: SCREENOVATE TECHNOLOGIES LTD., ISRAEL

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SHAPIRA, MATAN;REEL/FRAME:036844/0382

Effective date: 20151008

ZAAA Notice of allowance and fees due

Free format text: ORIGINAL CODE: NOA

ZAAB Notice of allowance mailed

Free format text: ORIGINAL CODE: MN/=.

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YR, SMALL ENTITY (ORIGINAL EVENT CODE: M2551); ENTITY STATUS OF PATENT OWNER: SMALL ENTITY

Year of fee payment: 4

AS Assignment

Owner name: INTEL CORPORATION, CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SCREENOVATE TECHNOLOGIES LTD.;REEL/FRAME:059478/0777

Effective date: 20220321

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20240223