CN110139107A - System and method for handling video data - Google Patents

System and method for handling video data Download PDF

Info

Publication number
CN110139107A
CN110139107A CN201910341517.XA CN201910341517A CN110139107A CN 110139107 A CN110139107 A CN 110139107A CN 201910341517 A CN201910341517 A CN 201910341517A CN 110139107 A CN110139107 A CN 110139107A
Authority
CN
China
Prior art keywords
video frame
frame
resolution video
resolution
video
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201910341517.XA
Other languages
Chinese (zh)
Inventor
瓦德·基思·万
达雷恩·杜安·诺伊曼
贾森·威廉·赫里克
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Avago Technologies International Sales Pte Ltd
Broadcom Corp
Original Assignee
Avago Technologies General IP Singapore Pte Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US13/929,739 external-priority patent/US8982277B2/en
Application filed by Avago Technologies General IP Singapore Pte Ltd filed Critical Avago Technologies General IP Singapore Pte Ltd
Publication of CN110139107A publication Critical patent/CN110139107A/en
Pending legal-status Critical Current

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/85Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
    • H04N19/88Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/33Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/597Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The invention discloses the system and method for handling video data.System and method processing has the video flowing of first resolution video frame, which includes multiple second resolution video frames.

Description

System and method for handling video data
The relevant information of divisional application
This case is divisional application.The female case of the division be the applying date be on 04 24th, 2014, application No. is 201410168639.0, the invention patent application case of entitled " system and method for handling video data ".
Cross reference to related applications
The application is non-provisional application and requires entitled " the System and Method submitted on April 25th, 2013 U.S. Provisional Patent Application the 61/815th, 860 equity of for Processing Video Data ", by quoting it Full content is incorporated herein.
Technical field
Present disclosure relates generally to the system and method for handling video data, and more particularly relates to handle The system and method for video data comprising the low resolution video data being embedded in high-resolution video data.
Background technique
Modern video constantly raises the boundary of video resolution.However, but cannot be always that modern video mentions For having the video with their the maximum resolution ratio for supporting that resolution ratio is equivalent.It may repeatedly provide to system and be supported lower than maximum The video of resolution ratio.In the case where having the memory and hard disk of the video format for supporting high-resolution, when support comes from When the low resolution video flowing of traditional standard, system may cannot be effectively utilized.
Summary of the invention
According to an aspect of the invention, there is provided a kind of method for video processing, which comprises receive packet First video flowing of the video frame containing first resolution, at least one of described first resolution video frame are defined to include more A second resolution video frame;And the second resolution video frame based on the first resolution video frame generates second Video flowing.
Wherein, the first resolution video frame has the first frame rate and multiple second resolution video frame tools There is the second frame rate.
Wherein, second frame rate is higher than first frame rate.
Wherein, the first resolution and the second resolution are bit depth.
Wherein, multiple second resolution video frames include: the first frame of the most significant bit comprising institute's bit depth With the second frame of the least significant bit comprising institute's bit depth.
Wherein, the first resolution has the resolution ratio higher than the second resolution.
Further, which comprises the first resolution frame is received in Video Codec.
Further, which comprises multiple second resolutions are cut from the first resolution video frame Video frame.
Further, which comprises second resolution video flowing is generated from the second resolution video frame.
According to the second aspect of the invention, a kind of system for encoded video data is provided, the system comprises: it deposits Storage unit;Frame processing unit, is operatively coupled to the storage unit, the frame processing unit be configured as retrieving it is multiple compared with Low resolution video frame, by lower resolution video frame combination in high-resolution video frame, and will be described higher Resolution video frame is stored in the storage unit;Encoder is operatively coupled to the storage unit, the encoder Including being configured as at least one module by the high-resolution video frame compression in coding stream.
Wherein, the high-resolution video frame definition frame rate lower than the lower resolution video frame.
Wherein, each framing bit of the multiple lower resolution video frame is in the region of the high-resolution video frame.
Wherein, each frame of the multiple lower resolution video frame includes the component frame of 3 D video frame.
Wherein, at least one frame of the multiple lower resolution video frame includes two-dimensional video content.
Wherein, at least one corresponding frame of the multiple lower resolution video frame includes three dimensional depth layer content.
According to another aspect of the invention, it provides a kind of for handling including being embedded in high definition video data The device of the video data of low resolution video data, described device include: storage unit;Processor is operatively coupled to To the storage unit, the processor includes being configured as at least one module below: high-resolution video frame is delayed There are in the storage unit;Multiple lower resolution video frames are cut from the high-resolution video frame;Described in rendering Multiple lower resolution video frames are to generate low resolution video flowing.
Wherein, the high-resolution video frame includes four lower resolution video frames.
Wherein, the multiple lower resolution video frame includes multiple camera angles of video broadcasting.
Wherein, each frame of multiple low definition video frames includes different program.
Wherein, the multiple low definition video frame includes at least one of multiple video flowings and the multiple video flowing Video flowing includes the specific aim market content for specific user.
Detailed description of the invention
Fig. 1 is the diagram of the video frame with the first resolution comprising multiple video frames with second resolution;
Fig. 2 is the frame for the system for the video frame for encoding the first resolution of multiple video frames comprising second resolution Figure;
Fig. 3 is the system for the video frame for decoding the first resolution of multiple video frames comprising second resolution;
Fig. 4 is the block diagram for showing multiple operation modes for Fig. 3 system introduced;
Fig. 5 is the process for the method for the video flowing for decoding the first resolution of the video content comprising second resolution Figure;
Fig. 6 is showing for the video frame of the first resolution of multiple video frames (comprising three-dimensional content) comprising second resolution Figure;
Fig. 7 is the block diagram for decoding the system of the video flowing comprising 3 d video content;
Fig. 8 is the flow chart for decoding the method for the video flowing comprising 3 d video content;
Fig. 9 is the diagram of the video frame of the first resolution comprising multiple video frames with increased frame rate;And
It includes first resolution with multiple video frames of increased frame rate and second resolution that Figure 10, which is for decoding, The system of the video frame of rate;And
Figure 11 is the diagram according to the video frame with the first bit depth of the introduction of present disclosure.
Specific embodiment
With reference to Fig. 1, the diagram of video frame is provided.Video frame can have first resolution and include to differentiate with second Multiple video frames of rate.In this embodiment, the video frame with first resolution can define high-resolution video frame 102 and lower resolution video frame 104 can be defined with the video frame of second resolution.Lower resolution video frame 104 It is combined the high-resolution frame 106 that can form combination.High-resolution video frame 102 may include the picture of 3840x 2160 Plain resolution ratio, commonly known as 4k x 2k resolution ratio or Ultra HD resolution ratio.In some embodiments, high-resolution The frame rate of video frame can be such as 120Hz.It should be appreciated, however, that frame rate can be any other video frame rate (example Such as, 60Hz, 240Hz, 480Hz etc.).Combined high-resolution frame 106 also may include the total resolution of 3840x 2160.Group The high-resolution frame 106 of conjunction may further include multiple lower resolution video frames, and such as, 1080p resolution ratio is (that is, every A pixel resolution with 1920x 1080).
In this example, combined high-resolution frame 106 may include the multiple portions comprising corresponding video content Or section.In one or more embodiments, correspond to first frame quadrant 108, the second frame quadrant 110, third frame quadrant 112, 4th frame quadrant 114.It can be in each of frame quadrant 108,110,112,114 four different programs (program) of middle display. In addition, each of frame quadrant 108,110,112,114 can be 1080p resolution ratio.
Although describing the resolution ratio of 1080p and 4k x 2k in this embodiment, it can be applied according to present disclosure His video resolution, for example, 720x 480,1280x 720,8k x 4k and other standards and off-gauge video differentiate Rate.In some embodiments, multiple frames may include any number in the multiple regions of combined high-resolution frame The frame of amount.For example, multiple frames may include two in the region of the corresponding number in the combined high-resolution frame extremely Ten frames.Multiple frames can further change the resolution ratio in the high-resolution frame individually combined.For example, the relatively high score of combination Resolution frame may include three 1080p frames, 720 frame of 1280x and 360 frame of 640x.
As used herein term program and video content may refer to the video flowing, video content, phase of diversified forms The video of machine angle, the video, computer program, 3 d video content and the other forms that are captured by software program.At this In embodiment, the frame from four independent video flowings can be embedded in combined high-resolution frame 106.For example, Four 1080p video flowings can be transmitted simultaneously in single 4k x 2k video flowing.Term high-resolution as discussed in this article and compared with Low resolution be used to compare the relative resolution of first resolution and second resolution, and be not construed as the limitation disclosure Content.In general, term resolution ratio can refer to the pixel resolution of video frame.However, at least one embodiment, point Resolution can refer to pixel depth, bit depth or color depth as described below.
With reference to Fig. 2, the system for encoding the video frame with first resolution is shown according to the introduction of present disclosure 202 block diagram.Video frame with first resolution may further include multiple video frames with second resolution.System It usually may include: Frame Handler 204, memory 206, video frame compiler 208 and encoder 210.Through present disclosure The memory in the system 202 and other systems of description may include various forms of memories, for example, arbitrary access is deposited Reservoir (RAM), dynamic ram (DRAM), synchronous dram (SDRAM) and the other forms for being configured as storage digital information Memory.
In this specific embodiment, Frame Handler 204 can be configured to retrieve single lower resolution video frame simultaneously And each video frame is buffered in memory 206 (such as DRAM).Memory 206 can have enough capacity to correspond to store The video frame of multiple cachings of at least one high-resolution video frame.In this embodiment, lower resolution video frame 214a, 214b, 214c and 214d can be 1080p video frame and high-resolution video frame 212 can be 4k x 2k video Frame.Video frame compiler 208 can retrieve single lower resolution video frame and combine the frame to be formed individually compared with high score Resolution (for example, 4k x 2k) video frame.Once high-resolution video frame is compiled, encoder can be retrieved each compared with high score Resolution video frame.Then, encoder can encode each video frame to create high resolution output video flowing in a compressed format.It is defeated Video by decoder processes and compresses a part as video feed (video feed) later out.
It nows refer to figures 1 and 2, compiler can be with various arrangements and sequential polymerization lower resolution frame 104.Lower resolution The particular arrangement of rate video frame and other features of each frame are collected by compiler and are embedded in metadata by encoder.First number According to usually may include recognition combination high-resolution frame 106 in lower resolution frame 104 presence and arrangement information. Metadata may further include other characteristic informations about high and lower resolution frame, for example, resolution ratio, compressed data, Frame rate, color depth and any information that lower resolution video frame and high-resolution video frame can be connected. Terminology metadata can refer to various forms of data, such as header information, data token, data packet and can connect in video streaming Other identification informations received.
Referring now to Figure 3, being shown according to the introduction of present disclosure for decoding the video frame with first resolution System.Video frame with first resolution may further include multiple video frames with second resolution.Decoding system 302 may include: usually decoder 304, storage unit 306 (for example, in the form of DRAM), video sealer 308, Video Composition Device 310.In response to the reception of input video stream, system 302 can be by one or more video frame buffers of high-definition video stream In storage unit 306.The metadata of frame in the accessible storage unit 306 of decoder 304 inputs high-resolution to determine The feature of video flowing.Then, decoder 304 can explain one or more high-resolutions corresponding to high-definition video stream The metadata of video frame is to determine the identification information about each high-resolution video frame.Identification information can indicate that input is high Resolution video stream includes conventional video data, for example, the single frame of the video data for single program, or each Multiple lower resolution video frames are embedded in high-resolution video frame.Then, entirely decoding system can be using input high-resolution The feature of rate video flowing is to handle multiple lower resolution video frames for showing.
Because decoder 304 decompresses each high-resolution video from video flowing, it is possible to will be each compared with high-resolution Rate video frame is stored in storage unit 306.Then, video feed device 312 can retrieve one or more high-resolution views Frequency frame and each high-resolution video frame is provided to video sealer 308.Video sealer 308 can be according to first number Each lower resolution video frame is cut and arranged according to the instruction of, controller or user's input.It is decompressed, as 120Hz's The received multiple video frames of 4k x 2k video frame can correspond to four 1080p video flowings (for example, 1080p program) of 120Hz It is corresponding.
Then, the video frame of the scaling from video sealer 308 can be captured by frame-grab unit 316.Frame-grab unit 316 can capture the scaling of each lower resolution video frame and position for showing with single, high-resolution video frame.Through Combined high-resolution video frame including multiple lower resolution video frames can be provided to video synthesizer by frame-grab 310.Then, video synthesizer 310 can assemble each frame with final video format to increase additional rendering figure and increasing By force.Then, the Final Format slave synthesizer of combined high-resolution video frame can be exported to display unit.In order to provide Flexible viewing experience, system 302 can be further provided for for being configured as changing user's input of watching mode, such as Fig. 4 It is shown.
With reference to Fig. 4, showing diagram according to the introduction of present disclosure can be used as the video exported via decoding system 302 The block diagram of mode.In first mode 402, video sealer can retrieve four lower resolution video frames.It can be contracted by video It puts device 404 and cuts and scale each lower resolution video frame.Video sealer 404 allows lower resolution video frame with single High-resolution frame is shown.Then, the frame-grab of single high-resolution frame can be completed by frame-grab unit 406.It can be by synthesizing Device 405 and output module 407 handle each program to output this to display screen 408.It can show that low resolution regards simultaneously Frequency frame, as shown in program 1, program 2, program 3 and program 4 or shown in position generate any variation.In the first mode 402 In, four independent low resolution video screens are shown simultaneously on high resolution display 408.In this example, each Individual lower resolution frame may include 1080p frame provided in 4kx2k higher resolution frame.
In second mode 410, user may prefer to watch single program on full screen, such as program 1.User can The mode is activated with request, so that the full screen display program 1 on display 412.In this case, video sealer 414 can To cut and amplify each frame of (upscale) program 1.In addition, each frame that frame-grab unit 416 can capture program 1 is used It is shown in full screen.Video frame can be handled exporting its full screen in display 412 by synthesizer 415 and output module 417 On.Then, user can watch program 1 in the form of traditional viewing.
It can be similar to the support the third mode of second mode 410 corresponding to the single frame of program 2 by retrieving and amplifying 418.Then, video sealer 422 can amplify the single video frame corresponding to program 2.In addition, frame-grab unit 424 can be caught Obtain the high-resolution frame for program 2.Video frame can be handled by synthesizer 425 and output module 427.Then, program 2 can be with It is shown on display 420 with the format of the high-resolution of amplification.Similar to mode 410 and 418, other modes can also be answered For third program and fourth program.Other embodiments can be further directed to being stored in single high-resolution video frame Multiple lower resolution video frames provide, and the subframe or correspondence of four frames shown in further can merging 408 In the part fractional part for any resolution ratio or multiple resolution ratio that can be arranged or transmit in single high-resolution video frame. E.g., including multiple frames of the resolution ratio of 1080p, 720p and 480p (DVD resolution ratio) can be combined into single, biggish 1080p, 4k x 2k or 8k x 4k video frame.Can also using with consistent other standards of present disclosure and off-gauge Video resolution.
Term program in present disclosure is generally used to illustrate extensive video content.In some embodiments, compared with Low-resolution frames may include the multiple camera angles for the broadcast that can correspond to Live Audio or record before.For example, competitive sports Multiple camera angles can be shown as program 1,2,3 and 4.In another embodiment, multiple independent video contents It can be shown as program 1,2,3 and 4.It is further for other examples of the embodiment of program can include: wide including business Announcement, Video Roundup, weather forecast, score, credit render video in perhaps any other video content at least one program. In the embodiment for including multiple camera views, user can be switched between camera view under screen mode toggle with Request System (see 412 and 420) or camera angle 408 is shown simultaneously.
In yet another embodiment, at least one program may include specific aim video flowing, which can Specific aim marketing content including being directed to niche audience.In this embodiment, as shown in 408, can for one or Multiple programs provide content simultaneously.In addition, as shown in frame 412 and 420, in response to metadata received in input video stream or The request that user's full screen is shown, can be with switch contents.Other examples of program may include computer software video content and Other computer application related contents.In these embodiments, display priority can pass through member number received in video flowing According to identification, can also be identified by user preference.Metadata may include that instruction will before user is likely to return to selected program The rule in period required for the advertisement of display.
The flow chart of the method for decoded video streams is provided with reference to Fig. 5,502.It can encode and pass with first resolution Defeated video flowing.However, video flowing may include the video content with second resolution according to the introduction of present disclosure.Fig. 3 System 302 can further apply the high-resolution to show in entire video display to show video content.Response In the content of input video stream, system 302 shows multiple with being also an option that property in each single high-resolution video frame Lower resolution video frame.Therefore, system can receive encoded video stream (504).In response to the reception of input video stream, input The video frame of video flowing can be buffered in memory (such as DRAM) (506).Video frame (508) can be retrieved by decoder.Solution Code device can be configured as the metadata (510) dismantled corresponding to each video frame.Based on metadata, system can identify input Whether video flowing includes insertion, lower resolution video frame (512).
Although the metadata of this paper refers to the identification information of one or more frames corresponding to input video stream, first number According to may also comprise for example comprising header file, mark various forms or with or video flowing receive other identification letter Breath.Metadata can also by interpreter, internal or external controller, additional processor or be configured as identification input video Other circuit parallels of the characteristic of stream are explained.Then, characteristic can be applied to decode and show input video stream by decoder. In this embodiment, each frame of input video stream can be appointed as including with convention video (for example, entire by metadata The single video flowing shown on display) the similar single frame aimed to show that.
If input video stream includes single frame, decodes single high-resolution video frame (514) and store it in In memory.Then, high-resolution video (518) are retrieved from memory.Scaling high-resolution video as needed (520).Then, high-resolution frame (524) are captured.In addition, for example by synthesizer then combined arrangement (526).It then, will be compared with High-resolution video is provided to output video feed (528).
Metadata single frame can also be appointed as comprising be embedded in a part of high-resolution video frame (for example, In different quadrants) multiple lower resolution video frames.If it is determined that having received the video including multiple lower resolution frames Frame, then decoding includes the high-resolution video frame (536) of lower resolution video frame and stores it in memory.Method It can continue to retrieve each lower resolution video frame (538) from memory.Then, scale and cut each high-resolution The lower resolution frame (542) of frame.Then, by low resolution frame-grab to final video frame (544).It is then used to display The capture of each video frame can be increased additional rendering content (546) by synthesizer.Synthesizer can will be embedded in individually compared with Multiple low resolution videos in high-resolution video are exported to display (528).
Referring now to Figure 6, providing the diagram with the video frame of first resolution.First resolution video frame can have First resolution format.In addition, the first resolution video frame with the first video format may include second resolution format Multiple video frames.It may include three-dimensional content according to multiple video frames of the introduction of present disclosure.In this embodiment, Single high-resolution video frame 602 can be implemented to show the multiple low resolutions that can be combined to create three-dimensional content Frame.Each lower resolution video frame may include the corresponding video counts corresponding to single low resolution 3 D video frame According to.
Lower resolution video frame may include video layer 604.Video layer 604 may further include two two dimensions (2D) Video frame component 606.In addition, two video layers 604 may include three-dimensional (3D) enhancement-layer video component 608 (enhancement layer).Such as It is shown, video layer 604 may include can be combined to create single 3D video frame two 2D video frame components 606 and two three Dimension enhancing layer assembly 608.More specifically, 2D video frame 606 may include left stereo-picture 610 and right stereo-picture 612.3D increases Strong layer 608 may include left enhancement layer depth map 614 and right enhancement layer depth map 616.Each low resolution layer can be combined 604 to form the single lower resolution video frame including 3D depth content.In an example, low resolution can be 1080p and high-resolution can be 4kx2k;However, it is possible to apply other resolution ratio according to the introduction of present disclosure.
Referring now to Figure 7, being shown according to the introduction of present disclosure for decoding the video flowing including 3D video content The block diagram of system.Similar with the system of Fig. 3, system 702 can receive input high-definition video stream in decoder 704.Solution Code device 704 can be cached with decoded input high-definition video stream and in memory 706 from the every of high-definition video stream Multiple video frames of a frame.The metadata corresponding to multiple video frames can be explained to identify the characteristic or feature of video flowing.Such as Fruit determines that input video stream includes the video layer corresponding to 3D content, and video feed device 708 can be retrieved including video layer 710 High-resolution frame.Video sealer 711 can cut, rearrange and scale lower from each of video feed device 708 Resolution video layer.Then, 3D rendering engine 712 can capture single 3D from combined low resolution video layer 710 and close Framing.Video feed device 708 can be using 3D synthetic frame as output 3D video flowing outflow.Video feed device 708 can be by video The frame rate that the speed control of stream is identified in the metadata.It can be by 3D video flowing and rendering graphical set in synthesizer 716 It shares in display 718.
In this embodiment, provided in single high-resolution video frame four video layers (710a, 710b, 710c, 710d).Four video layers can be combined to generate single, low resolution 3D video frame.When continuously generating, low resolution 3D video frame can form low resolution 3D video flowing.Method disclosed herein and system can be regarded in single high-resolution Various low resolution video contents are combined in frequency frame.The example of low resolution content may include various video contents, section Mesh, enhancement layer, interactive marketing information and multiple camera views.
Referring now to Figure 8, being shown according to the introduction of present disclosure for decoding the video flowing including 3D video content The flow chart (802) of method.This method can be initialized (804) by receiving coding high-definition video stream.The view Each frame of frequency stream can be buffered in memory, such as DRAM (806).Once at least one high-resolution video frame is In buffer memory, it can be retrieved from memory high-resolution video frame (808).It is then possible to dismantle metadata to know The characteristic (810) of not received video frame.From metadata, decoder can identify that high-resolution video frame includes 3D video Frame.It is then possible to decode high-resolution video frame and store it in memory (812).After the decoding, video is presented Send device that can retrieve from memory video frame (814).Then, video sealer can be with scaling video frame (816).Then, may be used With in 3D rendering engine composite video frame to create single low resolution 3D video frame (818).Then, it such as can close Middle combination 3D video frame of growing up to be a useful person and additional rendering figure (820).It can be presented 3D video frame as low resolution 3D video Send output to display (822).
With reference to Fig. 9, the diagram of video frame is provided according to the introduction of present disclosure.Video frame have first resolution and It can be set in the first frame rate.The video frame of first resolution may include have second resolution and the second frame rate more A video frame.In one embodiment, high-resolution video frame 902 has the multiple lower resolution frames being embedded within. High-resolution video frame 902 can have the resolution ratio of 4k x 2k and the frame rate of 60Hz.Multiple lower resolution video frames 904 may include four continuous 1080p frames of same video program, and each frame is 240Hz frame rate.Each low resolution The frame rate that the frame rate of frame may compare higher resolution frame is three times greater, however resolution ratio may be three times small.
In this embodiment, single program (being denoted as program 1) can have four continuous frames, first frame 906, second Frame 908, third frame 910 and the 4th frame 912.By the way that four lower resolution frames 904 are stored in high-resolution frame 902, It is four times of frame rate of high-definition video stream that video flowing including high-resolution video frame, which can be configured as transmitting to have, Low resolution video feed.In some embodiments, high-resolution video frame 902 can be the resolution ratio of 4kx2k It can be the resolution ratio of 1080p with lower resolution video frame 904.In other embodiments, each high-resolution frame and Lower resolution frame can respectively include different high and low frame resolution ratio.Another example of high-resolution frame can be tool There is 7,680 x, 4,320 frame of pixels (8k) for 16 1080p frames being embedded in 8k frame.Another example may include insertion 4k x 2k frame in 8k frame.
Lower resolution frame 904 in this embodiment be arranged with the first frame 906 of left upper quadrant, right upper quadrant 4th frame 912 of two frames 908, the third frame 910 of left lower quadrant and right lower quadrant.Further indicated by the arrow of display frame sequential The order.In some embodiments, each low resolution can be stored in the different quadrants of high-resolution frame 902 Frame.For example, first frame 906 can in left upper quadrant, the second frame 908 can in left lower quadrant, third frame 910 can upper right as Limit and the 4th frame 912 can be in right lower quadrants.Another embodiment may include first frame 906 in left upper quadrant, the second frame 908 in right upper quadrant, and third frame 910 is in left lower quadrant and the 4th frame 912 in right lower quadrant.Discuss about the embodiment The arrangement of lower resolution frame 904 is generally applicable to any embodiment discussed in the disclosure.
Referring now to Figure 10, showing the system 1002 for decoding the video frame with first resolution.With first point The video frame of resolution may include having multiple video frames of second resolution and increased frame rate.System 1002 usually can be with It include: decoder 1004, storage unit 1006, video sealer 1008 and synthesizer 1010.With the other embodiments of discussion Similar, in response to input video stream, decoder can be activated.Then, decoder 1004 can be by input video stream at least One frame buffer is in memory 1006 (such as DRAM).In this embodiment, input video stream may include multiple higher Resolution video frame, each high-resolution video frame include four lower resolution video frames.
Using at least one the high-resolution video frame cached in memory 1006, decoder can dismantle be used for it is higher The header information of resolution video frame and the characteristic for determining the lower resolution video frame being embedded in.As referred to Fig. 9 discussion, four A lower resolution video frame may include the 1080p frame of four 240Hz.It can be identified and be decoded each lower by decoder 1004 Resolution video frame.Then, can be retrieved from memory 1006 by video feed device 1012 lower resolution video frame and by its Supplied to video sealer 1008.In video sealer 1008, the single frame of four of single high-resolution frame can be cut out Cut and be scaled target display resolution.Then, frame-grab unit 1014 can capture single frame and generate increased frame speed The low resolution video flowing of rate.Then, synthesizer 1010 can increase additional video frame content and export video flowing To display 1018.
Once video sealer 1008 cuts and scales the one or four frame, next high-resolution video frame can be retrieved Four then frame.When handling input video stream, which can continue.System 1002 can be followed for from setting The instruction of the sequence of lower resolution frame in each high-resolution frame of metadata in video streaming.From display 1018 outputting video streams may include continuously display it is lower to provide each of exercise data of enhancing in low resolution Resolution video frame.Similar with other disclosed embodiments, the frame rate of low resolution and high-resolution video frame can With variation.Also some examples of the combination of adaptable frame rate may include 30Hz and 120Hz, 50Hz and 100Hz, 120Hz and 120Hz and 480Hz etc..
With reference to Figure 11, the diagram of the video frame 1102 with the first bit depth of the introduction according to present disclosure is shown. Bit depth can indicate the quantity for being used to indicate the position of the color of single pixel and can define color depth.In the embodiment party In formula, video frame 1102 may include the first color depth and can have high-resolution (for example, 4kx2k).Video frame 1102 may further include multiple video frames 1104 with the second color depth.Multiple video frames 1104 may include four Low resolution (for example, 1080p) video frame.In some embodiments, the first color depth can be defined than the second color The low color depth accuracy of depth.In this example, the first color depth can be 12 and the second color depth can be with It is 24.Color depth can define the quantity for being used to indicate the position of the color of each pixel of each video frame.
Although discussing color depth in this embodiment, bit depth can define other frames and pixel characteristic.Locating depth Degree can refer to the accuracy of pixel components data and pixel color component.In some instances, bit depth can refer to the bright of pixel Degree and coloration.It can be by may include 4: 2: 0,4: 2: 2,4: 2: 1,4: 1: 1,4: 1: 0,4: 4: 4 and other increment this programme Multiple increment this method transmit pixel components data.In some embodiments, bit depth can refer to can further define it is attached The image pixel intensities of the pixel data added.It includes that frame and pixel are special that various disclosed embodiments, which can further be combined to define, Any combination of multiple portions video frame of property.
Multiple video frames 1104 can further define the first part of the most significant bit including the second color depth of frame The second part video frame 1108 of video frame 1106 and the least significant bit including the second color depth.In this example, first Partial video frame 1106 may include 12 most significant bits of the color depth of each pixel of the first video frame.Second Point video frame 1108 may include 12 least significant bits of the color depth of each pixel of the second video frame.It can combine First part's video frame 1106 and second part video frame 1108 are to form first video frame 1110 with the second color depth.
Multiple video frames 1104 can further define the Part III view of the most significant bit including the second color depth The Part IV video frame 1114 of frequency frame 1112 and the least significant bit including the second color depth.With first part's video frame 1106 is similar, and Part III video frame 1114 may include 12 highests of the color depth of each pixel of the first video frame Significance bit.In addition, second part video frame 1108 can be with the 12 of the color depth for each pixel for including the 4th video frame The Part III video frame of a least significant bit is similar.Part III video frame 1112 and Part IV video frame can also be combined 1114 have second video of the second color depth (for example, twice of 24 accuracy or the first color depth) to be formed Frame 1116.
When the first video frame 1110 and the second video frame are all the combinations using its corresponding partial video frame, the first view Frequency frame 1110 and the second video frame 1116 can further define the sequence of lower resolution video frame 1118.Low resolution view The sequence of frequency frame 1118 can correspond to the successive frame of the low pixel resolution video stream with the second color depth.Pass through combination Partial video frame 1106 to 1108 and partial video frame 1110 to 1112 are to form the low resolution with the second color depth The sequence of video frame 1118, the sequence of lower resolution video frame can with spread in these similar systems disclosed herein It is defeated.
Although the first color depth and the second color depth in the example are 12 and 24 respectively, it is equally applicable Other combinations of color depth.Other examples of color depth combination may include 8 and 16,6 and 12 and 4 With 8 color depths.Video frame 1102 with the first color depth and multiple video frames 1104 with the second color depth Also it can change pixel resolution.In one embodiment, the video frame with the first color depth can have 8k resolution Rate and multiple video frames with second resolution can have 1080p resolution ratio.In this embodiment, there is second point Multiple video frames of resolution may include eight partial video frames.Eight partial video frames can be combined to be formed and there is the second face Four lower resolution video frames of color depth.It can also be according to other groups of present disclosure application color depth and resolution ratio It closes.
Similar with other processing of present disclosure introduction, the low resolution video flowing with the second color depth can be with It is transmitted in the high-definition video stream with the first color depth.It, can be with when receiving each frame of high-definition video stream Each frame is cached in the memory unit.Then, decoder can decompress the frame of high-resolution stream.It is then possible to explain adjoint The metadata of the frame of high-definition video stream includes multiple portions with low resolution with the frame for identifying high-definition video stream Divide video frame.Can cut in scaler and combine the partial video frame with low resolution has the second color to be formed Multiple lower resolution video frames of depth.It is then possible to by each lower resolution video frame with the second color depth Capture is into frame-grab unit and exports for showing.
It should be noted that various modules described herein and/or circuit (for example, coding module and/or circuit, decoder module and/ Or the rate adaptation module and/or circuit etc. of circuit, encoder) it can be single processing equipment or multiple processing equipment.This Kind processing equipment can be microprocessor, microcontroller, digital signal processor, microcomputer, central processing unit, scene Programmable gate array, programmable logic device, state machine, logic circuit, analog circuit, digital circuit, and/or referred to based on operation Enable any equipment for carrying out operation signal (analog and/or digital).Operational order is storable in memory.Memory can be list A storage equipment or multiple storage equipment.This storage equipment can be read-only memory (ROM), random access memory (RAM), volatile memory, nonvolatile memory, static memory, dynamic memory, flash memory, and/or storage Any equipment of digital information.Note that when processing module is real by state machine, analog circuit, digital circuit and/or logic circuit When applying mode one or more function, the memory of storage corresponding operating instruction can be embedded into circuit, which includes State machine, analog circuit, digital circuit and/or logic circuit.In this embodiment, memory stores, and is couple to that Processing module execute correspond to herein illustrate and/or describe at least some steps and/or function operational order.
The each of present disclosure is described by means of showing the formation function module of the performance of certain critical functions above A aspect.For the convenience of description, the boundary of these formation function modules is any definition.As long as certain critical functions are appropriate Ground executes, so that it may define alternative boundary.Similarly, flow chart is also arbitrarily defined certain important to illustrate herein Function.For use to a certain degree, the boundary of flow chart and sequence can be otherwise defined, as long as remaining to execute these important Function.Therefore this alternative definitions of formation function module and flow chart block and sequence fall into the model of claimed disclosure Within enclosing and being spiritual.
It will be apparent to those skilled in that understanding, foregoing description is intended to illustrate the principle of disclosure.The description is not It is expected that limiting the scope of the disclosure or applying, because not departing from present disclosure as defined by the following claims In the case where spirit, disclosure can be modified, alterations and modifications.

Claims (26)

1. a kind of method for video processing, which comprises
The first video flowing including first resolution video frame is received, at least one of described first resolution video frame is determined Justice be include multiple second resolution video frames;
Presentation user inputs selection;And
The first operator scheme of the multiple second resolution video frame is shown at the same time and shows the multiple second resolution It is selected between the second operator scheme of the scaled version of one of video frame.
2. according to the method described in claim 1, wherein, the first resolution video frame is than the second resolution video frame Pixel with greater number.
3. according to the method described in claim 2, wherein, the number for the pixel that the first resolution video frame has is described Four times of second resolution video frame.
4. according to the method described in claim 1, wherein, the one in the multiple second resolution video frame it is described Scaled version has the pixel of greater number than the multiple second resolution video frame.
5. according to the method described in claim 4, wherein, the one in the multiple second resolution video frame it is described In scaled version and the first resolution video frame it is described at least one with equal number of pixel.
6. according to the method described in claim 1, wherein, each of the multiple second resolution video frame includes difference Program.
7. according to the method described in claim 1, further comprising: receiving the first resolution in Video Codec Frame.
8. according to the method described in claim 7, further comprising: being cut from the first resolution video frame described more A second resolution video frame.
9. according to the method described in claim 1, wherein, the first resolution video frame is 4k × 2k video frame, described Two resolution video frames are 1080p video frames.
10. a kind of equipment for handling video data, the video data include being embedded in high definition video data Low resolution video data, the equipment include:
Memory cell;
Processor is operatively coupled to the memory cell, and the processor is configured to:
High-resolution video frame is buffered in the memory cell, the high-resolution video frame includes multiple lower Resolution video frame;And
The first operator scheme of the multiple lower resolution video frame is rendered at the same time and renders the multiple low resolution It is selected between the second operator scheme of the scaled version of one of video frame.
11. equipment according to claim 10, wherein the high-resolution video frame includes 4 low resolution views Frequency frame.
12. equipment according to claim 10, wherein each of the multiple lower resolution video frame includes not Same program.
13. equipment according to claim 10, wherein the institute of the one in the multiple lower resolution video frame State the pixel that scaled version has greater number than the multiple lower resolution video frame.
14. equipment according to claim 13, wherein the institute of the one in the multiple lower resolution video frame Stating scaled version and the high-resolution video frame has equal number of pixel.
15. a kind of system for handling video data, the system comprises:
Memory cell;
Frame processing unit, is operatively coupled to the memory cell, and the frame processing unit is configured to retrieve multiple High-resolution video frame, extracts multiple lower resolution video frames from the high-resolution video frame, and will it is described compared with Low resolution video frame is stored in the memory cell;And
Video output unit, is operatively coupled to the memory cell, and the video output unit is selectively shown The scaled version of one of the multiple lower resolution video frame or the multiple lower resolution video frame of display.
16. system according to claim 15, wherein the institute of the one in the multiple lower resolution video frame State the pixel that scaled version has greater number than the multiple lower resolution video frame.
17. system according to claim 16, wherein the institute of the one in the multiple lower resolution video frame Stating scaled version and the high-resolution video frame has equal number of pixel.
18. system according to claim 15, wherein the high-resolution video frame is than the low resolution video Frame has the pixel of greater number.
19. system according to claim 18, wherein the number for the pixel that the high-resolution video frame has is institute Four times for stating lower resolution video frame.
20. system according to claim 15, wherein each frame in the multiple lower resolution video frame includes not Same program.
21. a kind of system for handling video data, the system comprises:
Memory cell;
Frame processing unit, is operatively coupled to the memory cell, and the frame processing unit is configured to retrieve multiple Lower resolution video frame extracts the lower resolution video frame from high-resolution video frame, and will be described compared with high score Resolution video frame is stored in the memory cell;
Decoder is operatively coupled to the memory cell, and the decoder includes at least one module, it is described at least One module, which is configured to, decodes institute before extracting the lower resolution video frame in the high-resolution video frame State high-resolution video frame.
22. system according to claim 21, wherein the high-resolution video frame definition is than the low resolution The low frame rate of video frame.
23. system according to claim 21, wherein each framing bit in the multiple lower resolution video frame is in institute In the region for stating high-resolution video frame.
24. system according to claim 21, wherein each frame in the multiple lower resolution video frame includes three Tie up the component frame of video frame.
25. system according to claim 24, wherein at least one frame packet in the multiple lower resolution video frame Include two-dimensional video content.
26. system according to claim 24, wherein at least one of the multiple lower resolution video frame is corresponding Frame includes three dimensional depth layer content.
CN201910341517.XA 2013-04-25 2014-04-24 System and method for handling video data Pending CN110139107A (en)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US201361815860P 2013-04-25 2013-04-25
US61/815,860 2013-04-25
US13/929,739 US8982277B2 (en) 2013-04-25 2013-06-27 System and method for processing video data
US13/929,739 2013-06-27
CN201410168639.0A CN104125501A (en) 2013-04-25 2014-04-24 System and method for processing video data

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201410168639.0A Division CN104125501A (en) 2013-04-25 2014-04-24 System and method for processing video data

Publications (1)

Publication Number Publication Date
CN110139107A true CN110139107A (en) 2019-08-16

Family

ID=51685259

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201910341517.XA Pending CN110139107A (en) 2013-04-25 2014-04-24 System and method for handling video data
CN201410168639.0A Pending CN104125501A (en) 2013-04-25 2014-04-24 System and method for processing video data

Family Applications After (1)

Application Number Title Priority Date Filing Date
CN201410168639.0A Pending CN104125501A (en) 2013-04-25 2014-04-24 System and method for processing video data

Country Status (2)

Country Link
CN (2) CN110139107A (en)
DE (1) DE102014207607B4 (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111356016A (en) * 2020-03-11 2020-06-30 北京松果电子有限公司 Video processing method, video processing apparatus, and storage medium
CN113411670A (en) * 2021-06-18 2021-09-17 康佳集团股份有限公司 Video image display method and device, terminal equipment and storage medium

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP3041233A1 (en) * 2014-12-31 2016-07-06 Thomson Licensing High frame rate-low frame rate transmission technique
WO2016191694A1 (en) 2015-05-27 2016-12-01 Google Inc. Streaming spherical video
CN106657811A (en) * 2016-09-27 2017-05-10 大风(北京)科技有限责任公司 Live photo generating and displaying method and device
CN106657815B (en) * 2016-09-30 2019-11-22 南京巨鲨医疗科技有限公司 A kind of video-splicing processing system and method
CN108810636B (en) 2017-04-28 2020-04-14 华为技术有限公司 Video playing method, virtual reality equipment, server, system and storage medium
CN107493444B (en) * 2017-08-21 2019-11-08 康佳集团股份有限公司 A kind of 8K TV that supporting double-channel signal and its implementation
CN107749857A (en) * 2017-11-01 2018-03-02 深圳市普天宜通技术股份有限公司 Method, storage medium and client a kind of while that check multi-channel video
CN115695905A (en) * 2021-07-26 2023-02-03 华为技术有限公司 Video stream recovery method, electronic device and computer-readable storage medium

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030086003A1 (en) * 2001-10-04 2003-05-08 Tadaharu Koga Video data processing apparatus and method, data distributing apparatus and method, data receiving apparatus and method, storage medium, and computer program
CN1981522A (en) * 2004-05-07 2007-06-13 科巴尔特娱乐有限公司 Stereoscopic television signal processing method, transmission system and viewer enhancements
CN101138041A (en) * 2005-03-15 2008-03-05 汤姆森许可贸易公司 Method and apparatus for encoding plural video signals as a single encoded video signal, method and and apparatus for decoding such an encoded video signal
CN102027741A (en) * 2008-05-29 2011-04-20 国际商业机器公司 Generating a combined video stream from multiple input video streams

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4851826A (en) * 1987-05-29 1989-07-25 Commodore Business Machines, Inc. Computer video demultiplexer
JPH0622215A (en) 1992-07-06 1994-01-28 Toshiba Corp Video transmission method, video transmitter and video receiver used for the same
CA2380105A1 (en) * 2002-04-09 2003-10-09 Nicholas Routhier Process and system for encoding and playback of stereoscopic video sequences
CN101257607B (en) * 2008-03-12 2010-06-09 中兴通讯股份有限公司 Multiple-picture processing system and method for video conference
US8094234B2 (en) 2008-10-14 2012-01-10 Texas Instruments Incorporated System and method for multistage frame rate conversion
CN101742320B (en) * 2010-01-20 2012-09-12 李博航 Image processing method
CN103548357A (en) * 2011-05-19 2014-01-29 Lg电子株式会社 Video stream transmitting device, video stream receiving device, video stream transmitting method, and video stream receiving method

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030086003A1 (en) * 2001-10-04 2003-05-08 Tadaharu Koga Video data processing apparatus and method, data distributing apparatus and method, data receiving apparatus and method, storage medium, and computer program
CN1981522A (en) * 2004-05-07 2007-06-13 科巴尔特娱乐有限公司 Stereoscopic television signal processing method, transmission system and viewer enhancements
CN101138041A (en) * 2005-03-15 2008-03-05 汤姆森许可贸易公司 Method and apparatus for encoding plural video signals as a single encoded video signal, method and and apparatus for decoding such an encoded video signal
CN102027741A (en) * 2008-05-29 2011-04-20 国际商业机器公司 Generating a combined video stream from multiple input video streams

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111356016A (en) * 2020-03-11 2020-06-30 北京松果电子有限公司 Video processing method, video processing apparatus, and storage medium
US11488383B2 (en) 2020-03-11 2022-11-01 Beijing Xiaomi Pinecone Electronics Co., Ltd. Video processing method, video processing device, and storage medium
CN113411670A (en) * 2021-06-18 2021-09-17 康佳集团股份有限公司 Video image display method and device, terminal equipment and storage medium

Also Published As

Publication number Publication date
CN104125501A (en) 2014-10-29
DE102014207607A1 (en) 2014-10-30
DE102014207607B4 (en) 2019-09-19

Similar Documents

Publication Publication Date Title
US9197845B2 (en) System and method for processing video data
CN110139107A (en) System and method for handling video data
JP7289796B2 (en) A method and system for rendering virtual reality content based on two-dimensional ("2D") captured images of a three-dimensional ("3D") scene
KR102208129B1 (en) Overlay processing method and apparatus for 360 video system
US20180199042A1 (en) Methods and apparatus for signaling viewports and regions of interest
US10334285B2 (en) Apparatus, system and method
CN106060578B (en) Generate the method and system of video data
US20190259123A1 (en) System and method of data transfer in-band in video via optically encoded images
KR102640664B1 (en) A method for controlling VR device and a VR device
US20200389640A1 (en) Method and device for transmitting 360-degree video by using metadata related to hotspot and roi
US11197040B2 (en) Deriving and signaling a region or viewport in streaming media
US20130266065A1 (en) Coding and decoding of multiview videos
US20200112710A1 (en) Method and device for transmitting and receiving 360-degree video on basis of quality
CN108965929B (en) Video information presentation method, video information presentation client and video information presentation device
CN105900401A (en) Method, device, and computer program for encoding inter-layer dependencies
CN102833583B (en) Processing method and processing device of video image
US20110249091A1 (en) Video signal processing apparatus and video signal processing method
KR102157658B1 (en) Method and apparatus for transceiving metadata for multiple viewpoints
CN109362242A (en) A kind of processing method and processing device of video data
KR20130127423A (en) Method of picture-in-picture for multimedia applications
CN104602127A (en) Method and system for synchronously playing directed videos and video director
CN110933461B (en) Image processing method, device, system, network equipment, terminal and storage medium
CN112262570B (en) Method and computer system for automatically modifying high resolution video data in real time
EP2309766A2 (en) Method and system for rendering 3D graphics based on 3D display capabilities
US11575871B2 (en) Method and apparatus for streaming multi-view 360 degree video

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20190816