Classifications

• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/75—Media network packet handling
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/597—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding specially adapted for multi-view video sequence encoding
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
  • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
  • H04L65/60—Network streaming of media packets
  • H04L65/75—Media network packet handling
  • H04L65/752—Media network packet handling adapting media to network capabilities
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
  • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
  • H04N13/106—Processing image signals
  • H04N13/161—Encoding, multiplexing or demultiplexing different image signal components
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
  • H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
  • H04N19/136—Incoming video signal characteristics or properties
  • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
  • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
  • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
  • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
  • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
• • H—ELECTRICITY
  • H04—ELECTRIC COMMUNICATION TECHNIQUE
  • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
  • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
  • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
  • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction

Definitions

• Video image transmitting method device and system
• the present invention relates to video image processing technology, and in particular, to a video image transmitting method, apparatus and system. Background technique
• 3D display technology with realistic and stereoscopic effects has been gradually applied to various communication terminals such as televisions, game machines, notebook computers, video players, and mobile phones.
• Using a 3D video player can send 3D video, which can improve the user experience of 3D video services.
• the 3D video playback terminal has two image collection units (cameras), and the two image collection units simultaneously acquire two images taken at different angles, and the images are encoded and sent to the receiver terminal.
• the receiver decodes the received image, and the user feels the video with stereo effect.
• 3D video brings a lot of data when it is expressed in a certain format.
• a large amount of data challenges every aspect of acquiring video, encoding, sending, or displaying 3D video.
• network technology especially in wireless network technology, not only the bandwidth resources are very scarce, but also the power of the infinite terminal in the wireless transmission environment is limited, and a large amount of data is transmitted in a narrow bandwidth, which causes packet loss and image distortion, thereby This makes it impossible to send 3D video efficiently and reliably.
• the H.263 video communication protocol is a communication standard for transmitting video at a low rate (tens of Kbps). Using this communication protocol to transmit video can minimize the amount of data that needs to be transmitted and save bandwidth. Due to the large amount of data of 3D video, 3D video cannot be efficiently and reliably transmitted under the bandwidth required by the existing H.263. Summary of the invention
• Embodiments of the present invention provide a video image transmitting method, apparatus, and system for transmitting 3D video reliably and efficiently under the bandwidth required by the existing H.263.
• a method for transmitting a video image, which is applied to a 3D video image transmission between video playback devices includes:
• the first video image and the macroblock are transmitted to the network side when the I frame video image is transmitted.
• a video image comparing unit configured to compare the collected first video image and the second video image to obtain a macroblock in the second video image that is different from the first video image
• the video image transmitting unit transmits the first video image and the macroblock to the network side when transmitting the I frame video image.
• a video image receiving unit configured to receive a macroblock from the first video image and the second video image that is different from the first video image in the I frame video image from the network side;
• a video image generating unit configured to generate the second video image according to the first video image and the macroblock.
• a video image sending system provided by an embodiment of the present invention includes: a video image sender and a video image receiver;
• the video image sender is configured to compare the collected first video image and the second video image to obtain a macroblock in the second video image that is different from the first video image; and when transmitting the I frame video image, Transmitting the first video image and the macroblock to a network side;
• the video image receiver is configured to receive a first video image in the I frame video image sent by the video image sender and a macroblock in the second video image that is different from the first video image; according to the first The video image and the macroblock generate the second video image.
• a macroblock that is different from the first video image in the second video image is obtained by comparing the first video image and the second video image that are collected, and when the I frame video image is sent, The first video image and the macroblock are sent to the network side; the network side receives the first video image in the I frame video image and the macroblock in the second video image that is different from the first video image; The first video image and the macroblock generate the second video image, and the 3D video is transmitted reliably and efficiently under the bandwidth required by the existing H.263.
• FIG. 1 is a left side view and a right side view of a video image sending method according to an embodiment of the present invention
• FIG. 3 is a schematic flowchart of a video image sending method according to an embodiment of the present invention
• FIG. 4 is a schematic flowchart of a video image sending method according to an embodiment of the present invention
• FIG. 5 is a schematic diagram of a coding structure of an H.263 image layer according to an embodiment of the present invention
• FIG. 7 is a schematic diagram of the data structure of the macroblock layer according to the embodiment of the present invention
• FIG. 8 is a schematic flowchart of a video image receiving method according to an embodiment of the present invention
• FIG. 9 is a schematic structural diagram of a video image transmitting apparatus according to an embodiment of the present invention
• FIG. 10 is a video image according to an embodiment of the present invention
• FIG. 11 is a schematic structural diagram of a video image receiving apparatus according to an embodiment of the present invention
• FIG. 12 is a schematic structural diagram of a video image receiving apparatus according to an embodiment of the present invention
• FIG. 13 is a schematic structural diagram of a video image sending system according to an embodiment of the present invention.
• Embodiments of the present invention provide a video image transmitting method, apparatus, and system for transmitting 3D video reliably and efficiently under the bandwidth required by the existing H.263.
• 3D technology is also called stereo image technology.
• the basic principle is: People's left and right eyes have different views because of the different positions of the viewing scene. The brain processes the different images received from the left and right eyes. People feel the near and far differences of the objects and have a visual sense of the video they see.
• the 3D video playback terminal has two image acquisition units corresponding to the left and right eyes of the person, and the image acquisition unit may be a camera.
• the image acquisition unit can simultaneously collect two video images of the left eye and the right eye, and simultaneously transmit the collected left eye video image (or left view) and right eye video image (or right view) to the video encoder of the video playing terminal.
• the received video image is decoded by the video receiver and the 3D image is displayed. According to the imaging principle, the closer the distance between the object and the lens, the greater the difference between the imaging positions of the left and right cameras.
• the existing 3D technology tries to make the left and right eyes of the user simultaneously see two different video images on the left and right, so that people can feel the stereoscopic effect.
• the H.263 communication protocol is used in the video transmission process.
• This communication protocol uses differential transmission, motion prediction estimation and other means to process the video, which can save bandwidth to the greatest extent.
• the image is transmitted alternately by the I frame and the P frame, and each frame image can finally be in the image.
• the receiver generates two complete images.
• the I frame is commonly referred to as INTRA and is used to transmit a complete image, such as transmitting a complete image of a 2D video, or transmitting two images of a 3D video (a left eye image and a right eye image), and an I frame as a P frame.
• the reference frame is used.
• the P frame is usually called INTER, and is used to transmit the difference from the previous frame.
• the receiving end receives the difference value of the transmission of the P frame, and obtains the complete image of the current frame according to the complete image generated in the previous frame.
• One I frame is sent every time a plurality of P frames are spaced to prevent an image from being restored without a reference frame after an error.
• the above P frame uses predictive coding, and the image receiver and the sender simultaneously predict the moving position of the moving object in the next image for the moving feature of the object in the previous video. If the prediction result is correct, the sender does not need to send any data. , the receiver directly displays the next image according to its own prediction. If the prediction result is incorrect, that is, there is a difference, the image sender only needs to send a motion vector of a certain part of the image to the image receiver, and the receiver adjusts the image according to the motion vector sent by the sender according to the prediction result of the receiver. Get the next image, minimizing the amount of data sent by the network.
• the image transmitted per frame is changed from one frame to two (left eye video image and right eye video image), and the macroblock data of the image is doubled, I frame and P
• the frame needs to send twice the data equivalent to 2D video, which greatly increases the bandwidth of the transmitted data.
• the embodiment of the present invention adds a left and right view comparing unit to an existing image encoder or an image encoder, and adds a left and right view generating unit to an existing decoder or an image encoder.
• the left and right view comparing unit and the left and right view generating unit are configured to send only one of the complete view and the view of the other view that is different from the full view when transmitting the I frame image, without sending the complete The same image as the image saves bandwidth.
• the present invention when transmitting a p-frame image, only one set of motion vectors needs to be sent to the video interface. Receiving party. This is because if the scene in the video moves, the motion displacements in the left and right views are the same, that is, the motion vectors are the same, only one motion vector is sent, and the receiver can simultaneously apply the motion vectors to the left and right. On the view.
• the present invention can efficiently and preferably transmit 3D video under the bandwidth required by the existing H.263 communication protocol.
• a video image sending method for sending a 3D video, and the method includes the following steps:
• the method further includes: encoding the first video image and the macroblock, and sending the encoded video image to a network side .
• the motion displacement for characterizing the first video image or the second video image of the P frame video image with respect to the previous frame video image is transmitted to the network side.
• the network side described in the embodiment of the present invention is a device that receives video.
• a video number image receiving method includes the following steps:
• the second video image is generated according to the first video image and the macroblock.
• the second video image is generated according to the first video image and the macroblock, And: merging the same macroblock in the first video image as the second video image with the macroblock in the second video image that is different from the first video image to generate a second video image.
• the method further includes: The video image and the macroblock in the second video image that are different from the first video image are decoded.
• the video image receiving method further includes: a P-frame video image sent by the receiving terminal, used to represent the P-frame video image relative to the first video image or the second video image in the previous frame video image. Motion displacement; and performing motion prediction compensation on the first video image and the second video image in the video image of the previous frame according to the set of motion vectors to generate a first video image and a second video image of the current frame.
• the sender mentioned below is a video image terminal device
• the receiver is a device for receiving video images on the network side
• the sender device or the receiver device may be any video playback device supporting 3D video.
• the left view mentioned is the left eye video image
• the right view is the right eye video image.
• the video playback device When the video playing device is used as a sender for transmitting a video image, the video playback device provided by the embodiment of the present invention sends a video.
• the process is as shown in FIG. 4, and includes the following steps:
• the sender collects a left view and a right view.
• two video capturing units on the video playing device such as a camera, collect two video images at a certain time, image A and image B, at different angles (generally two left and right cameras), and each image includes Vision and close-up.
• the foreground is generally a background in a video frame or a scene farther from the lens. There is almost no difference in the position of the perspective in the left and right views in the entire view. Among them, left and right two cameras If the position difference of the imaging does not exceed a certain pixel, there is no difference, for example, no more than 2 pixels or 1 pixel is no difference. Similarly, the close-up is a scene in the video picture that is closer to the lens. The position of close-range imaging differs greatly. Therefore, in the process of transmitting a video image in the embodiment of the present invention, the foreground of the I frame does not need to be repeatedly transmitted.
• the macroblock is a macroblock having a difference in imaging positions in the left and right views.
• the image comparison unit placed in the image encoder or independent of the image encoder matches the macroblocks in the left view and the right view to find out differences in the left view and the right view.
• Macro block For example, find a macroblock with a left view that is different from the right view, as opposed to the right view.
• the image encoder encodes the macro block and the right view.
• the ninth bit in the PTYPE for representing the image type in the coding structure of the image layer is " ⁇
• the image representing the frame is INTRA encoded, that is, I frame.
• the ninth bit in PTYPE is "0"
• Block also known as a macroblock.
• the group of blocks of the I frame contains image information of the left and right views, wherein the right view is a complete image, and the left view includes only the portion of the actual data that is different from the right view at the pixel level.
• one block data (an macroblock data) in an I frame contains all macroblocks of one picture.
• one block group data only includes a macroblock and a close-up macroblock in one view, and a macroblock in another view that is different from the close-view macroblock, and one of the views is omitted.
• a distant macroblock and a partial close macroblock in the view (the partial close macroblock is A macroblock of the same macroblock in a view).
• comparing the I-frame 3D view including two complete views at the same time greatly reduces the data amount of the encoder-encoded image, and also reduces the transmission amount of the 3D video image data. Compared with the 2D view of the I frame encoding an image, only the macroblock with a small amount of data is added, and the bandwidth of the network transmission is hardly increased.
• encoding of an image includes I-frame coding, P-frame coding, and PB frame coding.
• I-frame coding In the image coding technique, encoding of an image includes I-frame coding, P-frame coding, and PB frame coding.
• P-frame coding In the image coding technique, encoding of an image includes I-frame coding, P-frame coding, and PB frame coding.
• PB frame coding The process of encoding the P frame of the present invention will be briefly described below.
• the P frame is a frame for motion prediction and differential transmission. In combination with motion prediction and compensation, only the difference between each image and the prediction is transmitted.
• the COD of the macroblock layer is 0, and no data is transmitted later. Otherwise, the COD is 1 and the motion vector is sent or the motion vector and block data are sent.
• the motion vector is different from the motion prediction.
• the image itself does not mutate, then the P frame only sends a set of motion vectors, and the set of motion vectors is the motion of the left view.
• the motion vector of the vector or the right view, the motion vector for sending the left view or the right view is the same, because once the scene in the video moves, the motion displacements in the left and right views are the same, and the motion vectors are the same. Therefore, only one set of motion vectors is transmitted.
• the receiver When the receiver receives the motion vector, it acts on the left and right maps that have been decoded. S304. Send the encoded macroblock and right view to the receiver.
• the macro block and the right view belong to an I frame image.
• the video playback device When the video playing device is used as a receiving device for receiving a video image, the video playback device provided by the embodiment of the present invention receives a video.
• the process is as shown in FIG. 8, and includes the following steps:
• S401 Receive an I-frame video image sent by the sender, specifically, receive the macroblock and the right view.
• the received full view (such as the right view) and the incomplete view (such as the view containing only the difference macroblocks) are decoded in the normal decoding mode.
• the video image generating unit splices the macroblocks that are the same as the left view in the decoded right view, and the macroblocks that are different from the left view and the right view, to generate a complete left view. Or by adding the same macroblock in the right view as the left view to the incomplete left view.
• a video sending apparatus includes:
• the video image comparing unit 11 is configured to compare the collected first video image and the second video image to obtain a macroblock in the second video image that is different from the first video image;
• the video image transmitting unit 12 transmits the first video image and the macroblock to the network side when transmitting the I frame video.
• the video image sending unit 12 is further configured to: when transmitting the P frame video image, send, to the network side, the video image used to represent the P frame relative to the video image in the previous frame video image or The motion displacement of the two video images.
• the device further includes:
• a video image encoding unit 13 which may be included in the video image comparing unit 11, or may be independent of the video image comparing unit 11 for the first video image and the macro
• the block is encoded, and the encoded video image is sent to the network side;
• a video image receiving unit 14 configured to receive a macroblock from the first video image and the second video image that is different from the first video image in the I frame video image from the network side; and receive the P frame video image for use in Characterizing a motion displacement of the P frame video image relative to the first video image or the second video image in the previous frame video image;
• a video image generating unit 15 configured to generate the second video image according to the first video image and the macro block, specifically, the same macro block as the second video image in the first video image, and And merging the macroblocks in the second video image that are different from the first video image to generate a second video image; and performing, according to the set of motion vectors, the first video image and the second video image of the previous frame.
• Motion prediction compensation generating a first video image and a second video image of the current frame.
• the video image decoding unit 16 is configured to decode the first video image and the macroblock.
• a video image receiving apparatus includes: a video image receiving unit 21, configured to receive a first video image and a second video image in an I frame video image from a network side. a macroblock having a difference in the video image; and a motion displacement in the received P frame video image for characterizing the video image of the P frame relative to the first video image or the second video image in the video image of the previous frame.
• a video image generating unit 22 configured to generate the second video image according to the first video image and the macroblock, specifically, the same macroblock in the first video image as the second video image, and A macroblock in the second video image that is different from the first video image is spliced, Generating a second video image; and performing motion prediction compensation on the first video image and the second video image of the previous frame according to the set of motion vectors to generate a first video image and a second video image of the current frame.
• the video image transmitting apparatus further includes:
• the video image decoding unit 23 is configured to decode the first video image and the macro block.
• a video image transmitting system includes: a video image sender 33 and a video image receiver 34;
• a video image sender 31 configured to compare the collected first video image and the second video image to obtain a macroblock that is different from the first video image in the second video image; and when transmitting the I frame video image, Transmitting the first video image and the macroblock to a network side;
• a video image receiver 32 configured to receive a first video image in an I frame video image sent by the video image sender and a macroblock in the second video image that is different from the first video image; according to the first video The image and the macroblock generate the second video image.
• the device for transmitting 3D video may be adapted to send 2D video.
• an image collection unit is turned off, and the video image comparison unit 11 is configured to receive the 2D video.
• the image is not processed, and the 2D video image is sent to the encoder for encoding; the video image generating unit 13 of the receiving party does not perform any processing on the received 2D image, and directly displays the 2D video.
• the method and device for transmitting 3D video provided by the embodiments of the present invention can be compatible with 2D and 3D video transmission under the existing H.263 protocol, so that users can transmit according to each other according to the feature of the model.
• 3D2D or / and 3D video playback support capability should be added in H.245 (Call Control Protocol in H.324 Protocol Cluster), and the number of macroblocks should be handshaked to avoid communication errors.
• H.245 Click Control Protocol in H.324 Protocol Cluster
• the 3D videophone will be activated only after the H.245 handshake is successful.
• the video portion is still The original H.263 protocol requires codec.
• the left and right view generating units are added by adding the left and right view comparing units in front of the existing image encoder or the image encoder, and adding the left and right view generating units to the existing decoder or the image encoder.
• the left and right view comparing unit and the left and right view generating unit are configured to send only one of the complete view and the view of the other view that is different from the full view when transmitting the I frame image, without sending the complete The same partial image of the image.
• transmitting a p-frame image it is only necessary to transmit the motion vector of one view in one frame to the video receiver. That is to say, the P frame for transmitting 3D video consumes the same amount of data as the 2D video.
• the I frame for transmitting 3D video consumes a slightly larger amount of data than the 2D video. Under the bandwidth required by the existing H.263 communication protocol, bandwidth is saved on the basis of ensuring 3D image transmission, and 3D video can be efficiently and qualityally transmitted.
• embodiments of the present invention can be provided as a method, system, or computer program product.
• the present invention can take the form of an entirely hardware embodiment, an entirely software embodiment, or a combination of software and hardware.
• the invention can be embodied in the form of one or more computer program products embodied on a computer-usable storage medium (including but not limited to disk storage and optical storage, etc.) containing computer usable program code.
• These computer program instructions can also be stored in a bootable computer or other programmable data processing
• the apparatus is readable in a computer readable memory in a particular manner, such that instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device implemented in one or more flows and/or block diagrams of the flowchart The function specified in the box or in multiple boxes.
• These computer program instructions can also be loaded onto a computer or other programmable data processing device such that a series of operational steps are performed on a computer or other programmable device to produce computer-implemented processing for execution on a computer or other programmable device.
• the instructions provide steps for implementing the functions specified in one or more of the flow or in a block or blocks of a flow diagram.
• the present invention obtains a macroblock which is different from the first video image in the second video image by comparing the first video image and the second video image collected; and when the I frame video image is transmitted, the first video is And the macroblock is sent to the network side; the network side receives the first video image in the I frame video image and the macroblock in the second video image that is different from the first video image; according to the first The video image and the macroblock generate the second video image, enabling reliable and efficient transmission of 3D video under the bandwidth required by the existing H.263.

Landscapes

• Engineering & Computer Science (AREA)
• Multimedia (AREA)
• Signal Processing (AREA)
• Computer Networks & Wireless Communication (AREA)
• Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
• Compression Or Coding Systems Of Tv Signals (AREA)
• Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)

Priority Applications (3)

Applications Claiming Priority (2)

Publications (1)

Family

ID=46483310

Family Applications (1)

Country Status (5)

Families Citing this family (10)

Citations (3)

Family Cites Families (30)

• 2012
  • 2012-02-27 CN CN2012100459460A patent/CN102595164A/zh active Pending
  • 2012-05-17 JP JP2014557966A patent/JP2015511780A/ja active Pending
  • 2012-05-17 EP EP12869783.6A patent/EP2822278A4/de not_active Withdrawn
  • 2012-05-17 US US14/380,990 patent/US9912714B2/en active Active
  • 2012-05-17 WO PCT/CN2012/075676 patent/WO2013127126A1/zh active Application Filing

Patent Citations (3)

Also Published As

Similar Documents

Legal Events