US20210368177A1 - Joint Source Channel Coding with H.265 Video Compression - Google Patents
Joint Source Channel Coding with H.265 Video Compression Download PDFInfo
- Publication number
- US20210368177A1 US20210368177A1 US17/051,531 US201917051531A US2021368177A1 US 20210368177 A1 US20210368177 A1 US 20210368177A1 US 201917051531 A US201917051531 A US 201917051531A US 2021368177 A1 US2021368177 A1 US 2021368177A1
- Authority
- US
- United States
- Prior art keywords
- coarse
- data
- refinement
- dct
- module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
- H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/124—Quantisation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/129—Scanning of coding units, e.g. zig-zag scan of transform coefficients or flexible macroblock ordering [FMO]
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
- H04N19/132—Sampling, masking or truncation of coding units, e.g. adaptive resampling, frame skipping, frame interpolation or high-frequency transform coefficient masking
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/136—Incoming video signal characteristics or properties
- H04N19/14—Coding unit complexity, e.g. amount of activity or edge presence estimation
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
- H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
- H04N19/159—Prediction type, e.g. intra-frame, inter-frame or bidirectional frame prediction
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/18—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a set of transform coefficients
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/46—Embedding additional information in the video signal during the compression process
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/85—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression
- H04N19/88—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using pre-processing or post-processing specially adapted for video compression involving rearrangement of data among different coding units, e.g. shuffling, interleaving, scrambling or permutation of pixel data or permutation of transform coefficient data among different blocks
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
- H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L69/00—Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
- H04L69/04—Protocols for data compression, e.g. ROHC
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/33—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
Definitions
- the present invention relates to video communications generally and more particularly to a system for hierarchal video transmission and reception.
- the video may be generated in an uncompressed manner (e.g. by a GPU overlaying graphics over movies) such that the input to the wireless system at one end may consist of pixels and the output of the system at the other end may be a reproduction of the input pixels.
- the wireless systems should generally provide for relatively high quality where the output pixels are as similar as possible to the input pixels.
- the systems should additionally provide for relatively low latency where the total delay of the transmission is low and for relatively high robustness to temporal degradations in the capacity of the wireless channel.
- They are generally configured to include a video encoder and a wireless transmitter.
- the video encoder may be based on a known compression standard such as, for example, H.264 or H.265.
- An advantage of using compression schemes based on known compression standards is that there is a relatively large offering of commercially available designs which may be purchased as hardware and/or software and may be integrated into new or existing systems. This may facilitate the development of “standard-compression” based systems.
- the wireless transmitter may additionally include a modem which may be based on a known communication standard such as, for example, Wi-Fi.
- JSCC Joint-Source-Channel-Coding
- a wireless video transmission system which may include a coarse compression module which may compress a video frame and to generate coarse data of the video frame, a coarse decompression module which may generate coarse frame information from the coarse data, distortions extractor which may generate coarse distortions from the coarse frame information and from the video frame, a refinement data encoder which may generate refinement data from the coarse frame distortions based on information derived from the coarse frame information, and a data combining and modulation module which may combine and transmit the coarse data and the refinement data.
- a coarse compression module which may compress a video frame and to generate coarse data of the video frame
- a coarse decompression module which may generate coarse frame information from the coarse data
- distortions extractor which may generate coarse distortions from the coarse frame information and from the video frame
- a refinement data encoder which may generate refinement data from the coarse frame distortions based on information derived from the coarse frame information
- a data combining and modulation module which may combine and transmit the coarse data and the refinement data.
- the refinement data encoder may generate quantized transform coefficients from the coarse data.
- the quantized transform coefficients may include quantized DCT coefficients.
- the refinement data encoder may additionally detect block energy in the quantized transform coefficients and may additionally estimate a probability density of the quantization error in the quantized transform coefficients.
- the refinement data encoder may generate and reorder transform coefficients from the coarse distortions.
- the refinement data encoder may additionally apply a gain to the reordered transform coefficients, wherein the gain is derived from the coarse data.
- the reordered transform coefficients may include quantized DCT coefficients.
- the refinement data encoder may entropy encode the reordered transfer coefficients.
- the entropy coding codebook used for the entropy encoding may be based on the probability density of the quantization error.
- the refinement data encoder may additionally compand the reordered transfer coefficients.
- the refinement data encoder may additionally perform a linear transform on the reordered transfer coefficients.
- the coarse compression module may divide the video frame into a plurality of blocks.
- a method of wireless video transmission which may include compressing a video frame to generate coarse data of the video frame, decompressing the coarse data to generate coarse frame information from the coarse data, generating coarse distortions from the coarse frame information and from the video frame, generating refinement data from the coarse distortions based on information derived from the coarse frame information, and combining and transmitting the coarse data and the refinement data.
- the method may include generating quantized transform coefficients from the coarse data.
- the quantized transform coefficients may include quantized DCT coefficients.
- the method may additionally include detecting block energy in the quantized transform coefficients.
- the method may additionally include estimating a probability density in the quantized transform coefficients.
- the method may include generating and reordering transform coefficients from the coarse distortions.
- the method may additionally include applying a gain to the reordered transfer coefficients.
- the reordered transform coefficients may include quantized DCT coefficients.
- the method may additionally include entropy encoding the reordered transfer coefficients.
- the method may additionally include companding the reordered transfer coefficients.
- the method may additionally include performing a linear transform on the reordered transfer coefficients.
- the method may include dividing the video frame into a plurality of blocks.
- FIG. 1 schematically illustrates an exemplary two-layer video compression and transmission system for transmitting video over a communication channel, according to an embodiment of the present invention
- FIG. 2 schematically illustrates an exemplary system for reception and reconstruction of video from a signal received over a communication channel, according to an embodiment of the present invention
- FIG. 3 schematically illustrates an exemplary refinement coding module, which may be used in the video compression and transmission system of FIG. 1 , according to an embodiment of the present invention
- FIG. 4 schematically illustrates an exemplary refinement decoding module, which may be used in the system for reception and reconstruction of video of FIG. 2 , according to an embodiment of the present invention
- FIG. 5A schematically illustrates an exemplary refinement coding module, which may be used in the video compression and transmission system of FIG. 1 , according to an embodiment of the present invention
- FIG. 5B schematically illustrates three exemplary graphs denoted (a), (b) and (c), representing probability density functions of three DCT taps, respectively, according to an embodiment of the present invention
- FIG. 6 schematically illustrates an exemplary refinement decoding module, which may be used in the system for reception and reconstruction of video of FIG. 2 , according to an embodiment of the present invention
- FIG. 7 schematically illustrates an exemplary DCT taps processor, which may be used with the refinement coding module of FIG. 5 , according to an embodiment of the present invention.
- FIG. 8 schematically illustrates several different types of compander functions, which may be used with the DCT taps processor of FIG. 7 , according to an embodiment of the present invention.
- wireless video transmission systems which combine use of JSCC with known compression block techniques including compression standards such as, for example, H.265, may suffer from bandwidth overhead.
- H.265 data with “analog” refinement data.
- the system may employ “coarse” bins and “fine” bins where the coarse bins are used for sending H.265 data (including interframe prediction) and the fine bins are used for sending the data for memoryless refinement of the received image.
- the analog refinement may have video-content-dependent power, and may suffer from additive noise and therefore, it may be desired to apply a video-content-dependent gain to bring the transmitted power to the highest level available by the hardware capabilities and the radio regulation.
- Video-content-dependent gain may be applied to single blocks (that is, each block will have a different gain), or to group of blocks (that is, each group will have different gain, but all the blocks in the group will have the same gain).
- bandwidth overhead in video transmission systems combining use of known compression block techniques (e.g. H.264 or H.265) with JSCC may be reduced, optionally eliminated, by utilizing information associated with the coarse data to generate control data for the refinement encoder, where the refinement encoder generates refinement data which may be transmitted to a video receiver for memoryless refinement of a video image.
- the transmission side may include a refinement encoder which may be adapted to perform a two-dimensional DCT transform on coarse distortions which may be in the form of error frames (in the pixel domain) or in the form of quantization errors of DCT taps generated from the coarse data to generate DCT taps for each block.
- the refinement encoder may additionally calculate the block energy to apply a gain to the coarse distortions DCT taps based on the power of the coarse DCT taps.
- the refinement encoder may estimate a probability density for the coarse DCT taps which may then be used to entropy encode the coarse distortions DCT taps.
- a compander may be used to apply non-linear gain to the DCT taps of the refinement encoder.
- the receiving side may include a refinement decoder which may decode the received refinement data based on the received coarse DCT taps.
- the refinement decoder may calculate the block energy to apply a gain to the received refinement DCT taps based on the power of the coarse DCT taps.
- the refinement decoder may estimate a probability density for the coarse DCT taps which may then be used to entropy decode the received refinement DCT taps.
- the same gain applied by the refinement encoder may be applied by the refinement decoder, thereby eliminating the need to send the gain information to the receiver and reducing bandwidth overhead.
- the same probability density function may be used for the entropy decoding, thereby eliminating the need to send the probability density function information to the receiver and reducing bandwidth overhead.
- FIG. 1 schematically illustrates an exemplary two-layer video compression and transmission system 100 for transmitting video over a communication channel 150 , according to an embodiment of the present invention.
- System 100 may include a coarse compression module 102 , a coarse decompression module 104 , a coarse distortions extractor module 106 , a refinement coding module 108 , a data combining and modulation module 110 .
- Coarse compression module 102 may receive pixels of an original video frame 10 and may generate a compressed description of the video which may be denoted as coarse data 12 .
- Coarse compression module 102 may use inter-frame prediction in performing the data compression to generate coarse data 12 .
- Inter frame prediction may include use of a forward inter-frame predictor to generate P-frames and optionally also a bi-directional inter-frame predictor to generate B-frames.
- coarse compression module 102 may wholly or partially avoid generating B-frames (bidirectionally predicted frames).
- Coarse decompression module 104 may parse and decode the coarse data 12 to generate a coarse frame 14 .
- the coarse distractions extractor may operate in the DCT domain, and subtract the DCT taps of the original frame from the DCT taps of the Coarse frame to generate quantization errors of the DCT taps.
- Refinement coding block module 108 may compress coarse distortions 16 and may generate a refinement data description (refinement data 18 ). Refinement coding block module 108 may adapt its coding parameters according to the content of the coarse data. For example, it may perform two dimensional DCT and may send only a subset of the DCT taps, for example, the first N taps. It may additionally select the value of N based on the coarse description of that block according to a pre-defined algorithm for selecting N. Since the coarse data is transmitted as well, the receiver may select the same value of N for decoding the refinement data, according to a pre-defined algorithm for selecting N, which is equivalent to the pre-defined algorithm in the encoder. This may eliminate the need to specifically encode and send the value of N.
- Data combining and modulation module 110 may combine coarse data 12 and refinement data 18 , and transmit them over a communication channel 150 .
- Data combining and modulation module 110 may modulate the signal using OFDM, with a portion of the bins (“digital bins”) allocated for the coarse data and a portion of the bins (“fine bins”) allocated to the refinement data.
- digital bins a portion of the bins
- fine bins a portion of the bins allocated to the refinement data.
- a different modulation and coding scheme may be used for the digital bins and the fine bins, and the fine bins may include properties similar to analog transmission.
- the same modulation and modulation scheme may be used for the coarse data and the fine data.
- FIG. 2 schematically illustrates an exemplary system 200 for reception and reconstruction of video from a signal received over a communication channel 250 , according to an embodiment of the present invention.
- System 200 may include a demodulation and data splitting module 202 , a coarse decompression module 204 , a refinement decoding module 206 , and a frame combiner module 208 .
- System 200 may be used to receive a video signal transmitted by transmission system 100 over channel 150 , in which case channel 150 and channel 250 is a same channel.
- Demodulation and data splitter module 202 may demodulate the received signal and may split the signal into coarse data 20 and received refinement data 22 .
- Coarse data 20 may be substantially identical to coarse data 12 sent by transmission system 100 , and may include a small bit error rate compared to coarse data 12 associated with, for example, packet errors.
- Coarse decompression module 204 may decode the video and may generate coarse frame information 24 .
- Coarse decompression module 204 may optionally be a standard video decoder such as, for example, a MPEG decoder, a H.264 decoder, or a H.265 decoder, among other standard video decoders.
- Refinement decoding module 206 may receive refinement data 22 and may generate reconstructed coarse distortions 26 . Refinement decoding module 206 may adapt its decoding parameters according to the content of coarse data 20 . For example, it may decode from coarse data 20 the value N of the number of transmitted DCT taps and may then reconstruct the DCT values by zeroing all the taps starting from tap N+1.
- Frame combiner module 208 may combine the coarse frame information 24 with reconstructed coarse data 26 , for example by summing the two frames together pixel by pixel, optionally reducing a fixed offset (e.g. 128 for 8 bits pixels) and optionally limit the result between two values, for example, 0 and 255. Frame combiner module 28 may then generate a reconstructed frame 28 . Alternatively, frame combiner module may combine the coarse frame information 24 with the reconstructed coarse data which may include the quantization errors of the DCT taps to generate the reconstructed frame 28 .
- a fixed offset e.g. 128 for 8 bits pixels
- FIG. 3 schematically illustrates an exemplary implementation of refinement coding module 108 , which may be used in video compression and transmission system 100 , according to an embodiment of the present invention.
- Refinement coding module 108 may include a quantized DCT taps extractor 302 , a block energy detector module 304 , an optional refinement DCT taps generator module 306 , and a DCT taps processor 308 .
- original frame 10 may be divided into blocks, for example, block of 8 ⁇ 8 pixel or 4 ⁇ 4 pixels or 16 ⁇ 16 pixel blocks.
- Coarse compression module 102 may perform a certain transform, for example a two-dimensional DCT, on each block.
- Coarse compression module 102 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps.
- the value of quantized DCT tap number j of color c of block number i may be denoted DCTQi,c,j.
- DCT tap extractor module 302 may extract from coarse data 12 the quantized DCT taps and may transfer this data as quantized DCT taps 13 to block energy detector module 304 .
- Block energy detector module 304 may determine, for each block, a certain gain, represented by refinement block gain 15 , which may be applied by DCT taps processor 308 to reordered DCT taps 17 to generate refinement data 18 .
- Refinement DCT taps Generator module 306 may perform a certain transform, for example a two-dimensional DCT, on each block of error frame in coarse distortions 16 if in the pixel domain DCT & reordering module 306 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps to generate reordered refinement DC taps 17 .
- a certain transform for example a two-dimensional DCT
- DCT taps processor 308 may generate refinement data 18 which may be a linear transform of the reordered DC taps 17 following the processor applying a gain to the DCT taps or to the linear transform of the DCT taps based on the refinement block gain determined by block energy detector module 304 .
- an exemplary mode of operation of block energy detector module 304 may be as follows:
- Block energy detector 304 may first calculate for each block i the block energy Ei by, for example:
- M is the number of DCT taps per block, and it is assumed that the 2D DCT is performed separately over each of the 3 colors.
- Block energy detector module 304 may then use a certain function, for example,
- G i min ⁇ ( G max , max ⁇ ( G min , E r ⁇ e ⁇ f E i ) )
- G i ⁇ G H if ⁇ ⁇ E i ⁇ T G L if ⁇ ⁇ E i ⁇ T
- another exemplary mode of operation of block energy detector module 304 may be as follows:
- Block energy detector module 304 may first calculate for each superblock, the superblock energy, denoted Esuperblock, by:
- M is the number of DCT taps per block, and it is assumed that the DCT is performed over 3 colors.
- Block energy detector module 304 may then use a function, such as:
- G i ′ min ( G max ′ , max ⁇ ( G min ′ ⁇ E ref ′ E superblock )
- FIG. 4 schematically illustrates an exemplary implementation of refinement decoding module 206 , which may be used in system 200 for reception and reconstruction of video, according to an embodiment of the present invention.
- Refinement decoding module 206 may include a quantized DCT taps extractor module 402 , a block energy detector module 404 , a DCT taps extractor 406 , and an optional reordering & IDCT module 408 .
- coarse data 20 may be substantially identical to coarse data 12 sent by transmission system 100 , and may include a small bit error rate compared to coarse data 12 associated with the packet error rate.
- quantized DCT taps extractor module 402 and block energy detector module 404 in refinement decoding module 206 may be substantially identical to quantized DCT taps extractor module 302 and block energy detector module 304 in refinement coding module 300 in FIG. 3 .
- refinement block gain 25 calculated by block energy detector module 404 may be substantially identical to refinement block gain 15 calculated by block energy detector module 304 of FIG. 3 .
- the quantized DCT taps extractor module and the energy block detector module are substantially identical to that in transmission system 100 , and since the values of DCTQ,i,c,j will be made available to refinement decoding module 206 in receiver system 200 , there is no need to send the value of Gi and of G′i over channel 150 , thereby allowing for a reduction (saving) in bandwidth. That is, the values Gi and G′i, which were calculated in refinement coding module 300 in FIG. 3 , may be regenerated in refinement decoding module 206 , and there is no need to send them.
- DCT taps extractor 406 may extract the DCT taps from the received refinement data 22 and may generate received reordered refinement DCT taps 27 from received refinement data 22 based on refinement block gain 25 computed by block energy detector module 404 .
- Optional reordering & IDCT module 408 may reorder received reordered DCT taps 27 and may perform an inverse transform on the DCT taps to generate reconstructed coarse distortions 26 in the form of error frames if received reordered refinement DCT taps 27 is in the pixel domain. If received reordered refinement DCT taps 27 includes quantization errors of the DCT taps, reordering & IDCT module 408 may be obviated.
- FIG. 5A schematically illustrates an exemplary refinement coding module 500 which may be used in video compression and transmission system 100 , according to an embodiment of the present invention.
- Refinement coding module 500 may include a quantized DCT taps extractor 502 , a refinement probability function estimator module 504 , an optional refinement DCT taps generator module 506 , and a DCT taps processor 508 .
- original frame 10 may be divided into blocks; for example, block of 8 ⁇ 8 pixels, or 4 ⁇ 4 pixel or 16 ⁇ 16 pixel blocks.
- Coarse compression module 102 may perform a certain transform, for example a two-dimensional DCT, on each block.
- Coarse compression module 102 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps.
- the DCT taps may be denoted as a random variable x and its quantized value may be denoted as Q(x).
- Quantized DCT taps extractor module 502 and DCT & reordering module 506 may be substantially similar to quantized DCT taps extractor module 302 and refinement DCT taps module 306 , respectively, in FIG. 3 .
- Refinement probability density function estimator module 504 may estimate, for each Quantized DCT tap 13 , the probability density function of the corresponding DCT tap of the error frame.
- DCT taps processor 508 may use refinement probability density function per DCT tap 30 to process reordered refinement DCT taps 17 and generate refinement data 18 .
- DCT taps processor 508 may further quantize reordered DCT taps 17 using a quantizer which may include quantization cell size smaller than that in the quantizer used by coarse compression module 102 . Following quantization, DCT taps processor 508 may use entropy encoding to compress the index of the quantization cell and generate a string of bits which may indicate the index of the fine quantization cell and where the string of bits may be optimized to reduce the average number of bits in the stream.
- Entropy encoders are generally optimized for a certain probability density of the source. To yield a good code book, the probability density of the source is estimated. However, as shown in FIG. 5B , the conditional probability density function of a certain DCT tap may vary and may depend on its quantized value.
- Refinement Probability density function estimator module 504 may receive the quantized value at its input, and responsively, may estimate the conditional probability density function to generate refinement probability density function per DCT tap 30 .
- DCT taps processor 508 may then use refinement probability density function per DCT tap 30 to select the proper entropy code-book which is best fitted to the probability density function.
- FIG. 5B schematically illustrates three exemplary graphs (a), (b) and (c) representing probability density functions 550 , 552 , and 554 of three DCT taps, respectively, according to an embodiment of the present invention.
- Graph (a) describes the probability density function 550 of a certain DCT tap. Also shown in the graph are the quantization points and quantization cell boundaries of a uniform quantizer with quantization cell size of ⁇ and quantization points at 0, ⁇ , ⁇ 2 ⁇ , ⁇ 3 ⁇ , . . . .
- Graph (c) describes the probability density function 554 of a certain DCT whose quantization point is +2 ⁇ . It may be appreciated that
- FIG. 6 schematically illustrates an exemplary refinement decoding module 600 , which may be used in system 200 for reception and reconstruction of video, according to an embodiment of the present invention.
- Refinement decoding module 600 may include a quantized DCT taps extractor module 602 , a refinement probability density function estimator module 604 , a DCT taps extractor 606 , and an optional reordering & IDCT module 608 .
- coarse data 20 may be substantially identical to coarse data 12 sent by transmission system 100 , and may include a small bit error rate compared to coarse data 12 associated with the packet error rate.
- quantized DCT taps extractor module 602 and refinement probability density function estimator module 604 in refinement decoding module 600 may be substantially identical to quantized DCT taps extractor module 502 and refinement probability density function estimator module 604 in refinement coding module 500 in FIG. 5A .
- refinement probability density function per DCT tap 32 determined by refinement probability density function estimator module 604 may be substantially identical to refinement probability density function per DCT tap 30 determined by refinement probability density function estimator module 504 of FIG. 5A .
- refinement probability density function per DCT tap 37 estimated at refinement decoding block 600 is substantially identical to refinement probability density function per DCT tap 30 estimated at the refinement encoding block 500 of FIG. 5A , there is no need for the transmitter to send the refinement probability density function per DCT tap to the receiver, which may allow for a reduction in bandwidth.
- DCT taps extractor 606 may extract the DCT taps from the received refinement data and may generate received reordered refinement DCT taps 27 from received refinement data 22 based on refinement probability density function per DCT tap 37 determined by refinement probability density function estimator module 604 .
- Optional reordering & IDCT module 608 may reorder received reordered DCT taps 27 and may perform an inverse transform on the DCT taps to generate reconstructed coarse distortions 26 in the form of error frames if received reordered refinement DCT taps 27 is in the pixel domain. If received reordered refinement DCT taps 27 includes quantization errors of the DCT taps, reordering & IDCT module 408 may be obviated.
- FIG. 7 schematically illustrates an exemplary DCT taps processor 700 , which may be used with refinement coding module 500 in video transmission system 100 , according to an embodiment of the present invention.
- DCT taps processor 700 may include a compander selector per DCT tap module 702 , a compander module 704 , and a linear transform module 706 .
- DCT taps processor 700 may be the same as DCT taps processor 508 in FIG. 5 .
- Compander module 704 may generate companded DCT taps 34 and may include use of a non-liner function to apply a gain to reordered refinement DCT taps 17 .
- a non-linear function which may be used in Compander 704 may be, for example, as in the following Equation (1):
- C1 is a constant
- X is the input
- C(x) is the output
- Equation (2) Another non-linear function which may be used, may be as in the following Equation (2):
- the transmission power is usually limited both by RF (radio frequency) regulations and by the capabilities of the power amplifier.
- RF radio frequency
- the optimal choice of the constants G 1 and G 2 and T generally depends on the probability density function of the input X. For example, if X is known to be zero-mean with low variance, then large values of G 1 may be used (which may improve the receiver SNR). But if X is known to be zero-mean with high variance, then small values of G 1 and G 2 are generally used, to avoid breaching the RF regulations or compressing and/or overloading the power amplifier.
- Compander selector per DCT taps module 702 receives, for each DCT tap, the refinement probability density as indicated by refinement probability density function per DCT tap 30 , and uses it to select the type of compander 704 as indicated by compander type 33 . That is, compander selector per DCT taps 702 may chose the values of G 1 and G 2 and T according to the refinement probability density.
- Linear transform module 706 may apply a linear transform to companded DCT taps 18 to generate refinement data 18 .
- FIG. 8 schematically illustrates several different types of compander functions 800 , which may be used with DCT taps processor 700 , according to an embodiment of the present invention.
- Type 1 compander function 802 may be suitable for high power DCT taps;
- Type 2 compander 804 may be suitable for medium power DCT taps;
- Type 3 compander 806 may be suitable for low power DCT taps.
- wired links and/or wired communications some embodiments of the present invention are not limited in this regard, and may include one or more wired or wireless links, may utilize one or more components of wireless communication, may utilize one or more methods or protocols of wireless communication, or the like. Some embodiments may utilize wired communication and/or wireless communication.
- the present invention may be implemented by using hardware units, software units, processors, CPUs, DSPs, integrated circuits, memory units, storage units, wireless communication modems or transmitters or receivers or transceivers, cellular transceivers, a power source, input units, output units, Operating System (OS), drivers, applications, and/or other suitable components.
- OS Operating System
- the present invention may be implemented by using code or program code or machine-readable instructions or machine-readable code, which is stored on a non-transitory storage medium or non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit), such that the program or code or instructions, when executed by a processor or a machine or a computer, cause such device to perform a method in accordance with the present invention.
- a non-transitory storage medium or non-transitory storage article e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit
- Embodiments of the present invention may be utilized with a variety of devices or systems having a touch-screen or a touch-sensitive surface; for example, a smartphone, a cellular phone, a mobile phone, a smart-watch, a tablet, a handheld device, a portable electronic device, a portable gaming device, a portable audio/video player, an Augmented Reality (AR) device or headset or gear, a Virtual Reality (VR) device or headset or gear, a “kiosk” type device, a vending machine, an Automatic Teller Machine (ATM), a laptop computer, a desktop computer, a vehicular computer, a vehicular dashboard, a vehicular touch-screen, or the like.
- ATM Automatic Teller Machine
- the system(s) and/or device(s) of the present invention may optionally comprise, or may be implemented by utilizing suitable hardware components and/or software components; for example, processors, processor cores, Central Processing Units (CPUs), Digital Signal Processors (DSPs), circuits, Integrated Circuits (ICs), controllers, memory units, registers, accumulators, storage units, input units (e.g., touch-screen, keyboard, keypad, stylus, mouse, touchpad, joystick, trackball, microphones), output units (e.g., screen, touch-screen, monitor, display unit, audio speakers), microphone(s) and/or sensor(s), wired or wireless modems or transceivers or transmitters or receivers, GPS receiver or GPS element or other location-based or location-determining unit or system, network elements (e.g., routers, switches, hubs, antennas), and/or other suitable components and/or modules.
- suitable hardware components and/or software components for example, processors, processor cores, Central Processing Units (CPU
- system(s) and/or devices of the present invention may optionally be implemented by utilizing co-located components, remote components or modules, “cloud computing” servers or devices or storage, client/server architecture, peer-to-peer architecture, distributed architecture, and/or other suitable architectures or system topologies or network topologies.
- calculations, operations and/or determinations may be performed locally within a single device, or may be performed by or across multiple devices, or may be performed partially locally and partially remotely (e.g., at a remote server) by optionally utilizing a communication channel to exchange raw data and/or processed data and/or processing results.
- Some embodiments may be implemented by using a special-purpose machine or a specific-purpose device that is not a generic computer, or by using a non-generic computer or a non-general computer or machine.
- Such system or device may utilize or may comprise one or more components or units or modules that are not part of a “generic computer” and that are not part of a “general purpose computer”, for example, cellular transceivers, cellular transmitter, cellular receiver, GPS unit, location-determining unit, accelerometer(s), gyroscope(s), device-orientation detectors or sensors, device-positioning detectors or sensors, or the like.
- Some embodiments may be implemented as, or by utilizing, an automated method or automated process, or a machine-implemented method or process, or as a semi-automated or partially-automated method or process, or as a set of steps or operations which may be executed or performed by a computer or machine or system or other device.
- Some embodiments may be implemented by using code or program code or machine-readable instructions or machine-readable code, which may be stored on a non-transitory storage medium or non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit), such that the program or code or instructions, when executed by a processor or a machine or a computer, cause such processor or machine or computer to perform a method or process as described herein.
- a non-transitory storage medium or non-transitory storage article e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit
- Such code or instructions may be or may comprise, for example, one or more of: software, a software module, an application, a program, a subroutine, instructions, an instruction set, computing code, words, values, symbols, strings, variables, source code, compiled code, interpreted code, executable code, static code, dynamic code; including (but not limited to) code or instructions in high-level programming language, low-level programming language, object-oriented programming language, visual programming language, compiled programming language, interpreted programming language, C, C++, C#, Java, JavaScript, SQL, Ruby on Rails, Go, Cobol, Fortran, ActionScript, AJAX, XML, JSON, Lisp, Eiffel, Verilog, Hardware Description Language (HDL, BASIC, Visual BASIC, Matlab, Pascal, HTML, HTML5, CSS, Perl, Python, PHP, machine language, machine code, assembly language, or the like.
- HDL BASIC, Visual BASIC, Matlab, Pascal, HTML, HTML5, CSS, Perl, Python, PHP, machine language, machine code,
- Discussions herein utilizing terms such as, for example, “processing”, “computing”, “calculating”, “determining”, “establishing”, “analyzing”, “checking”, “detecting”, “measuring”, or the like, may refer to operation(s) and/or process(es) of a processor, a computer, a computing platform, a computing system, or other electronic device or computing device, that may automatically and/or autonomously manipulate and/or transform data represented as physical (e.g., electronic) quantities within registers and/or accumulators and/or memory units and/or storage units into other data or that may perform other suitable operations.
- Some embodiments of the present invention may perform steps or operations such as, for example, “determining”, “identifying”, “comparing”, “checking”, “querying”, “searching”, “matching”, and/or “analyzing”, by utilizing, for example: a pre-defined threshold value to which one or more parameter values may be compared; a comparison between (i) sensed or measured or calculated value(s), and (ii) pre-defined or dynamically-generated threshold value(s) and/or range values and/or upper limit value and/or lower limit value and/or maximum value and/or minimum value; a comparison or matching between sensed or measured or calculated data, and one or more values as stored in a look-up table or a legend table or a list of reference value(s) or a database of reference values or ranges; a comparison or matching or searching process which searches for matches and/or identical results and/or similar results and/or sufficiently-close results, among multiple values or limits that are stored in a database or look-up table; utilization of one or more equations, formula, weighted
- plural and “a plurality”, as used herein, include, for example, “multiple” or “two or more”.
- “a plurality of items” includes two or more items.
- references to “one embodiment”, “an embodiment”, “demonstrative embodiment”, “various embodiments”, “some embodiments”, and/or similar terms, may indicate that the embodiment(s) so described may optionally include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Repeated use of the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may. Repeated use of the phrase “in some embodiments” does not necessarily refer to the same set or group of embodiments, although it may.
- Some embodiments may comprise, or may be implemented by using, an “app” or application which may be downloaded or obtained from an “app store” or “applications store”, for free or for a fee, or which may be pre-installed on a computing device or electronic device, or which may be transported to and/or installed on such computing device or electronic device.
- the present invention may comprise any possible combinations, re-arrangements, assembly, re-assembly, or other utilization of some or all of the modules or functions or components that are described herein, even if they are discussed in different locations or different chapters of the above discussion, or even if they are shown across different drawings or multiple drawings.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- Discrete Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A wireless video transmission system includes: a coarse compression module to compress a video frame and to generate coarse data of the video frame; a coarse decompression module to generate coarse frame information from the coarse data; a distortions extractor to generate coarse distortions from the coarse frame and from the video frame; a refinement data encoder to generate refinement data from coarse distortions based on the coarse frame information; and a data combining and modulation module to combine and transmit the coarse data and the refinement data.
Description
- This patent application claims priority and benefit: (I) from U.S. 62/672,889, filed on May 17, 2018; and (II) from U.S. 62/675,778, filed on May 24, 2018; and (III) from U.S. 62/777,159, filed on Dec. 9, 2018; and (IV) from U.S. 62/800,504, filed on Feb. 3, 2019; all of which are hereby incorporated by reference in their entirety.
- The present invention relates to video communications generally and more particularly to a system for hierarchal video transmission and reception.
- In some applications such as, for example, home AV (Audio-Video) systems, two-piece TV, VR (Virtual Reality), AR (Augmented Reality) and MR (Mixed Reality), there may be a need to wirelessly transmit raw video. In many cases, the video may be generated in an uncompressed manner (e.g. by a GPU overlaying graphics over movies) such that the input to the wireless system at one end may consist of pixels and the output of the system at the other end may be a reproduction of the input pixels.
- These wireless systems should generally provide for relatively high quality where the output pixels are as similar as possible to the input pixels. The systems should additionally provide for relatively low latency where the total delay of the transmission is low and for relatively high robustness to temporal degradations in the capacity of the wireless channel. They are generally configured to include a video encoder and a wireless transmitter. The video encoder may be based on a known compression standard such as, for example, H.264 or H.265. An advantage of using compression schemes based on known compression standards is that there is a relatively large offering of commercially available designs which may be purchased as hardware and/or software and may be integrated into new or existing systems. This may facilitate the development of “standard-compression” based systems. The wireless transmitter may additionally include a modem which may be based on a known communication standard such as, for example, Wi-Fi.
- In the wireless systems, tradeoffs are generally required between quality, latency, and robustness. To improve the tradeoff between quality, latency and robustness, methods of Joint-Source-Channel-Coding (JSCC) where introduced. Consequently, the systems may utilize the wireless spectrum efficiently.
- There is provided, in accordance with an embodiment of the present invention, a wireless video transmission system which may include a coarse compression module which may compress a video frame and to generate coarse data of the video frame, a coarse decompression module which may generate coarse frame information from the coarse data, distortions extractor which may generate coarse distortions from the coarse frame information and from the video frame, a refinement data encoder which may generate refinement data from the coarse frame distortions based on information derived from the coarse frame information, and a data combining and modulation module which may combine and transmit the coarse data and the refinement data.
- In some embodiments, the refinement data encoder may generate quantized transform coefficients from the coarse data. Optionally, the quantized transform coefficients may include quantized DCT coefficients. The refinement data encoder may additionally detect block energy in the quantized transform coefficients and may additionally estimate a probability density of the quantization error in the quantized transform coefficients.
- In some embodiments, the refinement data encoder may generate and reorder transform coefficients from the coarse distortions. The refinement data encoder may additionally apply a gain to the reordered transform coefficients, wherein the gain is derived from the coarse data. Optionally, the reordered transform coefficients may include quantized DCT coefficients.
- In some embodiments, the refinement data encoder may entropy encode the reordered transfer coefficients. Optionally, the entropy coding codebook used for the entropy encoding may be based on the probability density of the quantization error. The refinement data encoder may additionally compand the reordered transfer coefficients. The refinement data encoder may additionally perform a linear transform on the reordered transfer coefficients.
- In some embodiments, the coarse compression module may divide the video frame into a plurality of blocks.
- There is provided, in accordance with an embodiment of the present invention, a method of wireless video transmission which may include compressing a video frame to generate coarse data of the video frame, decompressing the coarse data to generate coarse frame information from the coarse data, generating coarse distortions from the coarse frame information and from the video frame, generating refinement data from the coarse distortions based on information derived from the coarse frame information, and combining and transmitting the coarse data and the refinement data.
- In some embodiments, the method may include generating quantized transform coefficients from the coarse data. Optionally, the quantized transform coefficients may include quantized DCT coefficients. The method may additionally include detecting block energy in the quantized transform coefficients. The method may additionally include estimating a probability density in the quantized transform coefficients.
- In some embodiments, the method may include generating and reordering transform coefficients from the coarse distortions. The method may additionally include applying a gain to the reordered transfer coefficients. Optionally, the reordered transform coefficients may include quantized DCT coefficients. The method may additionally include entropy encoding the reordered transfer coefficients. The method may additionally include companding the reordered transfer coefficients. The method may additionally include performing a linear transform on the reordered transfer coefficients.
- In some embodiments, the method may include dividing the video frame into a plurality of blocks.
- The subject matter regarded as the invention is particularly pointed out and distinctly claimed in the concluding portion of the specification. The invention, however, both as to organization and method of operation, together with objects, features, and advantages thereof, may best be understood by reference to the following detailed description when read with the accompanying drawings in which:
-
FIG. 1 schematically illustrates an exemplary two-layer video compression and transmission system for transmitting video over a communication channel, according to an embodiment of the present invention; -
FIG. 2 schematically illustrates an exemplary system for reception and reconstruction of video from a signal received over a communication channel, according to an embodiment of the present invention; -
FIG. 3 schematically illustrates an exemplary refinement coding module, which may be used in the video compression and transmission system ofFIG. 1 , according to an embodiment of the present invention; -
FIG. 4 schematically illustrates an exemplary refinement decoding module, which may be used in the system for reception and reconstruction of video ofFIG. 2 , according to an embodiment of the present invention; -
FIG. 5A schematically illustrates an exemplary refinement coding module, which may be used in the video compression and transmission system ofFIG. 1 , according to an embodiment of the present invention; -
FIG. 5B schematically illustrates three exemplary graphs denoted (a), (b) and (c), representing probability density functions of three DCT taps, respectively, according to an embodiment of the present invention; -
FIG. 6 schematically illustrates an exemplary refinement decoding module, which may be used in the system for reception and reconstruction of video ofFIG. 2 , according to an embodiment of the present invention; -
FIG. 7 schematically illustrates an exemplary DCT taps processor, which may be used with the refinement coding module ofFIG. 5 , according to an embodiment of the present invention; and -
FIG. 8 schematically illustrates several different types of compander functions, which may be used with the DCT taps processor ofFIG. 7 , according to an embodiment of the present invention. - It will be appreciated that for simplicity and clarity of illustration, elements shown in the figures have not necessarily been drawn to scale. For example, the dimensions of some of the elements may be exaggerated relative to other elements for clarity. Further, where considered appropriate, reference numerals may be repeated among the figures to indicate corresponding or analogous elements.
- In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the invention. However, it will be understood by those skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the present invention.
- Applicants have realized that wireless video transmission systems which combine use of JSCC with known compression block techniques including compression standards such as, for example, H.265, may suffer from bandwidth overhead. For example, consider a system which may combine H.265 data with “analog” refinement data. The system may employ “coarse” bins and “fine” bins where the coarse bins are used for sending H.265 data (including interframe prediction) and the fine bins are used for sending the data for memoryless refinement of the received image. The analog refinement may have video-content-dependent power, and may suffer from additive noise and therefore, it may be desired to apply a video-content-dependent gain to bring the transmitted power to the highest level available by the hardware capabilities and the radio regulation. A drawback using content-dependent gain is that the gain should be conveyed to the receiver, and the message which describes that gain, creates a bandwidth overhead. Video-content-dependent gain may be applied to single blocks (that is, each block will have a different gain), or to group of blocks (that is, each group will have different gain, but all the blocks in the group will have the same gain).
- Applicants have realized that bandwidth overhead in video transmission systems combining use of known compression block techniques (e.g. H.264 or H.265) with JSCC may be reduced, optionally eliminated, by utilizing information associated with the coarse data to generate control data for the refinement encoder, where the refinement encoder generates refinement data which may be transmitted to a video receiver for memoryless refinement of a video image. The transmission side may include a refinement encoder which may be adapted to perform a two-dimensional DCT transform on coarse distortions which may be in the form of error frames (in the pixel domain) or in the form of quantization errors of DCT taps generated from the coarse data to generate DCT taps for each block. The refinement encoder may additionally calculate the block energy to apply a gain to the coarse distortions DCT taps based on the power of the coarse DCT taps. Alternatively, the refinement encoder may estimate a probability density for the coarse DCT taps which may then be used to entropy encode the coarse distortions DCT taps. A compander may be used to apply non-linear gain to the DCT taps of the refinement encoder.
- The receiving side may include a refinement decoder which may decode the received refinement data based on the received coarse DCT taps. Similarly to the refinement encoder in the transmission side, the refinement decoder may calculate the block energy to apply a gain to the received refinement DCT taps based on the power of the coarse DCT taps. Alternatively, the refinement decoder may estimate a probability density for the coarse DCT taps which may then be used to entropy decode the received refinement DCT taps. As the received coarse data is substantially the same as the transmitted coarse data except for a small bit error rate associated with the packet error rate, the same gain applied by the refinement encoder may be applied by the refinement decoder, thereby eliminating the need to send the gain information to the receiver and reducing bandwidth overhead. Alternatively, the same probability density function may be used for the entropy decoding, thereby eliminating the need to send the probability density function information to the receiver and reducing bandwidth overhead.
- Reference is made to
FIG. 1 , which schematically illustrates an exemplary two-layer video compression andtransmission system 100 for transmitting video over acommunication channel 150, according to an embodiment of the present invention.System 100 may include acoarse compression module 102, acoarse decompression module 104, a coarsedistortions extractor module 106, arefinement coding module 108, a data combining andmodulation module 110. -
Coarse compression module 102 may receive pixels of anoriginal video frame 10 and may generate a compressed description of the video which may be denoted ascoarse data 12.Coarse compression module 102 may use inter-frame prediction in performing the data compression to generatecoarse data 12. Inter frame prediction may include use of a forward inter-frame predictor to generate P-frames and optionally also a bi-directional inter-frame predictor to generate B-frames. In order to reduce latency,coarse compression module 102 may wholly or partially avoid generating B-frames (bidirectionally predicted frames).Coarse decompression module 104 may parse and decode thecoarse data 12 to generate a coarse frame 14. - Coarse
distortions extractor module 106 may receiveoriginal frame 10 and may apply coarse frame information 14 to generatecoarse distortions 16. To generate the coarse distortions,coarse distortions extractor 106 may perform pixel subtraction optionally followed by addition of a fixed offset and optional limiting to generate an error frame. For example, a first pixel in a first line inoriginal frame 10 may have RGB values of 98, 61 and 80, and in coarse frame 14 the same pixel may have the values of 100, 60 and 80. The coarsedistortions extractor module 106 may add an offset of 128 to the difference, and may additionally limit the result from the bottom by 0 and from the top by 255. As a result, the first pixel in the first line inerror frame 16 may be min (255, max (0, 128+98−100,128+61−60,128+80−80))=126,129,128. - Alternatively, the coarse distractions extractor may operate in the DCT domain, and subtract the DCT taps of the original frame from the DCT taps of the Coarse frame to generate quantization errors of the DCT taps.
- Refinement
coding block module 108 may compresscoarse distortions 16 and may generate a refinement data description (refinement data 18). Refinementcoding block module 108 may adapt its coding parameters according to the content of the coarse data. For example, it may perform two dimensional DCT and may send only a subset of the DCT taps, for example, the first N taps. It may additionally select the value of N based on the coarse description of that block according to a pre-defined algorithm for selecting N. Since the coarse data is transmitted as well, the receiver may select the same value of N for decoding the refinement data, according to a pre-defined algorithm for selecting N, which is equivalent to the pre-defined algorithm in the encoder. This may eliminate the need to specifically encode and send the value of N. - Data combining and
modulation module 110 may combinecoarse data 12 andrefinement data 18, and transmit them over acommunication channel 150. Data combining andmodulation module 110 may modulate the signal using OFDM, with a portion of the bins (“digital bins”) allocated for the coarse data and a portion of the bins (“fine bins”) allocated to the refinement data. Alternatively, a different modulation and coding scheme may be used for the digital bins and the fine bins, and the fine bins may include properties similar to analog transmission. Alternatively, the same modulation and modulation scheme may be used for the coarse data and the fine data. - Reference is made to
FIG. 2 , which schematically illustrates anexemplary system 200 for reception and reconstruction of video from a signal received over acommunication channel 250, according to an embodiment of the present invention.System 200 may include a demodulation anddata splitting module 202, acoarse decompression module 204, arefinement decoding module 206, and aframe combiner module 208.System 200 may be used to receive a video signal transmitted bytransmission system 100 overchannel 150, in whichcase channel 150 andchannel 250 is a same channel. - Demodulation and
data splitter module 202 may demodulate the received signal and may split the signal intocoarse data 20 and receivedrefinement data 22.Coarse data 20 may be substantially identical tocoarse data 12 sent bytransmission system 100, and may include a small bit error rate compared tocoarse data 12 associated with, for example, packet errors. -
Coarse decompression module 204 may decode the video and may generatecoarse frame information 24.Coarse decompression module 204 may optionally be a standard video decoder such as, for example, a MPEG decoder, a H.264 decoder, or a H.265 decoder, among other standard video decoders. -
Refinement decoding module 206 may receiverefinement data 22 and may generate reconstructedcoarse distortions 26.Refinement decoding module 206 may adapt its decoding parameters according to the content ofcoarse data 20. For example, it may decode fromcoarse data 20 the value N of the number of transmitted DCT taps and may then reconstruct the DCT values by zeroing all the taps starting from tap N+1. -
Frame combiner module 208 may combine thecoarse frame information 24 with reconstructedcoarse data 26, for example by summing the two frames together pixel by pixel, optionally reducing a fixed offset (e.g. 128 for 8 bits pixels) and optionally limit the result between two values, for example, 0 and 255.Frame combiner module 28 may then generate a reconstructedframe 28. Alternatively, frame combiner module may combine thecoarse frame information 24 with the reconstructed coarse data which may include the quantization errors of the DCT taps to generate the reconstructedframe 28. - Reference is made to
FIG. 3 which schematically illustrates an exemplary implementation ofrefinement coding module 108, which may be used in video compression andtransmission system 100, according to an embodiment of the present invention.Refinement coding module 108 may include a quantized DCT tapsextractor 302, a blockenergy detector module 304, an optional refinement DCT tapsgenerator module 306, and a DCT tapsprocessor 308. - In an exemplary mode of operation,
original frame 10 may be divided into blocks, for example, block of 8×8 pixel or 4×4 pixels or 16×16 pixel blocks.Coarse compression module 102 may perform a certain transform, for example a two-dimensional DCT, on each block.Coarse compression module 102 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps. - The value of quantized DCT tap number j of color c of block number i may be denoted DCTQi,c,j.
- DCT
tap extractor module 302 may extract fromcoarse data 12 the quantized DCT taps and may transfer this data as quantized DCT taps 13 to blockenergy detector module 304. Blockenergy detector module 304 may determine, for each block, a certain gain, represented by refinement block gain 15, which may be applied by DCT tapsprocessor 308 to reordered DCT taps 17 to generaterefinement data 18. - Refinement DCT taps
Generator module 306 may perform a certain transform, for example a two-dimensional DCT, on each block of error frame incoarse distortions 16 if in the pixel domain DCT &reordering module 306 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps to generate reordered refinement DC taps 17. Alternatively, if coarse distortions include quantization errors of the DCT taps, DCT tapsprocessor 308 may generaterefinement data 18 which may be a linear transform of the reordered DC taps 17 following the processor applying a gain to the DCT taps or to the linear transform of the DCT taps based on the refinement block gain determined by blockenergy detector module 304. - In some embodiments, an exemplary mode of operation of block
energy detector module 304 may be as follows: - a.
Block energy detector 304 may first calculate for each block i the block energy Ei by, for example: -
E i=Σj=0 M−1Σc=1 3DCTQi,c,j 2 - wherein M is the number of DCT taps per block, and it is assumed that the 2D DCT is performed separately over each of the 3 colors.
- b. Block
energy detector module 304 may then use a certain function, for example, -
G i =f(E i) - to generate refinement block gain Gi.
An example for such a function may be given by: -
- where the following are constants: Gmin, Gmax, Eref.
Another example of the function may be given by: -
- for some threshold T.
- c. Alternatively or additionally, the frame may be divided into superblocks, with N blocks per superblock (e.g., N=240), and an additional refinement block gain G′i may be applied, where the additional refinement block gain is substantially identical for all the blocks in the superblock.
- In some embodiments, another exemplary mode of operation of block
energy detector module 304 may be as follows: - a. Block
energy detector module 304 may first calculate for each superblock, the superblock energy, denoted Esuperblock, by: -
E superblock=Σi=1 NΣj=0 M−1Σc=1 3DCTQi,c,j 2 - where M is the number of DCT taps per block, and it is assumed that the DCT is performed over 3 colors.
- b. Block
energy detector module 304 may then use a function, such as: -
G i ′=f(E superblock) - to generate the refinement block gain G′i.
An example of such a function may be given by: -
- where the following are constants: G′min, G′max, E′ref.
- Reference is made to
FIG. 4 , which schematically illustrates an exemplary implementation ofrefinement decoding module 206, which may be used insystem 200 for reception and reconstruction of video, according to an embodiment of the present invention.Refinement decoding module 206 may include a quantized DCT tapsextractor module 402, a blockenergy detector module 404, a DCT tapsextractor 406, and an optional reordering &IDCT module 408. - In some embodiments, as previously discussed with reference to
FIG. 2 ,coarse data 20 may be substantially identical tocoarse data 12 sent bytransmission system 100, and may include a small bit error rate compared tocoarse data 12 associated with the packet error rate. As such, quantized DCT tapsextractor module 402 and blockenergy detector module 404 inrefinement decoding module 206 may be substantially identical to quantized DCT tapsextractor module 302 and blockenergy detector module 304 in refinement coding module 300 inFIG. 3 . Additionally,refinement block gain 25 calculated by blockenergy detector module 404 may be substantially identical to refinement block gain 15 calculated by blockenergy detector module 304 ofFIG. 3 . It may be appreciated by the skilled person that since the quantized DCT taps extractor module and the energy block detector module are substantially identical to that intransmission system 100, and since the values of DCTQ,i,c,j will be made available torefinement decoding module 206 inreceiver system 200, there is no need to send the value of Gi and of G′i overchannel 150, thereby allowing for a reduction (saving) in bandwidth. That is, the values Gi and G′i, which were calculated in refinement coding module 300 inFIG. 3 , may be regenerated inrefinement decoding module 206, and there is no need to send them. - DCT taps
extractor 406 may extract the DCT taps from the receivedrefinement data 22 and may generate received reordered refinement DCT taps 27 from receivedrefinement data 22 based onrefinement block gain 25 computed by blockenergy detector module 404. Optional reordering &IDCT module 408 may reorder received reordered DCT taps 27 and may perform an inverse transform on the DCT taps to generate reconstructedcoarse distortions 26 in the form of error frames if received reordered refinement DCT taps 27 is in the pixel domain. If received reordered refinement DCT taps 27 includes quantization errors of the DCT taps, reordering &IDCT module 408 may be obviated. - Reference is now made to
FIG. 5A , which schematically illustrates an exemplaryrefinement coding module 500 which may be used in video compression andtransmission system 100, according to an embodiment of the present invention.Refinement coding module 500 may include a quantized DCT tapsextractor 502, a refinement probabilityfunction estimator module 504, an optional refinement DCT tapsgenerator module 506, and a DCT tapsprocessor 508. - In an exemplary mode of operation,
original frame 10 may be divided into blocks; for example, block of 8×8 pixels, or 4×4 pixel or 16×16 pixel blocks.Coarse compression module 102 may perform a certain transform, for example a two-dimensional DCT, on each block.Coarse compression module 102 may additionally reorder the DCT taps according to a certain predefined order, for example in a. zig-zag order, and may additionally quantize the DCT taps. The DCT taps may be denoted as a random variable x and its quantized value may be denoted as Q(x). - Quantized DCT taps
extractor module 502 and DCT &reordering module 506 may be substantially similar to quantized DCT tapsextractor module 302 and refinement DCT tapsmodule 306, respectively, inFIG. 3 . Refinement probability densityfunction estimator module 504 may estimate, for eachQuantized DCT tap 13, the probability density function of the corresponding DCT tap of the error frame. DCT tapsprocessor 508 may use refinement probability density function perDCT tap 30 to process reordered refinement DCT taps 17 and generaterefinement data 18. - In some embodiments, DCT taps
processor 508 may further quantize reordered DCT taps 17 using a quantizer which may include quantization cell size smaller than that in the quantizer used bycoarse compression module 102. Following quantization, DCT tapsprocessor 508 may use entropy encoding to compress the index of the quantization cell and generate a string of bits which may indicate the index of the fine quantization cell and where the string of bits may be optimized to reduce the average number of bits in the stream. - Entropy encoders are generally optimized for a certain probability density of the source. To yield a good code book, the probability density of the source is estimated. However, as shown in
FIG. 5B , the conditional probability density function of a certain DCT tap may vary and may depend on its quantized value. Refinement Probability densityfunction estimator module 504 may receive the quantized value at its input, and responsively, may estimate the conditional probability density function to generate refinement probability density function perDCT tap 30. DCT tapsprocessor 508 may then use refinement probability density function perDCT tap 30 to select the proper entropy code-book which is best fitted to the probability density function. - Reference is now made to
FIG. 5B , which schematically illustrates three exemplary graphs (a), (b) and (c) representingprobability density functions probability density function 550 of a certain DCT tap. Also shown in the graph are the quantization points and quantization cell boundaries of a uniform quantizer with quantization cell size of Δ and quantization points at 0, ±Δ, ±2Δ, ±3Δ, . . . . Graph (b)probability density function 552 of a certain DCT tap whose quantization point is 0. Graph (c) describes theprobability density function 554 of a certain DCT whose quantization point is +2Δ. It may be appreciated that -
f(x−Q(x)|Q(x)=0) - is very different from
-
f(x−Q(x)|Q(x)=2Δ) - and consequently, it may be preferable to use different codebooks to compress X in each of the cases.
- Reference is made to
FIG. 6 , which schematically illustrates an exemplaryrefinement decoding module 600, which may be used insystem 200 for reception and reconstruction of video, according to an embodiment of the present invention.Refinement decoding module 600 may include a quantized DCT tapsextractor module 602, a refinement probability densityfunction estimator module 604, a DCT tapsextractor 606, and an optional reordering & IDCT module 608. - In some embodiments, as previously discussed with reference to
FIG. 2 ,coarse data 20 may be substantially identical tocoarse data 12 sent bytransmission system 100, and may include a small bit error rate compared tocoarse data 12 associated with the packet error rate. As such, quantized DCT tapsextractor module 602 and refinement probability densityfunction estimator module 604 inrefinement decoding module 600 may be substantially identical to quantized DCT tapsextractor module 502 and refinement probability densityfunction estimator module 604 inrefinement coding module 500 inFIG. 5A . Additionally, refinement probability density function perDCT tap 32 determined by refinement probability densityfunction estimator module 604 may be substantially identical to refinement probability density function perDCT tap 30 determined by refinement probability densityfunction estimator module 504 ofFIG. 5A . It may be appreciated by the skilled person that, since refinement probability density function per DCT tap 37 estimated atrefinement decoding block 600 is substantially identical to refinement probability density function perDCT tap 30 estimated at therefinement encoding block 500 ofFIG. 5A , there is no need for the transmitter to send the refinement probability density function per DCT tap to the receiver, which may allow for a reduction in bandwidth. - DCT taps
extractor 606 may extract the DCT taps from the received refinement data and may generate received reordered refinement DCT taps 27 from receivedrefinement data 22 based on refinement probability density function per DCT tap 37 determined by refinement probability densityfunction estimator module 604. Optional reordering & IDCT module 608 may reorder received reordered DCT taps 27 and may perform an inverse transform on the DCT taps to generate reconstructedcoarse distortions 26 in the form of error frames if received reordered refinement DCT taps 27 is in the pixel domain. If received reordered refinement DCT taps 27 includes quantization errors of the DCT taps, reordering &IDCT module 408 may be obviated. - Reference is made to
FIG. 7 , which schematically illustrates an exemplary DCT tapsprocessor 700, which may be used withrefinement coding module 500 invideo transmission system 100, according to an embodiment of the present invention. DCT tapsprocessor 700 may include a compander selector perDCT tap module 702, acompander module 704, and alinear transform module 706. In some embodiments, DCT tapsprocessor 700 may be the same as DCT tapsprocessor 508 inFIG. 5 . -
Compander module 704 may generate companded DCT taps 34 and may include use of a non-liner function to apply a gain to reordered refinement DCT taps 17. In some embodiments, a non-linear function which may be used inCompander 704 may be, for example, as in the following Equation (1): -
C(x)=sign(x)·C 1·√{square root over (|x|)} - where C1 is a constant, X is the input, and C(x) is the output.
- Another non-linear function which may be used, may be as in the following Equation (2):
-
- for some constants G1 and G2 and T, and where X is the input, and C(x) is the output.
- In wireless systems the transmission power is usually limited both by RF (radio frequency) regulations and by the capabilities of the power amplifier. To optimize transmission range and receiver SNR (signal-to-noise ratio), it is desired to optimize the values of G1 and G2 and T in the above Equation (2), such that the signal may be transmitted with the highest gain, but without breaching RF regulations, or compressing and/or overloading the power amplifier. The optimal choice of the constants G1 and G2 and T generally depends on the probability density function of the input X. For example, if X is known to be zero-mean with low variance, then large values of G1 may be used (which may improve the receiver SNR). But if X is known to be zero-mean with high variance, then small values of G1 and G2 are generally used, to avoid breaching the RF regulations or compressing and/or overloading the power amplifier.
- Compander selector per DCT taps
module 702 receives, for each DCT tap, the refinement probability density as indicated by refinement probability density function perDCT tap 30, and uses it to select the type ofcompander 704 as indicated by compander type 33. That is, compander selector per DCT taps 702 may chose the values of G1 and G2 and T according to the refinement probability density.Linear transform module 706 may apply a linear transform to companded DCT taps 18 to generaterefinement data 18. - Reference is made to
FIG. 8 , which schematically illustrates several different types of compander functions 800, which may be used with DCT tapsprocessor 700, according to an embodiment of the present invention.Type 1compander function 802 may be suitable for high power DCT taps;Type 2compander 804 may be suitable for medium power DCT taps; andType 3compander 806 may be suitable for low power DCT taps. - Although portions of the discussion herein relate, for demonstrative purposes, to wired links and/or wired communications, some embodiments of the present invention are not limited in this regard, and may include one or more wired or wireless links, may utilize one or more components of wireless communication, may utilize one or more methods or protocols of wireless communication, or the like. Some embodiments may utilize wired communication and/or wireless communication.
- The present invention may be implemented by using hardware units, software units, processors, CPUs, DSPs, integrated circuits, memory units, storage units, wireless communication modems or transmitters or receivers or transceivers, cellular transceivers, a power source, input units, output units, Operating System (OS), drivers, applications, and/or other suitable components.
- The present invention may be implemented by using code or program code or machine-readable instructions or machine-readable code, which is stored on a non-transitory storage medium or non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit), such that the program or code or instructions, when executed by a processor or a machine or a computer, cause such device to perform a method in accordance with the present invention.
- Embodiments of the present invention may be utilized with a variety of devices or systems having a touch-screen or a touch-sensitive surface; for example, a smartphone, a cellular phone, a mobile phone, a smart-watch, a tablet, a handheld device, a portable electronic device, a portable gaming device, a portable audio/video player, an Augmented Reality (AR) device or headset or gear, a Virtual Reality (VR) device or headset or gear, a “kiosk” type device, a vending machine, an Automatic Teller Machine (ATM), a laptop computer, a desktop computer, a vehicular computer, a vehicular dashboard, a vehicular touch-screen, or the like.
- The system(s) and/or device(s) of the present invention may optionally comprise, or may be implemented by utilizing suitable hardware components and/or software components; for example, processors, processor cores, Central Processing Units (CPUs), Digital Signal Processors (DSPs), circuits, Integrated Circuits (ICs), controllers, memory units, registers, accumulators, storage units, input units (e.g., touch-screen, keyboard, keypad, stylus, mouse, touchpad, joystick, trackball, microphones), output units (e.g., screen, touch-screen, monitor, display unit, audio speakers), microphone(s) and/or sensor(s), wired or wireless modems or transceivers or transmitters or receivers, GPS receiver or GPS element or other location-based or location-determining unit or system, network elements (e.g., routers, switches, hubs, antennas), and/or other suitable components and/or modules.
- The system(s) and/or devices of the present invention may optionally be implemented by utilizing co-located components, remote components or modules, “cloud computing” servers or devices or storage, client/server architecture, peer-to-peer architecture, distributed architecture, and/or other suitable architectures or system topologies or network topologies.
- In accordance with embodiments of the present invention, calculations, operations and/or determinations may be performed locally within a single device, or may be performed by or across multiple devices, or may be performed partially locally and partially remotely (e.g., at a remote server) by optionally utilizing a communication channel to exchange raw data and/or processed data and/or processing results.
- Some embodiments may be implemented by using a special-purpose machine or a specific-purpose device that is not a generic computer, or by using a non-generic computer or a non-general computer or machine. Such system or device may utilize or may comprise one or more components or units or modules that are not part of a “generic computer” and that are not part of a “general purpose computer”, for example, cellular transceivers, cellular transmitter, cellular receiver, GPS unit, location-determining unit, accelerometer(s), gyroscope(s), device-orientation detectors or sensors, device-positioning detectors or sensors, or the like.
- Some embodiments may be implemented as, or by utilizing, an automated method or automated process, or a machine-implemented method or process, or as a semi-automated or partially-automated method or process, or as a set of steps or operations which may be executed or performed by a computer or machine or system or other device.
- Some embodiments may be implemented by using code or program code or machine-readable instructions or machine-readable code, which may be stored on a non-transitory storage medium or non-transitory storage article (e.g., a CD-ROM, a DVD-ROM, a physical memory unit, a physical storage unit), such that the program or code or instructions, when executed by a processor or a machine or a computer, cause such processor or machine or computer to perform a method or process as described herein. Such code or instructions may be or may comprise, for example, one or more of: software, a software module, an application, a program, a subroutine, instructions, an instruction set, computing code, words, values, symbols, strings, variables, source code, compiled code, interpreted code, executable code, static code, dynamic code; including (but not limited to) code or instructions in high-level programming language, low-level programming language, object-oriented programming language, visual programming language, compiled programming language, interpreted programming language, C, C++, C#, Java, JavaScript, SQL, Ruby on Rails, Go, Cobol, Fortran, ActionScript, AJAX, XML, JSON, Lisp, Eiffel, Verilog, Hardware Description Language (HDL, BASIC, Visual BASIC, Matlab, Pascal, HTML, HTML5, CSS, Perl, Python, PHP, machine language, machine code, assembly language, or the like.
- Discussions herein utilizing terms such as, for example, “processing”, “computing”, “calculating”, “determining”, “establishing”, “analyzing”, “checking”, “detecting”, “measuring”, or the like, may refer to operation(s) and/or process(es) of a processor, a computer, a computing platform, a computing system, or other electronic device or computing device, that may automatically and/or autonomously manipulate and/or transform data represented as physical (e.g., electronic) quantities within registers and/or accumulators and/or memory units and/or storage units into other data or that may perform other suitable operations.
- Some embodiments of the present invention may perform steps or operations such as, for example, “determining”, “identifying”, “comparing”, “checking”, “querying”, “searching”, “matching”, and/or “analyzing”, by utilizing, for example: a pre-defined threshold value to which one or more parameter values may be compared; a comparison between (i) sensed or measured or calculated value(s), and (ii) pre-defined or dynamically-generated threshold value(s) and/or range values and/or upper limit value and/or lower limit value and/or maximum value and/or minimum value; a comparison or matching between sensed or measured or calculated data, and one or more values as stored in a look-up table or a legend table or a list of reference value(s) or a database of reference values or ranges; a comparison or matching or searching process which searches for matches and/or identical results and/or similar results and/or sufficiently-close results, among multiple values or limits that are stored in a database or look-up table; utilization of one or more equations, formula, weighted formula, and/or other calculation in order to determine similarity or a match between or among parameters or values; utilization of comparator units, lookup tables, threshold values, conditions, conditioning logic, Boolean operator(s) and/or other suitable components and/or operations.
- The terms “plurality” and “a plurality”, as used herein, include, for example, “multiple” or “two or more”. For example, “a plurality of items” includes two or more items.
- References to “one embodiment”, “an embodiment”, “demonstrative embodiment”, “various embodiments”, “some embodiments”, and/or similar terms, may indicate that the embodiment(s) so described may optionally include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Repeated use of the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may. Repeated use of the phrase “in some embodiments” does not necessarily refer to the same set or group of embodiments, although it may.
- As used herein, and unless otherwise specified, the utilization of ordinal adjectives such as “first”, “second”, “third”, “fourth”, and so forth, to describe an item or an object, merely indicates that different instances of such like items or objects are being referred to; and does not intend to imply as if the items or objects so described must be in a particular given sequence, either temporally, spatially, in ranking, or in any other ordering manner.
- Some embodiments may comprise, or may be implemented by using, an “app” or application which may be downloaded or obtained from an “app store” or “applications store”, for free or for a fee, or which may be pre-installed on a computing device or electronic device, or which may be transported to and/or installed on such computing device or electronic device.
- Functions, operations, components and/or features described herein with reference to one or more embodiments of the present invention, may be combined with, or may be utilized in combination with, one or more other functions, operations, components and/or features described herein with reference to one or more other embodiments of the present invention. The present invention may comprise any possible combinations, re-arrangements, assembly, re-assembly, or other utilization of some or all of the modules or functions or components that are described herein, even if they are discussed in different locations or different chapters of the above discussion, or even if they are shown across different drawings or multiple drawings.
- While certain features of the present invention have been illustrated and described herein, many modifications, substitutions, changes, and equivalents may occur to those skilled in the art. Accordingly, the claims are intended to cover all such modifications, substitutions, changes, and equivalents.
Claims (25)
1. A wireless video transmission system comprising:
a coarse compression module to compress a video frame and to generate coarse data of said video frame;
a coarse decompression module to generate a coarse frame information from said coarse data;
a distortions extractor to generate coarse distortions from said coarse frame information and from said video frame;
a refinement data encoder to generate refinement data from said coarse distortions based on said coarse frame information; and
a data combining and modulation module to combine and transmit said coarse data and said refinement data.
2. The system according to claim 1 ,
wherein said refinement data encoder is adapted to generate quantized transform coefficients from said coarse data.
3. The system according to claim 2 ,
wherein said quantized transform coefficients comprise quantized DCT coefficients.
4. The system according to claim 3 ,
wherein said refinement data encoder is further adapted to detect block energy in said quantized transform coefficients.
5. The system according to claim 4 ,
wherein said refinement data encoder is further adapted to estimate a probability density of the quantization error in said quantized transform coefficients.
6. The system according to claim 5 ,
wherein said refinement data encoder is adapted to generate and reorder transform coefficients from said coarse distortions.
7. The system according to claim 6 ,
wherein said refinement data encoder is further adapted to apply a gain to said reordered transform coefficients, wherein said gain is derived from said coarse data.
8. The system according to claim 7 ,
wherein said reordered transform coefficients comprise quantized DCT coefficients.
9. The system according to claim 8 ,
wherein said refinement data encoder is further adapted to entropy encode said reordered transfer coefficients.
10. The system according to claim 9 ,
wherein an entropy coding codebook used for said entropy encoding is based on said probability density of the quantization error.
11. The system according to claim 10 ,
wherein said refinement data encoder is further adapted to compand said reordered transfer coefficients.
12. The system according to claim 11 ,
wherein said refinement data encoder is further adapted to perform a linear transform on said reordered transfer coefficients.
13. The system according to claim 12 ,
wherein said coarse compression module divides said video frame into a plurality of blocks.
14. A method of wireless video transmission, the method comprising:
compressing a video frame to generate coarse data of said video frame;
decompressing said coarse data to generate coarse frame information from said coarse data;
generating coarse distortions from said coarse frame information and from said video frame;
generating refinement data from said coarse distortions based on information derived from said coarse data; and
combining and transmitting said coarse data and said refinement data.
15. The method according to claim 14 , further comprising:
generating quantized transform coefficients from said coarse data.
16. The method according to claim 15 , wherein said quantized transform coefficients comprise quantized DCT coefficients.
17. The method according to claim 16 , further comprising:
detecting block energy in said quantized transform coefficients.
18. The method according to claim 17 , further comprising:
estimating a probability density in said quantized transform coefficients.
19. The method according to claim 18 , further comprising:
generating and reordering transform coefficients from said coarse distortions.
20. The method according to claim 19 , further comprising:
applying a gain to said reordered transfer coefficients.
21. The method according to claim 20 ,
wherein said reordered transform coefficients comprise quantized DCT coefficients.
22. The method according to claim 21 , further comprising:
entropy encoding said reordered transfer coefficients.
23. The method according to claim 22 , further comprising:
companding said reordered transfer coefficients.
24. The method according to claim 23 , further comprising:
performing a linear transform on said reordered transfer coefficients.
25. The method according to claim 24 , further comprising:
dividing said video frame into a plurality of blocks.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US17/051,531 US20210368177A1 (en) | 2018-05-17 | 2019-05-14 | Joint Source Channel Coding with H.265 Video Compression |
Applications Claiming Priority (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201862672889P | 2018-05-17 | 2018-05-17 | |
US201862675778P | 2018-05-24 | 2018-05-24 | |
US201862777159P | 2018-12-09 | 2018-12-09 | |
US201962800504P | 2019-02-03 | 2019-02-03 | |
US17/051,531 US20210368177A1 (en) | 2018-05-17 | 2019-05-14 | Joint Source Channel Coding with H.265 Video Compression |
PCT/IL2019/050541 WO2019220433A1 (en) | 2018-05-17 | 2019-05-14 | Joint source channel coding with h.265 video compression |
Publications (1)
Publication Number | Publication Date |
---|---|
US20210368177A1 true US20210368177A1 (en) | 2021-11-25 |
Family
ID=68540894
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/051,531 Abandoned US20210368177A1 (en) | 2018-05-17 | 2019-05-14 | Joint Source Channel Coding with H.265 Video Compression |
US17/051,210 Active US11323713B2 (en) | 2018-05-17 | 2019-05-14 | Bit rate reduction for scalable video coding |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/051,210 Active US11323713B2 (en) | 2018-05-17 | 2019-05-14 | Bit rate reduction for scalable video coding |
Country Status (2)
Country | Link |
---|---|
US (2) | US20210368177A1 (en) |
WO (2) | WO2019220433A1 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
GB2598701B (en) | 2020-05-25 | 2023-01-25 | V Nova Int Ltd | Wireless data communication system and method |
Family Cites Families (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
EP0443114A3 (en) * | 1990-02-23 | 1993-12-15 | Ibm | Methods and apparatus for image-data decompression |
EP0699003A4 (en) * | 1994-02-15 | 1999-10-06 | Sony Corp | Method and device for encoding image signal |
US6292512B1 (en) | 1998-07-06 | 2001-09-18 | U.S. Philips Corporation | Scalable video coding system |
US6819803B2 (en) | 2001-07-02 | 2004-11-16 | International Business Machines Corporation | Faster lossless rotation of JPEG images |
US7349583B2 (en) * | 2003-09-05 | 2008-03-25 | The Regents Of The University Of California | Global motion estimation image coding and processing |
JP2007507147A (en) * | 2003-09-25 | 2007-03-22 | アミモン リミテッド | Wireless transmission of high-quality video |
US20060078049A1 (en) * | 2004-10-13 | 2006-04-13 | Nokia Corporation | Method and system for entropy coding/decoding of a video bit stream for fine granularity scalability |
US8848787B2 (en) * | 2007-10-15 | 2014-09-30 | Qualcomm Incorporated | Enhancement layer coding for scalable video coding |
WO2018025211A1 (en) * | 2016-08-03 | 2018-02-08 | Amimon Ltd. | Successive refinement video compression |
-
2019
- 2019-05-14 US US17/051,531 patent/US20210368177A1/en not_active Abandoned
- 2019-05-14 US US17/051,210 patent/US11323713B2/en active Active
- 2019-05-14 WO PCT/IL2019/050541 patent/WO2019220433A1/en active Application Filing
- 2019-05-14 WO PCT/IL2019/050540 patent/WO2019220432A1/en active Application Filing
Also Published As
Publication number | Publication date |
---|---|
US11323713B2 (en) | 2022-05-03 |
WO2019220433A1 (en) | 2019-11-21 |
US20210377536A1 (en) | 2021-12-02 |
WO2019220432A1 (en) | 2019-11-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11727548B2 (en) | Method and apparatus for encoding and decoding HDR images | |
JP2661985B2 (en) | Digital video signal encoding device and corresponding decoding device | |
US20120327997A1 (en) | Video Coding | |
JPH04323989A (en) | Encoding device for high definition television | |
US20120328002A1 (en) | Video Coding | |
US20160212427A1 (en) | Video encoding apparatus and video encoding method | |
US20170064328A1 (en) | Video encoding method and apparatus and video decoding method and apparatus | |
US9681140B2 (en) | Device and a method for encoding an image and corresponding decoding method and decoding device | |
US10869029B2 (en) | Hybrid digital-analog coding | |
US20210368177A1 (en) | Joint Source Channel Coding with H.265 Video Compression | |
KR20210135247A (en) | In-Loop Reshaping Adaptive Reshaping Machine Orientation | |
EP3119088A1 (en) | Method and device for encoding an image, method and device for decoding an image | |
KR20220058958A (en) | Chroma residual scaling to predict a correction value to be added to the luma mapping gradient values | |
US8781023B2 (en) | Method and apparatus for improving transmission of data on a bandwidth expanded channel | |
US9749015B2 (en) | Method and system for transmitting video data over a powerline communications transmission channel | |
US20200090374A1 (en) | Rate distortion optimization for adaptive subband coding of regional adaptive haar transform (raht) | |
US20140298145A1 (en) | Method and apparatus for improving transmission of data on a bandwidth mismatched channel | |
US9692616B1 (en) | Apparatus for and method of channel estimation buffer compression via decimation, prediction, and error encoding | |
EP3035685A1 (en) | A device and a method for encoding an image and corresponding decoding method and decoding device | |
WO2023116443A1 (en) | Point cloud encoding method, point cloud decoding method, communication node, and storage medium | |
WO2024078892A1 (en) | Image and video compression using learned dictionary of implicit neural representations | |
Louw et al. | Efficient conditional entropy estimation for distributed video coding | |
KR100196847B1 (en) | Circuit for practicing differential pulse code modulation | |
El Meddeb et al. | JSCC based On OPTA for wireless communication over flat fading channel |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: AMIMON LTD., ISRAEL Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:REZNIC, ZVI;REEL/FRAME:057600/0783 Effective date: 20190519 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |