WO2011042905A1 - Distributed video compression system - Google Patents
Distributed video compression system Download PDFInfo
- Publication number
- WO2011042905A1 WO2011042905A1 PCT/IL2010/000818 IL2010000818W WO2011042905A1 WO 2011042905 A1 WO2011042905 A1 WO 2011042905A1 IL 2010000818 W IL2010000818 W IL 2010000818W WO 2011042905 A1 WO2011042905 A1 WO 2011042905A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- pixel
- value
- receiving device
- sending device
- memory
- Prior art date
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
- H04N19/395—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability involving distributed video coding [DVC], e.g. Wyner-Ziv video coding or Slepian-Wolf video coding
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
- H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
- H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
- H04N19/182—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a pixel
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
- H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
- H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A distributed system for transferring compressed video from a limited-resources sending device to a receiving device. The sending device comprises: a camera for capturing a sequence of video frames; a subtracting unit for outputting the difference between the real pixel values and predicted pixel values as received from said receiving device; and a transmitter for transmitting to said receiving device said difference. The receiving device comprises: a memory for storing at least several frames older than the presently captured frame; a predictor for predicting the current pixel, and for transmitting said predicted pixel value to said sending device, and conveying the predicted pixel value to an adder; and a receiver for receiving from said sending device said difference between said real pixel and said predicted pixel; an adder for adding said difference and said predicted value, and for outputting as the sum the real pixel.
Description
0 000818
- 1 -
DISTRIBUTED VIDEO COMPRESSION SYSTEM
Field of the Invention
The present invention relates in general to the field of streaming a wireless signal from a sending device to a receiving device. More specifically, the invention relates to the field of streaming a live, wireless signal from a sending device with limited resources to a receiving device which is essentially not limited in terms of its resources compared to the sending device. Even more specifically, the invention relates to a system for transmitting in real-time a wireless video signal from a sending device having very limited resources to a receiving device which is essentially not limited in terms of its resources compared to the sending device. The term "limited resources at the sending device" relates herein to a limited capacity at the sending device of hardware resources such as memory size and processing power, as well as to the limited availability of space and energy at the sending device.
Background of the Invention
Wireless communication is emerging as the preferred method for transferring video, particularly High-definition (HD) video between different devices. However, this task imposes major challenges when implemented on mobile devices or energy limited fixed devices, including the need to handle very high data bandwidth under limited Radio Frequency (RF) band allocation, a limited availability of space, memory size and processing power, as well as the need to keep a strict power consumption budget to extend the work time at the mobile device.
Video and specifically HD video cameras can be found in increasing number of mobile and limited power fixed applications, such as mobile and cordless phones, digital cameras, miniature video cameras, medical
8
- 2 - implant devices such as in vivo medical implants and capsule endoscopes, miniature in vivo robotics, battery powered security and surveillance cameras, baby monitoring devices, mobile ultrasound devices etc. In many of such applications, the availability of resources at the sending device, such as space for the hardware circuitry, processing power, transmission power, and availability of energy (such as battery size and power) are very limited.
Video compression is one of the common techniques for transferring video over limited RF bands. However, efficient inter-frame video compression algorithms with high compression ratios, such as H.264 encoder, consume high processing power and large memory resources, and consume a significant amount of energy, which may considerably limit the operation time of the sending device.
Real time video transmission is one of the challenges that energy limited devices supporting HD video cameras face. For specific applications such as in vivo medical implants, capsule endoscopies, videophones and miniature video cameras, it is required to capture and transmit HD video from a sending device which is very limited in terms of space, processing power, memory as well as energy. On the other hand, in these applications the receiving device has essentially no limitation with respect to its energy, space, or processing power compared to the sending device.
In medical in vivo applications the video camera is integrated in a miniature implant or capsule and is required to transmit the captured video stream to a receiver located outside the body, which in turn conveys the video data to an external recording device, or to a display. It should be noted that the term "video" which is used throughout this application should be viewed in its broader sense, which does not depend on the type of the images capturing or producing device. For example, the video,
images, and frame series according to the present invention may be a result of an ultrasound capturing device, sonar and Radar imaging, all types of medical radiology such as Magnetic resonance imaging (MRI) and Computed Tomography (CT), and electromagnetic devices which produce images, etc.
One of the ways to reduce transmission bandwidth, power consumption and to achieve a longer operation time at the source device is to reduce the video resolution and to use fewer video frames per second. This option becomes unacceptable as HD resolution and high video frame rate become a standard de facto.
A conventional approach for complying with the limited transmission bandwidth and the limited source device power consumption is to apply a video compression algorithm with a low implementation complexity that results in minimal video quality degradation. A simple low power solution commonly used in such applications is an algorithm for compressing the video stream on a frame-by-frame basis, such as the well known JPEG algorithm. However, this low complexity algorithm can only achieve a limited compression ratio and is not sufficient for applications using HD video transmission, particularly when the available bandwidth is limited.
Motion based algorithms such as H.264 encoder can achieve substantially higher compression ratios. These, however, are high complexity algorithms that cannot be implemented today in applications working under tight energy limitations, and having a limited processing power, limited memory size or limited space.
An efficient video compression based on H.264 encoder requires the evaluation over several latest video frames at the sending side. Naturally, such an evaluation requires the storage of at least said several latest frames at the sending device. When dealing with HD video, the number of
pixels in each frame is significantly increased compared to the case of standard video (SD), and the size of required memory storage at the sending side also increases accordingly. Unfortunately, the memory, the powerful processor, and the interaction between the video processor and the memor3 are the highest consumers of power. Therefore, a solution for overcoming this burden is required.
US 6,594,395, issued July 15 2003 discloses a video encoder with distributed computation and some sort of feedback. More specifically, US 6,594,395 proposes a system in which motion prediction is applied at a receiving device. Then, the receiving device feeds back the motion parameter, as calculated, to the sending device, which in turn uses it to generate the new predicted frame. Then, the sending device calculates the residue between the current frame and its prediction and feeds it forward to the receiving device. Therefore, the sending device of US 6,594,395 requires having the ability to generate the prediction frame. For that purpose, it requires a memory containing at least one frame (previous frame) and a high complexity mechanism to produce the predicted frame from the previous frame and the motion parameters. Therefore, the saving of the previous frame in the memory of the sending device (i.e., transmitter side) is expensive in terms of price, silicon and memory size, and power consumption. This becomes more substantial when using High Definition video. It is therefore necessary to find a solution where the sending device has a reduced memory size, consumes much less energy, and which requires much less computation power.
It is therefore an object of the present invention to provide a system for transmitting video from a sending device having very limited resources to a receiving device having essentially unlimited resources compared to the sending device.
It is still another object of the present invention to provide such a system in which the required processing power, memory size, as well as space at the sending device are significantly reduced.
It is still another object of the present invention to provide a system which is particularly adapted to the transmission of HD video from a sending device with limited resources to a receiving device having essentially unlimited resources compared to the sending device.
It is still another object of the present invention to provide a system which is particularly suitable for in vivo medical implants and capsule endoscopes, where the amount of energy and space are very limited.
Other objects and advantages of the present invention will become apparent as the description proceeds.
Summary of the Invention
The invention relates to a distributed system for transferring a compressed video from a sending device with limited-resources to a receiving device, which comprises: (A) at the sending device: (a) a camera for capturing a sequence of video frames; (b) a pixel-by-pixel streamer for conveying in series to a subtracting unit real pixel values of a video frame which is currently captured by said camera; (c) a subtracting unit for receiving from said pixel-by-pixel streamer at its first input one real pixel value at a time, and for receiving at its second input a predicted value of the same specific pixel as received from said receiving device via a first receiver, and for outputting the difference between said real value of the pixel and said predicted value of the pixel; and (d) a first transmitter for transmitting to said receiving device said difference between said real value of the pixel and said predicted value of the pixel; and (B) at the receiving device: (a) a memory for storing at least several frames older
than the presently captured frame; (b) a present pixel predictor for, based on said several older frames, predicting the value of said specific pixel, and for transmitting said predicted pixel value to said sending device via a second transmitter, and simultaneously conveying the same predicted pixel value to an adder; (c) second receiver for receiving from said sending device said difference between said real value of the specific pixel and said predicted value of the pixel, and for conveying said difference to an adder; (d) an adder for receiving from said second receiver at its first input said difference between said real value of the specific pixel and said predicted value of the pixel, and for receiving at its second input said predicted value of the same specific pixel as predicted by said predictor, and for outputting the sum between said two inputs, said sum being the real value of said specific pixel; and (e) means for storing said real value of said specific pixel at said memory; wherein said sending device and said receiving device operate synchronously and sequentially on pixels, until fully completing the presently captured frame, and then they proceed to pixels of a next frame.
Preferably, said sending device and said receiving device operate on block of pixels, however, said subtracting and said adding are performed separately for each pixel.
Preferably, the size of each block is up to 32x32 pixel values.
Preferably, a delay is introduced between the output of said predictor and the input of said adder, to account for the delay passed until receiving of said difference value.
Preferably, the system further comprises a pre-processing unit at the sending device for dividing the presently captured frame to blocks, and an inverse preprocessing unit at the receiving device for separating each block of pixel values to separate pixel values, before storing them at the
memory.
Preferably, the system further comprises a post compression unit at the sending device, for further compressing said difference values, before sending them to the receiving device, and further comprising an inverse post compression unit at the receiving device, for decompressing the received difference, before forwarding it to the adder.
Preferably, the camera is an imaging device.
Preferably, before the first operation of the system, pixel values of several older frames, either exact or approximated, are sent and store in said memory.
Preferably, one or more synchronizers are provided at the sending device, for ensuring that real and predicted values of a same pixel are introduced simultaneously at the two inputs of the subtracting unit.
Preferably, said predictor applies an algorithm which predicts a value of a pixel of a next frame, given one or more of previous frames, or portions thereof.
Preferably, said predictor applies an optical flow motion estimation algorithm on one or more of older frames for predicting the pixel value within the currently captured frame.
Preferably, said predictor operates in a delta encoder manner of prediction.
The invention also relates to a distributed method for transferring a compressed video from a limited-resources sending device to a receiving
device, which comprises: (a) capturing by a camera at the sending device several older frames, transmitting the pixel values of said several older frames to said receiving device, and storing these values in a memory within said receiving device; (b) conveying at said sending device the real value of a first pixel of a present frame from said camera to a first input of a subtracting unit; (c) having said pixel values of said older frames at said memory, predicting a the value of said first pixel of the present frame, introducing said predicted result to a second input of an adder at said receiving device, and further sending the predicted result to the sending device, and introducing the same to a second input of said subtracting unit; (d) finding by said subtracting unit at the sending device the difference between said real value of the first pixel of the present frame and between the predicted value of the first pixel of the present frame as provided at said first and second inputs of said subtracting unit respectively, and transmitting said difference to said receiving device, and introducing said difference at a first input of said adder at the receiving device; (e) adding by said adder at the receiving device said difference value of the first pixel to said predicted value of the first pixel, as provided at said first and second inputs of the adder, respectively, thereby outputting from the adder the exact value of said first pixel, and storing said exact value of said first pixel at said memory; (f) repeating the above procedure for a next pixel, until completion having all the pixel values of the present frame at the memory of the receiving device; and (g) repeating all the above procedure for a next frame, while considering the present frame as stored in the memory of the receiving device, as one of the older frames.
Brief Description of the Drawings
In the drawings:
Fig. 1 illustrates in a general block diagram form an embodiment of the invention which operates on single pixels;
Fig. 2 illustrates an embodiment of the invention similar to the embodiment of Fig. 1, while operating with blocks of plurality of pixels; and
Fig. 3 illustrates still another embodiment of the invention, which comprises pre-processing and post processing units respectively at the sending device, and inverse pre-processing and inverse postprocessing units at the receiving device;
Fig. 4 shows still another embodiment of the invention similar to the embodiment of Fig. 3, with the addition of a compression unit at the receiving device, and a decompressing unit at the sending device, for compressing and decompressing the predicted values, respectively; and
Fig. 5 illustrates an embodiment of the procedure of the present invention in a flow diagram form.
Detailed Description of Preferred Embodiments
As discussed above, the process of video compression (such as H.264 encoder type of video compression) consumes a significant amount of resources at the sending device (the "sending device" captures video frames, compresses the video data to produce a compressed video, and sends the compressed video to a receiving device). More specifically, the process of video compression requires saving of at least several full frames of the captured video in memory within the sending device, and it also requires performing a very large number of calculations while interacting with said memory to produce the compressed video. Said large number of calculations involves many accesses to the memory during the compression process, while each access to the memory consumes a significant amount of energy at the sending device. There are many cases where the availability of energy at the sending device is very limited, and therefore saving of energy is highly required. Moreover, there are cases where the space available at the sending device prohibits the use of
components having a strong processing power. This problem becomes even more severe when dealing with a high definition (HD) video, as a HD video is much larger in terms of the number of pixels, and data. Therefore, HD video requires a significant increase of processing power, the size of the memory, and the number of the accesses to the memory. All these aspects require a significant increase to the energy consumption, to the processing power, memory size and to the available space of the sending device. Hereinafter, the energy consumption, space, memory size, and processing power will be referred to by the general term "resources".
The term Camera, when used throughout this application refers to any imaging device which produces sequential images. This device may operate, for example, in the visual range, non-visual range, electromagnetic range, sonic or ultrasonic range etc. For example, an ultrasound device which produces images is also within the scope of the invention.
The present invention provides a structure which can very significantly reduce the amount of resources at the sending device. It should be noted, that according to the present invention, the sending device of the prior art is replaced by a limited resources bidirectional communication device having both transmission and reception capabilities, and this limited resources device will be referred herein as a "sending device". Similarly, the receiving device of the prior art is replaced by a bidirectional communication device having essentially no limitation to its resources compared to the sending device, and which also has both transmission and reception capabilities. This device will be referred herein as a "receiving device".
Fig. 1 illustrates in a general block diagram form the structures of the sending device 1 and of the receiving device 2 according to an embodiment
2010/000818
- 11 - of the present invention. The available resources at the sending device 1 are very limited compared to the resources at the receiving device 2 in terms of energy, memory size, processing power, and space. Camera 11 at the sending device captures video frames in a conventional manner. A single "current frame" 12 is indicated as frame Fc. As is conventional, the video frame Fc is represented by plurality of pixels FcPi-FcPn wherein pixels Pi-Pn represent all the pixels within each frame, respectively. The first frames at the beginning of a transmission session are handled differently than the rest of the frames in the session. They may be compressed using any intra-frame compression method, such as JPEG or any other compression method, to fit into the available RF transmission bandwidth. Then they are transmitted to the receiving device and stored in memory 22. The frames that are stored in memory 22 are always older than the currently captured frame 12 at the sending device 1, and the memory 22 is typically updated in a FIFO manner such that it always stores at least several older frames than the presently captured frame Fc. Pixel by pixel streamer 13 at the sending device serially transfers one by one pixel P% of the presently captured frame Fc to subtracting unit 15 through optional first synchronizing unit 14 whose function will be described hereinafter. Simultaneously, present pixel predictor 23 at the receiving device 2 predicts the value of present pixel Px using the content of several of the older frames (i.e., of frames Fc-i-Fc-m, wherein in the example of Fig. 1 m-3) that are stored within memory 22. The process which is performed by present pixel predictor 23 is conventional and well known in the art, and may use any suitable algorithm for next pixel prediction based on pixel history. One such example is the optical flow motion estimation algorithm. Another prediction method is the delta encoder in which a pixel from the previous frame is used to predict current frame pixel. In still another example, present pixel predictor may use the dynamics within the content of older frames Fc-i-Fc-3 to predict the value of pixel FcPx. The result of the present pixel predictor 23 is a predicted value
of pixel Px which is indicated herein as Px , or more specifically FCPX . The value of FcPx is transmitted by transmitter 24 to the sending device 1, and is received at its receiver 16. Receiver 16 transfers the received value of FCPX to subtracting unit 15, optionally through a second synchronizer 17.
The optional one or more synchronizers 14 and 17 ensure that the subtracting unit 15 receives at its first and second inputs the exact and predicted values of a same pixel, as received from the pixel by pixel streamer 13 and from the receiver 16 respectively. Subtracting unit 15 subtracts the predicted value of FCPX from the exact value FcPx, and issues a value which indicates the difference AFcPx between these input values. Generally, the values of FcPx and FCPX are expected to be very close, and therefore the value of the difference AFcPx is expected to be a very small number that can be expressed by a very small number of bits. The difference AFcPx which as said is generally expressed by a very small number of bits (and therefore can be considered as "compressed") is transmitted by transmitter 28 to the receiving device 2. The compressed difference AFcPx is received at the receiver 25, and is conveyed to adding unit 26. Adding unit 26 adds the value of AFcPx, which in fact reflects the "error" between the predicted value of Px (i.e., FCPX ) as predicted by predictor 23 and the true value of Pr as measured at the sending device, to the value of FCPX to obtain the reconstructed value of Px, i.e., FcPx. This value of FcPx is updated at the memory 22, and the same procedure is repeated for the next pixel of frame Fc, i.e., pixel FcPx+i, and similarly to all the rest of the pixels of Fc. Upon completion of obtaining and storing all the values of the pixels of frame Fc at the memory 22, the frame can be displayed 100, and the same procedure repeats with respect to the next frame Fc+i while the present pixel predictor uses the data of the (now) previously obtained frame Fc, as well as other older frames in memory 22, to find each time the predicted value of the present pixel. It should be
noted that delay 27 is necessary at the receiving device in order for the value of FCPX to "wait" until receipt of the calculated AFcPx at the receiving device 2.
As demonstrated, the invention of Fig. 1 very significantly reduces the amount of memory at the sending device. Theoretically, the amount of memory at the receiving device may be reduced to one pixel only, in compare to a memory that has to contain the values of all the pixels of several frames as is typically required in an H.264 sending device of the prior art. It should be noted that typically each frame of high definition video in a 1080P format requires 50Mbit of memory. The many accesses to the memory in the prior art sending device in order to perform the compression require a strong processor having a high computation power at the sending device, and even more important, the many accesses to the memory consume very high amount of energy in compare to the present invention where both the memory size as well as the computation power are very significantly reduced at the sending device.
Fig. 1 discloses the basic structure of the sending and receiving devices of the system of the invention. In the system of Fig. 1, and before the system can operate, the memory 22 of the receiving device 2 has to contain pixels data of several older frames, in order to enable the present pixel predictor 23 to operate. The description of Fig. 1 shows that when the memory 22 contains exact values of the pixels of older frames (prior to the calculation of the pixels of the present frame); the calculations at the receiving device provide accurate results for all the pixels of the present frame. Special care is given to the initial conditions of the transmission from the sending device to the receiving device. At the beginning of a transmission session, the first transmitted frame has no frame history at the receiving unit, and thus cannot be compressed using the procedure of the invention. Therefore, one or a few (for example, up to three) of the first frames are
sent to the sending device either compressed or not, and stored in memory 22. For example, in order to suit the transmitter bandwidth at the sending device, one or more initial frames may be compressed using any intra- frame compression method, such as JPEG or another compression method, to fit them into the available RF bandwidth. Then, the procedure of the present invention can be applied for the following frames and onward.
Fig. 1 above shows an embodiment of the invention which handles single pixels in series (i.e., it operates in a "pixel by pixel" manner). In another embodiment of the invention, as demonstrated in Fig. 2, the system of the invention operates in a "block by block" manner, wherein each block comprises plurality of pixels, for example, 16 or 64 pixels per block. The operation is essentially the same as described with respect to Fig. 1, however, all the units of Fig. 2 are fed by blocks of pixels rather than one pixel only. It should be noted, however, that the calculations, for example, by units 115 and 126 are performed separately with respect to corresponding values that relate to a same single pixel (even if these units are fed by blocks). For example, while the subtracting unit 115 is fed by the values of FCBX and FCBX respectively, the subtraction is performed by unit 115 separately between each two corresponding pixels within Bx and ^ . The same comment is applied, mutatis mutandis also with respect to adder 126. These units perform said calculations serially with respect to all the pixels within the block, and when they finish, they are fed by a next block.
It should also be noted that conventional encoders are included within the transmitters 118 and 124 respectively, and conventional decoders are included within the receivers 116 and 125 respectively.
Fig. 3 shows still another embodiment of the invention. The system
operates essentially as described in Fig. 2, besides the following issues. The sending device of Fig. 3 comprises camera 211 that provides a stream of frame pixel matrix in Bayer CFA format, or any other RGB format. Preprocessing unit 219 divides each frame to blocks, for example having a size between 1 pixel to 8x8 pixels per block. Pre-compression unit 219 also performs various image transformations such as RGB to YCrCb and sub sampling of Cr and Cb. This is in order to provide a ready to compression pixels data. The sending unit 201 further comprises a post-processing unit 230 which performs quantization of the transmitted data, as well as compression in order to further reduce the signal bandwidth. The receiving device 202 in turn comprises an inverse post processing unit 228 which performs decompression and essentially an inverse process with respect to the process as performed by the post processing unit 230 at the sending device 201. The receiving device 202 further comprises an inverse pre-processing unit 229, which performs an inverse process with respect to the process as performed by the preprocessing unit 219 at the sending device. The output from the inverse pre-processing unit 229 is fed to memory 222, until completion of the construction of the current frame. The complete current frame Fc can be used for real time display 100 or it may be stored in memory for future use, and this frame should also be maintained in the memory 222 as a history frame for the purpose of block prediction by the present block predictor 223 to construct the next frame.
Fig. 4 shows still another embodiment of the invention. This embodiment is similar to the embodiment of Fig. 3; however with the addition of a compression unit 232 at the receiving device for compression the predicted values before their transmission to the sending device, and a decompression unit 240 at the sending device for decompressing said predicted values, upon receiving them at the sending device. These compression and decompression units may be of any type known in the art.
IL2010/000818
- 16 -
Fig. 5 illustrates the procedure of the invention in a flow diagram form. The procedure is performed as follows:
In step 401: capturing by a camera at a sending device one or more older frames, transmitting the pixel values of said older frames to a receiving device, and storing said pixel values of said older frames in a memory at said receiving device;
In step 402: conveying at said sending device the real value of a first pixel of a present frame from said camera to a first input of a subtracting unit;
In step 403: having said pixel values of said older frames at said memory of the receiving device, predicting at the receiving device the value of said first pixel within the present frame, introducing the predicted result to a second input of an adder at said receiving device, and further sending the predicted result to the sending device, and introducing the same predicted pixel value to a second input of said subtracting unit;
In step 404: finding by said subtracting unit at the sending device the difference between said real value of the first pixel of the present frame and between the predicted value of the first pixel of the present frame as provided at said first and second inputs of said subtracting unit respectively, and transmitting said difference to said receiving device, and at the receiving device introducing said difference at a first input of said adder;
In step 405: adding by said adder at the receiving device said difference value of the first pixel to said predicted value of the first pixel, as provided at said first and second inputs of the adder, respectively, thereby outputting from the adder the
IL2010/000818
- 17 - real value of said first pixel, and storing said real value of said first pixel at said memory;
- In step 406: checking whether obtaining and storing within the memory of all the pixel values of the present frame have been completed; if not, repeating the above procedure from step 402 for a next pixel of the present frame; If, however, all the pixels of the present frame have been obtained and stored within the memory at the receiving device, continuing in step 407 to a next frame, and repeating the procedure from step 402 for the first pixel of said next frame; and
repeating all the above procedure of steps 402-407 for said next frame, while considering the present frame as stored in the memory as one of the older frames.
As shown, the sending device as described above is very efficient in terms of the resources used, particularly in terms of the memory size, the consumption of energy, and the computation power. For example, while in the prior art there is a need for a memory size in the order of tens to hundreds of Megabits at the sending device in order to handle the compression and transmission of high definition video, according to the present invention there is a need for a memory size in the order of tens of Kilobits for the compression and transmission of a compressed high definition video. Furthermore, the fact that the size of the memory is very significantly reduced, and the fact that the complicated computations for compression of the video, that involve substantive number of accesses to the memory are significantly reduced both in terms of their number and in terms of their complexity, result in a very significant reduction in the consumed energy at the sending device. Adding a transmission channel between the receiving device and the sending device in the present invention adds very little resources at the sending device compared to the prior art, since receiver circuitry is considerably less resource hungry
compared to transmitter or to video compression circuitry. For example, while in a typical case the H.264 video compression at the sending device consumes power in the order of hundreds of mW, the video compression plus the additional receiver at the sending device of the present invention consumes power in the order of several tens of milliWatts (while having similar transmission power in both said cases).
Claims
1. A distributed system for transferring a compressed video from a sending device with limited-resources to a receiving device, which comprises: at the sending device:
a camera for capturing a sequence of video frames; a pixel-by-pixel streamer for conveying in series to a subtracting unit real pixel values of a video frame which is currently captured by said camera;
a subtracting unit for receiving from said pixel-by-pixel streamer at its first input one real pixel value at a time, and for receiving at its second input a predicted value of the same specific pixel as received from said receiving device via a first receiver, and for outputting the difference between said real value of the pixel and said predicted value of the pixel;
and
- a first transmitter for transmitting to said receiving device said difference between said real value of the pixel and said predicted value of the pixel;
and
at the receiving device:
a memory for storing at least several frames older than the presently captured frame;
a present pixel predictor for, based on said several older frames, predicting the value of said specific pixel, and for transmitting said predicted pixel value to said sending device via a second transmitter, and simultaneously conveying the same predicted pixel value to an adder; a second receiver for receiving from said sending device said difference between said real value of the specific pixel and said predicted value of the pixel, and for conveying said difference to an adder;
an adder for receiving from said second receiver at its first input said difference between said real value of the specific pixel and said predicted value of the pixel, and for receiving at its second input said predicted value of the same specific pixel as predicted by said predictor, and for outputting the sum between said two inputs, said sum being the real value of said specific pixel; and
means for storing said real value of said specific pixel at said memory;
wherein said sending device and said receiving device operate synchronously and sequentially on pixels, until fully completing the presently captured frame, and then they proceed to pixels of a next frame.
A distributed system according to claim 1, wherein said sending device and said receiving device operate on block of pixels, however, said subtracting and said adding are performed separately for each pixel.
A distributed system according to claim 2, wherein the size of each block is up to 32x32 pixel values.
System according to claim 1, wherein a delay is introduced between the output of said predictor and the input of said adder, to account for the delay passed until receiving of said difference value.
System according to claim 2, further comprising a pre-processing unit at the sending device for dividing the presently captured frame to blocks, and an inverse preprocessing unit at the receiving device for separating each block of pixel values to separate pixel values, before storing them at the memory.
6. System according to claim 2, which further comprises a post compression unit at the sending device, for further compressing said difference values, before sending them to the receiving device, and further comprising an inverse post compression unit at the receiving device, for decompressing the received difference, before forwarding it to the adder.
7. System according to claim 1, wherein the camera is an imaging device .
8. System according to claim 1, wherein before the first operation of the system, pixel values of several older frames, either exact or approximated, are sent and store in said memory.
9. System according to claim 1, wherein one or more synchronizers are provided at the sending device, for ensuring that real and predicted values of a same pixel are introduced simultaneously at the two inputs of the subtracting unit.
10. System according to claim 1, wherein said predictor applies an algorithm which predicts a value of a pixel of a next frame, given one or more of previous frames, or portions thereof.
11. System according to claim 1, wherein said predictor applies an optical flow motion estimation algorithm on one or more of older frames for predicting the pixel value within the currently captured frame.
12. System according to claim 1, wherein said predictor operates in a delta encoder manner of prediction.
13. A distributed method for transferring a compressed video from a limited-resources sending device to a receiving device, which comprises:
capturing by a camera at the sending device several older frames, transmitting the pixel values of said several older frames to said receiving device, and storing these values in a memory within said receiving device;
conveying at said sending device the real value of a first pixel of a present frame from said camera to a first input of a subtracting unit;
having said pixel values of said older frames at said memory, predicting a the value of said first pixel of the present frame, introducing said predicted result to a second input of an adder at said receiving device, and further sending the predicted result to the sending device, and introducing the same to a second input of said subtracting unit;
finding by said subtracting unit at the sending device the difference between said real value of the first pixel of the present frame and between the predicted value of the first pixel of the present frame as provided at said first and second inputs of said subtracting unit respectively, and transmitting said difference to said receiving device, and introducing said difference at a first input of said adder at the receiving device;
adding by said adder at the receiving device said difference value of the first pixel to said predicted value of the first pixel, as provided at said first and second inputs of the adder, respectively, thereby outputting from the adder the exact value of said first pixel, and storing said exact value of said first pixel at said memory;
repeating the above procedure for a next pixel, until completion having all the pixel values of the present frame at the memory of the receiving device; and
repeating all the above procedure for a next frame, while considering the present frame as stored in the memory of the receiving device, as one of the older frames.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US27842809P | 2009-10-08 | 2009-10-08 | |
| US61/278,428 | 2009-10-08 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2011042905A1 true WO2011042905A1 (en) | 2011-04-14 |
Family
ID=43856426
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IL2010/000818 WO2011042905A1 (en) | 2009-10-08 | 2010-10-07 | Distributed video compression system |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2011042905A1 (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105763826A (en) * | 2016-02-22 | 2016-07-13 | 青岛海信电器股份有限公司 | Video data input method, video data output method, video data input device, and video data output device |
| CN109978968A (en) * | 2019-04-10 | 2019-07-05 | 广州虎牙信息科技有限公司 | Video rendering method, apparatus, equipment and the storage medium of Moving Objects |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6304602B1 (en) * | 1994-08-22 | 2001-10-16 | Nec Corporation | Method and system for processing data on motion pictures by motion compensation combined with image segmentation |
| US20030007697A1 (en) * | 1996-10-30 | 2003-01-09 | Matsushita Electric Industrial Co., Ltd. | Image decoding apparatus using pixel values from at least three reference pixels |
| US6594395B1 (en) * | 1997-07-02 | 2003-07-15 | Forskarpatent I Linkoping, Ab | Real-time mobile video communication with low power terminals |
| US6885704B1 (en) * | 1999-07-28 | 2005-04-26 | Matsushita Electric Industrial Co., Ltd. | Coding apparatus and coding method of time-varying image signal |
| US20090016627A1 (en) * | 2007-07-09 | 2009-01-15 | Kabushiki Kaisha Toshiba | Image compressor, image expander and image processing apparatus |
-
2010
- 2010-10-07 WO PCT/IL2010/000818 patent/WO2011042905A1/en active Application Filing
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6304602B1 (en) * | 1994-08-22 | 2001-10-16 | Nec Corporation | Method and system for processing data on motion pictures by motion compensation combined with image segmentation |
| US20030007697A1 (en) * | 1996-10-30 | 2003-01-09 | Matsushita Electric Industrial Co., Ltd. | Image decoding apparatus using pixel values from at least three reference pixels |
| US6594395B1 (en) * | 1997-07-02 | 2003-07-15 | Forskarpatent I Linkoping, Ab | Real-time mobile video communication with low power terminals |
| US6885704B1 (en) * | 1999-07-28 | 2005-04-26 | Matsushita Electric Industrial Co., Ltd. | Coding apparatus and coding method of time-varying image signal |
| US20090016627A1 (en) * | 2007-07-09 | 2009-01-15 | Kabushiki Kaisha Toshiba | Image compressor, image expander and image processing apparatus |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105763826A (en) * | 2016-02-22 | 2016-07-13 | 青岛海信电器股份有限公司 | Video data input method, video data output method, video data input device, and video data output device |
| CN105763826B (en) * | 2016-02-22 | 2019-08-06 | 青岛海信电器股份有限公司 | A kind of input of video data, output method and device |
| CN109978968A (en) * | 2019-04-10 | 2019-07-05 | 广州虎牙信息科技有限公司 | Video rendering method, apparatus, equipment and the storage medium of Moving Objects |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11843794B2 (en) | CABAC decoder with decoupled arithmetic decoding and inverse binarization | |
| US11924465B2 (en) | Luma-based chroma intra-prediction for video coding | |
| US8160136B2 (en) | Probabilistic bit-rate and rate-distortion cost estimation for video coding | |
| US10448027B2 (en) | Method of encoding video data, video encoder performing the same and electronic system including the same | |
| EP1958448B1 (en) | Multi-dimensional neighboring block prediction for video encoding | |
| RU2582057C2 (en) | Device and method for image processing | |
| US8374444B2 (en) | Method and apparatus for providing higher resolution images in an embedded device | |
| KR101056096B1 (en) | Method and system for motion compensated frame rate up-conversion for both compression and decompression video bitstreams | |
| EP2635028A2 (en) | Image processing device and method | |
| EP1982531A2 (en) | Video encoding | |
| USRE49077E1 (en) | Ultra low latency video communication | |
| EP2629520B1 (en) | Apparatus and method for shooting moving picture in camera device | |
| Lin et al. | Novel chroma subsampling strategy based on mathematical optimization for compressing mosaic videos with arbitrary RGB color filter arrays in H. 264/AVC and HEVC | |
| Mohammed et al. | Lossless compression in Bayer color filter array for capsule endoscopy | |
| US20110255597A1 (en) | Method and System for Reducing Flicker Artifacts | |
| WO2011123882A2 (en) | Video transmission system having reduced memory requirements | |
| US20130301700A1 (en) | Video encoding device and encoding method thereof | |
| Liu et al. | A complexity-efficient and one-pass image compression algorithm for wireless capsule endoscopy | |
| WO2011042905A1 (en) | Distributed video compression system | |
| JP2022549773A (en) | Lossless encoding of video data | |
| Song et al. | Towards a multi-terminal video compression algorithm using epipolar geometry | |
| JP2022548555A (en) | Motion compensation method for video coding | |
| JP2012039181A (en) | Image transmission apparatus | |
| JP2007329946A (en) | Capsule endoscope system | |
| Shilpa et al. | Performance Evaluation of Video Compression Techniques: x. 263, x. 264 and x. 265 to Improve Video Streaming Quality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 10821675 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 10821675 Country of ref document: EP Kind code of ref document: A1 |