CN100385453C - System and method for processing image data - Google Patents
System and method for processing image data Download PDFInfo
- Publication number
- CN100385453C CN100385453C CNB2004800416613A CN200480041661A CN100385453C CN 100385453 C CN100385453 C CN 100385453C CN B2004800416613 A CNB2004800416613 A CN B2004800416613A CN 200480041661 A CN200480041661 A CN 200480041661A CN 100385453 C CN100385453 C CN 100385453C
- Authority
- CN
- China
- Prior art keywords
- image data
- frame
- data layer
- layer
- data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Compression Or Coding Systems Of Tv Signals (AREA)
Abstract
A method of transmitting a plurality of frames of image data over a communications channel from a display image source to a display device is disclosed, wherein the plurality of frames of image data are transferred on a frame-by-frame basis at a predetermined time interval. The method includes encoding at least a portion of a selected frame of image data via hierarchical encoding to form a hierarchically encoded set of image data layers, and sending only a subset of the set of image data layers over the communications channel, wherein the subset of image data layers includes fewer image data layers than the set of image data layers.
Description
The cross reference of related application
The application requires to submit on Dec 16th, 2003, sequence number is the right of priority of 60/530,469 U.S. Provisional Patent Application, and its complete being incorporated herein by reference is used for all purposes.
Technical field
Present invention relates in general to be used for device, the system and method for deal with data, more specifically, relate to the device, the system and method that the multiple image data are transferred to display device from the displayed map image source by communication channel.
Background technology
Usually, provide view data by the image source as on knee or desktop computer, PDA(Personal Digital Assistant) or other computing equipments to display device.Some display device is configured to wirelessly receive view data from image source, for example by the communication protocol as 802.11b (or other 802.11 agreements), bluetooth etc.These display devices can allow the almost any position connection layout image source apace in the meeting room, thereby help single display device is used a plurality of image sources.
Yet, support to use a plurality of image sources can bring all difficulties to single display device.For example, different image sources may utilize different software to generate and/or show the image file of different-format.In the case, support the display device of many image sources may need to comprise the software that is fit to that is used to decompress, provide and/or show the image file of number of different types.Under many circumstances, this software may be provided by the company outside the display device manufacturer.Therefore, install and upgrade these softwares and can make display device be exposed to software virus, programming defective, and exceed under the other problems of display device manufacturer control.In addition, storage may require relative big memory space and processing power with a plurality of software programs that execution needs when showing all required image data layouts.
A kind of feasible method that reduces software amount required on the display device is only raw data file to be sent to display device from each image source, rather than the image data file of transformatization.In the case, display device can only need to support single image data format of planting, and this can simplify the software requirement of display device., compare with formative image file, such raw data file may be very big, therefore may need the long relatively time to be sent to display device from image source, and this depends on the bandwidth of employed communication channel.When needs showed real-time video with such display device, the bandwidth of communication channel was for may be too little with typical video frame frequency (being about for 20 frame/seconds or higher usually) transmission original image data file.
Summary of the invention
One aspect of the present invention discloses and a kind ofly by communication channel the multiple image data has been transferred to the method for display device from the displayed map image source, and wherein the multiple image data transmitted on basis frame by frame by the predetermined time interval.This method comprise by hierarchical coding encode the frame image data selected to small part with the set of diagrams that forms hierarchical coding as data Layer, reach the subclass that only sends this set of image data layer by communication channel, wherein this subclass of image data layer comprises than this set of image data layer image data layer still less.At least a portion of the frame image data that coding is selected comprises: a frame image data of more described selection and a previous frame image data that sends, discern in the frame image data of described selection and compare the part that comprises change, and will comprise the part that has changed pixel and be divided into a plurality of zones that are used to encode with a frame image data of described previous transmission.
Another aspect of the present invention provides a kind of method by traffic channel multiple image data, wherein at interval transmits multiple image data by regular time by communication channel on basis frame by frame.This method comprises in the selected frame image data in location to be compared and comprises the part that changes pixel with next-door neighbour's former frame view data, to comprise the part that has changed pixel and be divided into a plurality of zones that are used to encode, and pyramid is encoded each zone so that each zone is formed set of diagrams as data Layer, wherein every set of image data layer comprises the image data layer coded message that comprises least amount of detail, comprise the image data layer coded message of highest level details, and comprise the image data layer coded message of intermediate level details.This method also comprises the image data layer coded message that comprises least amount of detail of transmitting each zone, and expiring at interval up to regular time to small part in other image data layer in each zone of transmission.
Another aspect of the present invention, a kind of method by traffic channel multiple image data is provided, wherein multiframe comprises first frame and second frame, described first frame has more a spot of change with respect to being close to the frame that transmits before first frame, and described second frame has relatively large change with respect to the frame that transmits before next-door neighbour's second frame.This method comprises that the part that will comprise change in first frame and second frame is divided into a plurality of zones that are used to encode, pyramid is encoded each zone so that each zone is formed set of diagrams as data Layer, transmit all images data Layer of All Ranges in first frame, and only transmit in the image data layer of All Ranges in second frame and comprise the part of comparing change with the view data of previous transmission.
The method according to this invention can reduce the data volume of image data transmission, and avoids transmission and display error.
Description of drawings
The present invention is by the example in the accompanying drawing but not describes by the restriction in the accompanying drawing that wherein identical label is represented identical element, and wherein:
Fig. 1 is the synoptic diagram of image data processing system according to an embodiment of the invention.
Fig. 2 is the process flow diagram of the exemplary method of image data processing according to an embodiment of the invention.
Fig. 3 is the view that is presented at the example image on the image source display, and the change part of the image that is divided into the zone that is used for encoding is shown.
Fig. 4 is the pyramid coding in the zone that each marks off among Fig. 3 and the synoptic diagram of compression.
Fig. 5 is the synoptic diagram of the transmission of view data from the image source to the display device of complete one group of pyramid coding and compression.
Fig. 6 is that the image of Fig. 3 shows the variation view of function between the reception reconciliation compression period of some example image data Layers as the time.
Fig. 7 is the synoptic diagram that only subclass of the view data of pyramid coding and compression is transferred to display device from image source.
Fig. 8 is that the view data of Fig. 7 shows the variation view of function between the reception reconciliation compression period of some example image data Layers as the time.
Fig. 9 is the process flow diagram according to the example embodiment of the method that view data is transferred to display device from image source of the present invention.
Embodiment
Fig. 1 always illustrates the synoptic diagram according to the image processing system of the first embodiment of the present invention at 10 places.Image processing system 10 comprises and is configured to the display device 12 of watching display image on the face 14.Display device 12 can be the display device of any suitable type.Its example includes but not limited to that LCD (LCD) and digital light are handled (DLP) projector, television system, and computer monitor, or the like.
Get back to Fig. 1, image source 18 can comprise can provide equipment 16 that any suitable equipment of view data is provided to image.Its example includes but not limited to, desktop computer and/or server 18
a, laptop computer 18
b, PDA(Personal Digital Assistant) 18
c, mobile phone 18
dOr the like.In addition, image source 18 can provide equipment 16 to carry out telecommunication with image in any suitable manner.In described embodiment, each image source 18 all provides equipment 16 to carry out telecommunication by wireless network 20 and image.Yet image source 18 also can be by cable network, provide equipment 16 to communicate by wireless or wired direct connection or the like or their any combination and image.
When image source 18 is configured to handle the view data of multiple form, the data that image provides equipment 16 can be configured to decode every kind of required image data layout.Yet as mentioned above, this may require image to provide equipment 16 to have the independent software program that enough storeies are stored the every kind of required form that is used to decode.In addition, much may provide the source outside the manufacturer of equipment 16 to provide in these software programs by image.Therefore, using such software can reduce image provides the manufacturer of equipment 16 to provide the control of the software program on equipment and/or the display device 12 to being installed in image.This can make these display devices be opened under virus, defective and other problems introduced by external software during software installation, upgrading or the similar operations.
For simplified image provides the operation and the software requirement of equipment 16, each image source 18 can comprise the bitmap that is configured to generate the image on the display 22, provides the software of equipment 16 to be used for being shown by display device 12 with this bitmap transmissions to image then.This provides such advantage, be that image provides 16 needs of equipment to comprise to be used to and receives the decode single software of planting the view data of form, and therefore help to prevent to install and/or reproducting periods is incorporated into image with virus, defective and other problems and provides on the equipment 16 at software.Yet as mentioned above, unpressed bitmap file may be quite big, and therefore being transferred to image provides equipment 16 to consume the long relatively time, and this depends on the bandwidth of employed communication channel.To the image of relative high resolution scheme, as XGA and above especially true.When data were video data, new data frame is sent to image provided the speed of equipment 16 may be about for 20 frame/seconds or higher.In the case, frame frequency may provide the speed of equipment 16 faster than generating whole bitmap and it being transferred to image, and this may cause the transmission of video image and the error in the demonstration.
For fear of transmission and display error, can before transmission, handle the bitmap that the image on generates from be presented at image source 18, to reduce to every frame image data data quantity transmitted.Fig. 2 always illustrates the example embodiment of the method for the processing bitmap image data that the display on 22 generates from image source 18 at 100 places.Method 100 is usually by software code realization, and this software code is stored in the storer on the image source 18 usually, and can be carried out by the processor on each image source.
Be transferred to the data volume that image provides equipment 16 in order to reduce, method 100 only transmit a frame or set of diagrams as in the data with that frame or those different parts of that set of image data of tightly before present frame, transmitting.Therefore, one group or a frame image data N and current a group or a frame image data N+1 who is presented on the display 22 that method 100 had at first more before been transmitted at 102 places determine to be different among the frame N+1 part of frame N then at 104 places.
Compare two frame image datas at 102 places and determine that at 104 places having changed the zone can carry out in any suitable manner.For example, each can be stored in frame N and N+1 in the impact damper, each pixel that is stored in the view data in the N+1 impact damper can be compared with each pixel that is stored in the view data in the N impact damper then.
Navigating to when changing, can limit and changed the zone and compress in any suitable manner being used for.For example, in certain embodiments, detected change can by among the frame N+1 that comprises view data that is drawn all changed the zone the single rectangular area with variable-size limit.Yet, may there be such situation, wherein like this qualification scheme that the changed the zone mass data that causes in fact not the frame with previous transmission to change is compressed and transmits.For example, with reference to figure 3, during still image 200 demonstrations, during generating slideshow at the display from laptop computer, the user may activate the drop-down menu 202 in the display upper left corner, and the clock symbol 204 that is positioned at the display lower right corner has simultaneously upgraded.In the case, having changed the zone with single square boundary qualification can cause in fact whole screen all to be comprised by this border.This pressure that can cause much not changing data is compiled and transmission, thereby can reduce the efficient of Image Data Compression and transmission.
As limit another selection that all have changed the zone with single border, method 100 is included in 106 places and is divided into the change part that m different zone limits image data frame N by changing part.Schematically point out by 206 in Fig. 3 in these zones.Zone 206 has the set size that equates usually, and can be any size.In described embodiment, drop-down menu is covered by four zones 206, and the clock icon is covered by single zone.Yet, may wish to use relatively little zone 206 to reduce with changing the amount that does not change data that data are compressed and transmitted.
Refer again to Fig. 2, before the change with image 200 partly is divided into zone 206, simultaneously or afterwards, method 100 can be included in that 108 places determine just to be encoded and the palette of image transmitted, and at 110 places the renewal of relevant palette is transferred to image and provides equipment 16 to assist the decompression of compressing image data.This is because 24 colors can be abridged with 8 values of searching in the palette.When reusing color, the data of 8 feasible transmission of abbreviation still less.
Next, method 100 is included in the 112 loop initialization counter j=1 of place, then each zone in being encoded altogether in m zone in the layering of 114 places.At the initialized cycle counter in 112 places to the number count in the zone 206 of hierarchical coding.An example of hierarchical coding is the pyramid coding.Pyramid is encoded to relate to image (or part of image) is encoded to the multilayer details, wherein each layer comprises that quantity is four times in time " pixel " or the data point of the regional outward appearance in the high-rise presentation video, and wherein each pixel in each layer all is the mean value of four neighbors in time low layer.Data in each layer can be expressed as actual value, or are expressed as and are coded in the difference between the mean value in time high level.Though described embodiment is described as using the pyramid coding, should be understood that the hierarchy encoding method that can use any other to be fit to.
Each regional pyramid is coded among Fig. 4 schematically illustrated.Being transferred to image provides each image-region 206 of equipment 16 to illustrate along Fig. 4 top, and the schematically illustrating below the zone of a pyramidal layer group 208 in each zone illustrates.Each layer is all used symbol L
MnMark, wherein subscript m represents this level is associated with which zone 206, and the level " degree of depth " in the subscript n representational level group 208 wherein.For example, leftmost regional 206
aDetailed level (" minimum " level) shown in 210, and be labeled as L
1nSubscript " 1 " represents that this level comprises the data of first area (m=1), and n represents that this level is minimum in " n " individual level.The maximal value of n depends on to make up what pyramid levels, and is configured in the software usually.Far Left zone 206
aThe 3rd the most rough level (" tri-layer ") L
13, the second the most rough level (" second level ") L
12, and the most rough level (" first level " or " highest level ") L
11The pyramid level among Fig. 4 respectively shown in 212,214 and 216.
Relation between the value in first level 216, second level 214 and the tri-layer 212 also illustrates in Fig. 4 by a dotted line.The mean value of four consecutive values 222 in value 220 expressions second level 214 in first level 216, and the mean value of four values 224 in each the value 222 expression tri-layer 212 in second level.
After regional pyramid is encoded to n layer with each, can compress view data in each pyramidal layer by one or more compression steps, with the data volume of further minimizing by traffic channel.Each of view data layer L
MnCompressed format in Fig. 4 by symbol C
MnExpression.View data in each pyramidal layer can be compressed in any suitable manner.In the embodiment of Fig. 2, the view data in each pyramidal layer is all compressed by a plurality of compression steps, with obtain than the compress technique by any independent use the higher compression degree that may reach.
Compression to each pyramidal layer in the zone of pyramid coding starts from 116 places, and at the count initialized device k of this place, counter k counts the quantity of the pyramid level that compressed.Next, at 118 places the view data in each pyramidal layer is transformed into luminance/chrominance color space alternatively.The example of the luminance/chrominance color space that is fit to comprises device-dependent color space, as the YCrCb color space, and device-independent color space, as CIEXYZ and CIE L*a*b* color space.Another example of the device independent color space that is fit to is as described below.Color space comprises brightness r value and colourity s and t value, and derives from CIE L*a*b* color space by following formula:
(1)r＝(L*-L*
min)(r
max/(L*
max-L*
min))
(2)s＝(a*-a*
min)(s
max/(a*
max-a*
min))
(3)t＝(b*-b*
min)(t
max/(b*
max-b*
min))
The r that calculates from these formula, s and t value can round off or be truncated to immediate round values, so that digital format is changed into integer data format from floating-point, thereby simplify the calculating that relates to the value in the color space.In these formula, value L*
Max, L*
Min, a*
Max, a*
Min, b*
MaxAnd b*
MinCan be corresponding to each the practical limit in L*, a* and the b* color space coordinate, or corresponding to another color space, maximal value and minimum value when the color space of the vision facilities 18 selected is on being mapped to CIE L*a*b* color space.Value r
Max, s
MaxAnd t
MaxCorresponding to each max-int of r, s and t color coordinates, and depend on the figure place that is used to represent each coordinate.For example,, there are 64 possible round valuess (0--63) in each coordinate representing each coordinate time with six, and r
Max, s
MaxAnd t
MaxEach all has value 63.
After the view data in each pyramidal layer is transformed into luminance/chrominance color space, can filter the low data that change at 120 places, so that non-cg data (" non-CG data ") similar computer graphics datas (" CG data ").The image that comprises the CG data as video-game, digital slide presentation files or the like and the image that comprises non-CG data, is compared as film, still photo or the like, trends towards the view data of higher frequency is had sharper keen color boundaries.Because these data types are at the different qualities at color boundaries place, different compression algorithms trends towards that the CG data are compared non-CG data and works better.It is also right and wrong CG data of CG data that some existing image data processing system is attempted specified data, then every type data is used different compressor reducers.Yet, CG data mistake is identified as non-CG data, or opposite, can cause the loss of compression efficiency in these systems.Therefore, filter low delta data 120 and can comprise that its variation of identification is lower than the adjacent image data value of the changes of threshold of selecting in advance, this may show the transition between the similar color, changes some image data value then and reduces this variation, thus the color boundaries of the similar CG data of establishment.Therefore the filtration of low delta data can allow suitably to compress non-CG data and CG data with same compressor reducer.The change that non-CG data are made is only made the consecutive value with the variation that is lower than perceptual threshold usually, though can make a change value alternatively with the variation that is higher than perceptual threshold.
Can use any suitable method to come to filter the view data in image data layer low delta data.An example of the method that is fit to is to utilize simple trap noise filter to eliminate low delta data.The trap noise filter can be realized as described below.If p
cThe expression current pixel, p
lThe pixel on the expression current pixel left side, and p
rThe pixel on expression current pixel the right.At first, calculate p
cAnd p
lBetween difference d
lAnd p
cAnd p
rBetween difference d
rNext, compare d
lAnd d
rIf d
lAnd d
rAbsolute value unequal, and d
lAnd d
rIn one lower absolute value be lower than in advance the perceptual threshold of selecting, then can be with p
cReset to and equal p
lOr p
rWith with d
lAnd d
rIn lower one change into 0.Perhaps, can be with p
lAnd p
rIn any change into and equal p
cRealize identical result.
If d
lAnd d
rAbsolute value equate, then with p
cChange into and equal p
lBe equivalent to p
cChange into and equal p
rIn the case, if d
lAnd d
rAbsolute value be lower than predetermined perceptual threshold, then can be with p
cChange into and equal p
lAnd p
rIn any.In addition, if d
lAnd d
rBoth absolute values all are higher than the perceptual threshold of selecting in advance, then do not change p
c, p
lOr p
rIn any.Should be understood that above-mentioned filter method is exemplary, and can use the low delta data of other filtrations that are fit to so that the method for the similar CG data of non-CG.For example, at d
lAnd d
rAbsolute value equate and when being lower than the perceptual threshold of selecting in advance, can using decision function to determine whether to change current pixel and mate the right or the left side, perhaps top or following neighbor.
Make the similar CG data of non-CG data that except that filtering low delta data method 100 also is included in 122 places the chromatic value of the view data in each pyramidal layer is carried out double sampling.Generally speaking, double sampling is a kind of compress technique of at least one color space component being sampled with the spatial frequency lower than at least one other color space component that relates to.Decompression apparatus recomputates the component of losing.The common subsampled data form that is used for luminance/chrominance color space comprises the 4:2:2 double sampling, wherein with half spatial frequency of luminance component chromatic component is sampled and with identical spatial frequency chromatic component is sampled in vertical direction in the horizontal direction; And the 4:2:0 double sampling, wherein all chromatic component is sampled with half spatial frequency of luminance component along vertical and horizontal direction.In these double sampling forms any, or any other double sampling form that is fit to all can be used for the chromatic component of the view data in each pyramidal layer is carried out double sampling.
At the low delta data of 120 places filtration and after 122 places carry out double sampling to chroma data, next method 100 uses one or more other compress techniques further to reduce institute's data quantity transmitted at 124 places.Usually, use the compression method that the CG data is provided good compression.In described embodiment, method 100 is used the delta modulation compression step at 126 places, and uses the LZO compression step at 128 places.LZO be one in real time, portable, harmless data compression library, it pays the utmost attention to speed rather than ratio of compression, and is specially adapted to the Real Time Compression of CG data.LZO also provides other advantage.For example, LZO decompresses needs minimum storer, and compression only needs the storer of 64 kilobyte.After having compressed each pyramidal layer of data, at 130 places with layer C
MnOutputing to output buffer is transferred to image with wait equipment 16 is provided.Next, method 100 is checked at 132 places, determines whether whole group 208 n pyramid image data layer of constricted zone 206 by the quantity n that compares counter k and level.If k is not equal to n, then k increases progressively 1 at 134 places, and begins the compression of next pyramidal layer at 118 places.In case all pyramid image data layer of constricted zone 206, method 100 with regard to verification k=n, continue at 136 places then by determining whether that counter j=m determines whether pyramid coding of all m zones 206.If j is not equal to m, then j increases progressively 1 at 138 places, and method 100 is circulated back to the pyramid coding that 114 places begin next zone 206.In this way, each zone 206 in the method 100 pyramid coded images 200 is with one group 208 of the correspondence that forms the pyramid image data layer, and compresses each pyramidal layer in each zone.
In case encoded by pyramid and compress in each zone 206, compressed and pyramid image encoded data just can be transferred to image equipment 16 is provided.When transmitting video data, the big I of the image data set of the selected frame of expression of pyramid coding and compression is greater than the maximum amount of data that can stride traffic channel during frame period.In the case, image source 18 can be configured to only transmit before the compression of next frame and transmission beginning and can be the as many data of data that a frame image data sends.
Determining to send what layers can carry out in any suitable manner.For example, whole group 208 of the pyramidal layer in a zone 206 can send before any layer in the next zone 206 of transmission.Perhaps, as shown in Figure 5, can at first send the ground floor 230 of the pyramid coded data in each zone 206, next second pyramid coding layer 232, the 3rd layer 234 then, by that analogy, up to the data that sent the maximum that during frame period, can send.At next the more detail layer C that begins to transmit any zone
Mn+1Transmit the selected image data layer C in all m zones 206 before
MnProvide such advantage: All Ranges 206 is all transferred to the small part data, even only part transmission place is also like this in view data.
Fig. 5 illustrates the transmission of whole set of image data.This can only take place when the former frame view data that is close to partly changes at a frame image data of selecting.In video display, for example, all images frame that constitutes single scene demonstration can comprise common element, as the background color in the video display, setting color, costume color or the like.Therefore, in single scene, the change between the frame is caused by the position switching of common element usually, and can be coded in the zone of relatively small amount.In the case, the region quantity of coding can be enough little, to allow the compressing image data that one group of complete pyramid of transmission is encoded during single frame period.
The order that Fig. 6 illustrates the whole group 208 of each regional pyramid coded data in the image 200 provides and shows.It is image by display device 12 projections that label 200 ' is used in reference to image in the diagrammatic sketch 6, rather than is presented at the image in the image source 18.Similarly, label 206 ' is used for the zone of indicating image 200 ' corresponding to the zone in the compressed and encoded image 200 206.
At first, shown in 250, first pyramidal layer of transmission, decompression and demonstration compressing image data, this layer is by symbol C
11-C
51Expression.Next, shown in 252, second pyramidal layer of transmission, decompression and demonstration compressing image data, this layer is by symbol C
12-C
52Expression, this is incorporated into more details in the zone 206 ' of shown image.The decompression and the demonstration of the 3rd pyramidal layer of packed data are incorporated into further details in the zone 206 ' shown in 254.Therefore, for the decompression and the demonstration of each pyramid level of view data, the grade of details all changes with 2 times on the level in each zone 206 ' and vertical both direction.For view data C
1n-C
5nThe decompression and the demonstration of detailed one deck, shown in 256, all details that original image 200 is not lost in the lossy compression method step all have been attached in the image 200 '.Should be understood that the change shown in Fig. 6 can can't help display device 12 actual displayed, but can make, and only when decoding/decompression of all of a frame image data having been transmitted data has been finished, show the data that are kept in the impact damper.
On the other hand, may need the zone 206 of larger amt to come change between some frame of the view data in the encoded video demonstration sometimes.For example, the frame when appearing at scene change, wherein background, clothes and other elements change fully, may have only considerably less common pixels.Therefore, the change of encoding in these frames may need basically entire frame to be divided into the zone 206 that is used to encode.When a frame image data of selecting has the zone 206 of larger amt, during single frame period, can not send all pyramidal layer of a frame image data.This is schematically illustrated in Fig. 7.In the case, image data transmission can stop after last complete image layer 236 that can be transmitted has been transmitted, maybe can be parked in the place, centre positions in next detail image data Layer 238, for example is parked in by the position shown in the line 240.
Fig. 8 illustrates the decompression of a frame image data, has wherein changed zone 206 " occupied entire image basically.The example of image pattern 6 is respectively in the decompression of the ground floor shown in 260,262 and 264, the second layer and the 3rd tomographic image data with show that the level of detail that will increase is incorporated in the image.Yet unlike the example of Fig. 6, the transmission of image data layer is parked in a certain intermediate point place, shown in 266, makes image lack some meticulous details, as the meticulous color transition of the boundary vicinity between color.Such loss of detail concerning the beholder, may discover less than.For example, the view data of only transmitting some layer takes place during the change in big zone and higher movement rate probably, and these are that human aware system is not easy to give careful note to details time of loss.In addition, subsequently the frame of transmission can enough apace details be added in the image with minimize or removal of images in perceptible error.
Can in decompression process, realize helping to improve the various features of decompression performance, thereby improvement display device 12 and image provide the performance of equipment 16 when display video image.For example, in order to assist the decompression of subsampled image data, image provides equipment 16 to comprise to be used for storing image data decompression buffer during decompressing, this decompression buffer less than with the cache memory of the relational processor of carrying out decompression calculations.
The existing decompression systems of subsampled image data of being used for decompressing will be put in order group usually before the chromatic value that calculating is lost compressing image data reads in decompression buffer.Usually, compressing image data is copied to when being read into impact damper in the cache memory, and this value that allows to be stored in the high-speed cache can be visited more quickly to be used for decompression calculations.Yet, because the size of compressed image file may be greater than cache memory, so when copying to compressing image data in the impact damper, some view data in the cache memory may be covered by other view data.The view data that covers in the cache memory may cause when cache misses just takes place when the processor of decompressed image data is searched the data that are capped in cache memory.Too much the appearance of cache misses can be slowed down image decompressor so that harmful stage.
Use helps avoid the generation of cache misses less than the decompression buffer of cache memory.Because the storer that cache memory is normally relatively little, therefore such decompression buffer may be also little than most image files.In other words, when pictorial data representation comprised the image of A * B pel array, decompression buffer can be configured to preserve the array of image data of A * C, and wherein C is less than B.Such impact damper can be used for by one group of subsampled image data is read in impact damper and cache memory this group subsampled image data that decompresses as a series of less subclass of view data.Can be before a new subclass of compressing image data be read in decompression buffer decompress(ion) each subclass of in device output image data of postponing that contracts.Because decompression buffer is less than cache memory, be used to decompression calculations simultaneously so unlikely take place any view data in the cache memory to be capped.
Decompression buffer can have any suitable size.Generally speaking, decompression buffer is more little with respect to cache memory, and the possibility that a large amount of cache misses take place is low more.In addition, the type of the type of the subsampled image data that decompress in decompression buffer and being used for calculating that compressing image data is decompressed can influence the size of decompression buffer.For example, the chromatic component of losing in the 4:2:0 view data can depend on subsampled chrominance value be coordination also be non-coordination and differently calculate.The chromatic value of coordination is positioned on image and the identical physical location of selecting of brightness value, but not the chromatic value of coordination is clipped between several relevant brightness values.The chromatic value of losing of 4:2:0 coordination view data can be calculated by subsampled chrominance value on row identical with the value of losing or adjacent row, and this depends on the physical location of the chromatic value of losing that is calculated.Therefore, the view data that decompression buffer with 4:2:0 coordination view data of the data that do not comprise chromatic value of embarking on journey can be configured to preserve more than delegation that is used to decompress allows to calculate the chromatic value of losing by vertical adjacent chromatic value.
Can use any suitable method to determine that can send what image data layer from image source 18 during single frame period provides equipment 16 to image.For example, a kind of simple method can be when the frame image data on the detected activity image transmitted source 18 changes, and uses detected change to begin new compression and transmission course as trigger.In this way, the image data transmission of pyramid coding and compression can continue, and detects change in the image that shows on the image source of selecting, and should not stop if the data transmission of at this moment previous picture frame is finished as yet.
Another example of determining to send the appropriate method of what image data layer during single frame period comprises the bandwidth of definite communication channel, can stride communication channel by the frame frequency calculating of detected bandwidth and known view data during single frame period then and send how many view data.Bandwidth can be before compressing image data transmission or during disposablely determine, or can periodically detect and upgrade.
Being suitable for only transmitting equivalent always illustrates at 300 places in Fig. 9 in an example embodiment of the method for compressing image data of the pyramid coding that can transmit during single frame period.Method 300 at first is included in the available bandwidth that 302 places determine the image data transmission of pyramid coding and compression to be provided to image the communication channel of equipment 16 thereon.The known frame frequency of the view data by the bandwidth measured and pyramid coding and compression is determined the maximum amount of data that sends by communication channel between can the sampling at each frame image data at 304 places.
In case determined the maximum amount of data that can send during a frame period, method 300 just begins data transmission.At first, method 300 is included in 306 count initialized device k of place and tracking variable " total (total) ".Counter k is used to follow the tracks of the quantity of the complete layer that has sent.Next, method 300 enters the circulation corresponding to counter k, wherein count initialized device j at 308 places.Counter j is used to follow the tracks of the transmission progress of individual data layer.
Next, method 300 beginning image data transmission.At transmission data Layer C
JkBefore, will descend one deck C at 312 places
JkSize be added on the value of variable " total " the relatively value of " total " and the maximum amount of data that can during single frame period, transmit at 314 places then.If the value of " total " is at next data Layer C that adds up
JkSize after greater than the maximum amount of data that can during single frame period, transmit, then do not transmit next data Layer C
Jk, and finish data transmission.
On the other hand, if the value of " total " less than the maximum amount of data that can transmit, is then transmitted next data Layer C at 316 places during single frame period
JkNext, method 300 determines at 318 places whether counting variable j equals m, is then to show all m zone 206 sent the image data layer of selecting.If j is not equal to m, then method 300 adds 1 and be circulated back to 312 places at 320 places to counting variable j.This continuation that circulates is up to all data Layer C that reach fixed value k having been transmitted in all m zones 206
Jk
In case all data Layer C that reach fixed value k have been transmitted in all m zone
Jk, method 300 just determines whether to exist any image data layer that is not transmitted as yet at 322 places by comparing counting variable k and number of plies n.If k is less than n, then k increases progressively 1 at 324 places, and method 300 is circulated back to 308 and sentences the view data that begins to transmit from next more detail layer.
The software of implementation method 100 and 300 various compressions and transmission operation can be used as single thread, individual process operation, or can be used as a plurality of threads or a plurality of process, or their any combination is operated.Multithreading or multi-process method can allow more effectively to utilize than single-threaded or one process method the resource of system 10, as transmission bandwidth.Various operations can be realized by the different threads or the process of any suitable quantity.For example, in one embodiment, use three independently thread operations of coming manner of execution 100 and 300.These threads can be called receiver, processor and transmitter.Receiver thread can obtain the data bitmap that the image from image source 18 screens generates.Processor thread can be carried out comparison, pyramid coding, and other compression steps of method 100.Sender thread can manner of execution 300 bandwidth monitor and transmitting step.Should be understood that this is a kind of exemplary software architecture, and the software architecture that can use any other to be fit to.
In addition, though the present invention includes specific embodiment, specific embodiment should not be regarded as having limited significance, because may there be a large amount of variants.Theme of the present invention comprises various element disclosed herein, feature, function, and/or all novel and non-obvious combination and sub-portfolios of attribute.Following claim particularly points out and is considered as novel and non-obvious particular combinations and sub-portfolio.These claims may be quoted " one " element or " first " element or its equivalence.Such claim should be understood to include the combination to one or more such elements, rather than requires or get rid of two or more such elements.Other combinations of each feature, function, element and/or attribute and sub-portfolio can be by the modifications of claim of the present invention or by providing new claim to ask for protection in the application or related application.No matter such claim is to require wideer, narrower, equivalence or different than original rights on scope, all should be regarded as being included within the theme of the present invention.
Claims (30)
1. one kind is transferred to the method for display device with the multiple image data from the displayed map image source by communication channel, and wherein said multiple image data transmit by the predetermined time interval on basis frame by frame, and described method comprises:
By hierarchical coding encode at least a portion of the frame image data selected with the set of diagrams that forms hierarchical coding as data Layer, a wherein said part is meant with a frame image data of previous transmission compares the part that comprises change; And
Only send the subclass of described set of diagrams as data Layer by communication channel, the subclass of wherein said image data layer comprises than described set of diagrams as data Layer image data layer still less.
2. the method for claim 1 is characterized in that, the subclass of described image data layer includes only the image data layer that can send during the described predetermined time interval.
3. the method for claim 1 is characterized in that, also is included in the described set of diagrams of transmission and determines can send what image data layer by described communication channel in the described predetermined time interval before as the subclass of data Layer.
4. the method for claim 1 is characterized in that, at least a portion of the frame image data that described coding is selected comprises:
One frame image data of more described selection and a previous frame image data that sends;
Discern in the frame image data of described selection and compare the part that comprises change with a frame image data of described previous transmission; And
To comprise the part that has changed pixel and be divided into a plurality of zones that are used to encode.
5. method as claimed in claim 4 is characterized in that, also comprises by hierarchical coding encoding each zone independently to form the multiple series of images data Layer.
6. method as claimed in claim 5, it is characterized in that, described hierarchical coding is the pyramid coding, every set of image data layer of the multiple series of images data Layer behind the wherein said coding comprises that a coded message has the layer of least amount of detail, a coded message has the layer of highest level details, reaches the layer that a plurality of coded messages have intermediate level details.
7. method as claimed in claim 6 is characterized in that, at first transmits the layer that described coded message has least amount of detail, the layer that transmission subsequently incrementally has more details.
8. method as claimed in claim 4 is characterized in that described area size is identical.
9. the method for claim 1, it is characterized in that, one frame image data of described selection is first frame image data, and described set of diagrams is the first set of image data layer as data Layer, also comprise by hierarchical coding and encode at least a portion of second frame image data, transmit all images data Layer in the second set of image data layer then to form the second set of image data layer.
10. method as claimed in claim 9 is characterized in that, described second frame image data comprises than the change with respect to the former frame view data still less of first frame image data.
11. the method for claim 1 is characterized in that, also is included in the described image data layer of transmission and compresses each image data layer before.
12. method as claimed in claim 11 is characterized in that, with each image data layer of at least a technique compresses of selecting one group of technology being formed from double sampling, color space conversion, delta modulation and LZO compression.
13. the method for claim 1 is characterized in that, by encode at least a portion of the frame image data selected of pyramid.
14. the method for claim 1 is characterized in that, described display device is a projector equipment.
15. a method of passing through traffic channel multiple image data to projector equipment is wherein at interval transmitted described multiple image data by regular time by described communication channel on basis frame by frame, described method comprises:
Compare with next-door neighbour's former frame view data in the frame image data that the location is selected and comprise the part that changes pixel;
The described part that changes pixel that comprises is divided into a plurality of zones that are used to encode;
Pyramid is encoded each zone so that each zone is formed set of diagrams as data Layer, wherein every set of image data layer comprises that a coded message has the image data layer of least amount of detail, a coded message has the image data layer of highest level details, and a plurality of coded messages have the image data layer of intermediate level details;
Transmit the image data layer coded message that comprises least amount of detail in each zone; And
Transmit in other image data layer in each zone and comprise the part of comparing change with the view data of previous transmission, expire at interval up to regular time.
16. method as claimed in claim 15 is characterized in that, the transmission of described other image data layer is with the sequential delivery of the details of increase.
17. method as claimed in claim 16 is characterized in that, before the transmission of any zone was incrementally had the image data layer of more details, transmission had all images data Layer of the level of detail of selection to All Ranges.
18. method as claimed in claim 15 is characterized in that, also comprises the maximum amount of data that can stride described traffic channel according to the bandwidth calculation of the communication channel of determining in described interim regular time.
19. method as claimed in claim 18, it is characterized in that, also comprise the maximum amount of data by can striding described traffic channel in described interim regular time and the frame frequency of known view data, calculate the quantity that to stride the layer of described traffic channel in described interim regular time.
20. method as claimed in claim 15 is characterized in that, also is included in the described data Layer of transmission and compresses each image data layer before.
21. method as claimed in claim 20 is characterized in that, with described each image data layer of at least a technique compresses of selecting one group of technology being formed from double sampling, color space conversion, delta modulation and LZO compression.
22. method as claimed in claim 15, it is characterized in that, described different image data frame comprises the change pixel of varying number, and the image data layer that varying number has changed the different images data frame transfer varying number of pixel that comprises to selecting wherein.
23. method by traffic channel multiple image data, wherein said multiframe comprises first frame and second frame, described first frame has more a spot of change with respect to the frame of transmission before next-door neighbour's first frame, and described second frame has relatively large change with respect to the frame that transmits before next-door neighbour's second frame, and described method comprises:
The part that comprises change in described first frame and second frame is divided into a plurality of zones that are used to encode;
Pyramid is encoded each zone so that each zone is formed set of diagrams as data Layer;
Transmit all images data Layer of All Ranges in first frame; And
Only transmit the part of the image data layer of All Ranges in second frame.
24. method as claimed in claim 23, it is characterized in that, every set of image data layer of the multiple series of images data Layer that described a plurality of regional pyramid codings back forms comprises that a coded message has the image data layer of least amount of detail, a coded message has the image data layer of highest level details, and a plurality of coded messages have the image data layer of intermediate level details, and wherein before other image data layer transfer encoding information have the image data layer of a few details.
25. method as claimed in claim 23 is characterized in that, presses the described multiple image data of regular time gap periods ground transmission by described communication channel on basis frame by frame.
26. method as claimed in claim 25 is characterized in that, also is included in before the image data layer of described first frame of transmission or second frame, determines can send what image data layer by described communication channel in described interim regular time, wherein,
Describedly determine that can send what image data layer by described communication channel in described interim regular time comprises: determine the bandwidth of communication channel, and calculate during described Fixed Time Interval, can send what image data layer by described communication channel according to the frame frequency of described bandwidth and known view data.
27. method as claimed in claim 26, it is characterized in that, determine that can send what image data layer in described interim regular time comprises and periodically determine the maximum amount of data that can send by described communication channel in described interim regular time.
28. method as claimed in claim 25 is characterized in that, the part of being transmitted in the image data layer of described second frame is the image data layer of the maximum quantity that can send by described communication channel in described interim regular time.
29. method as claimed in claim 23 is characterized in that, also is included in each image data layer that the compression before of the described image data layer of transmission is transmitted.
30. method as claimed in claim 29 is characterized in that, with the described image data layer of selecting one group of technology being formed from double sampling, color space conversion, delta modulation and LZO compression of at least a technique compresses.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US53046903P | 2003-12-16 | 2003-12-16 | |
| US60/530,469 | 2003-12-16 | ||
| US11/012,985 | 2004-12-14 |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN1914624A CN1914624A (en) | 2007-02-14 |
| CN100385453C true CN100385453C (en) | 2008-04-30 |
Family
ID=37722641
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNB2004800416613A Expired - Fee Related CN100385453C (en) | 2003-12-16 | 2004-12-15 | System and method for processing image data |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN100385453C (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103596011A (en) * | 2013-11-20 | 2014-02-19 | 北京中星微电子有限公司 | Storage processing method and device for image data |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130251261A1 (en) * | 2012-03-23 | 2013-09-26 | Marvell World Trade Ltd. | Method And Apparatus For Image Data Compression |
| CN103491371B (en) * | 2013-09-04 | 2017-04-12 | 华为技术有限公司 | Encoding method, device and equipment based on hierarchy |
| EP3281385B1 (en) * | 2015-04-07 | 2022-01-05 | Intel Corporation | Cloud computing for mobile client devices |
| WO2018195819A1 (en) * | 2017-04-26 | 2018-11-01 | 深圳市大疆创新科技有限公司 | Image processing method and device |
| CN107486991A (en) * | 2017-10-05 | 2017-12-19 | 南京中高知识产权股份有限公司 | 3D is layered Method of printing and 3D layering printing equipments |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1274446A (en) * | 1997-07-14 | 2000-11-22 | 萨尔诺夫公司 | Appts. and method for macroblock based rate control in coding system |
-
2004
- 2004-12-15 CN CNB2004800416613A patent/CN100385453C/en not_active Expired - Fee Related
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1274446A (en) * | 1997-07-14 | 2000-11-22 | 萨尔诺夫公司 | Appts. and method for macroblock based rate control in coding system |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103596011A (en) * | 2013-11-20 | 2014-02-19 | 北京中星微电子有限公司 | Storage processing method and device for image data |
| CN103596011B (en) * | 2013-11-20 | 2017-07-04 | 北京中星微电子有限公司 | The storage processing method and device of view data |
Also Published As
| Publication number | Publication date |
|---|---|
| CN1914624A (en) | 2007-02-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11373338B2 (en) | Image padding in video-based point-cloud compression CODEC | |
| US11288843B2 (en) | Lossy compression of point cloud occupancy maps | |
| US20200286261A1 (en) | Mesh compression | |
| US10424083B2 (en) | Point cloud compression using hybrid transforms | |
| US11122279B2 (en) | Point cloud compression using continuous surface codes | |
| CN110892453B (en) | Point cloud and grid compression using image/video codec | |
| US20220014819A1 (en) | Video image processing | |
| US20190197739A1 (en) | Handling duplicate points in point cloud compression | |
| CN1669233B (en) | Video compression system | |
| CN101160574B (en) | Image processing systems and methods with tag-based communications protocol | |
| US11210812B2 (en) | Single-pass boundary detection in video-based point cloud compression | |
| US11922663B2 (en) | Decision-making rules for attribute smoothing | |
| CN106375841A (en) | Wireless screen projection data processing method and device, video data display method and device, and electronic device | |
| CN105144729A (en) | Transmitting display management metadata over HDMI | |
| CN102771119A (en) | Systems and methods for video-aware screen capture and compression | |
| CN104105006A (en) | Video image processing method and system | |
| CN103716643A (en) | System and method for improving video encoding using content information | |
| US20230308684A1 (en) | Tiling for video based point cloud compression | |
| WO2002031806A1 (en) | Method of and system for low-bandwidth transmission of color video | |
| US7483583B2 (en) | System and method for processing image data | |
| US11006184B2 (en) | Enhanced distribution image system | |
| CN100385453C (en) | System and method for processing image data | |
| CN112999654A (en) | Image processing method, image processing device, computer equipment and storage medium | |
| CN100591126C (en) | Manipulating sub-pictures of a compressed video signal | |
| CN107318021B (en) | Data processing method and system for remote display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| ASS | Succession or assignment of patent right |
Owner name: RPX COMPANY Free format text: FORMER OWNER: INFOCUS CO., LTD. Effective date: 20100608 Owner name: SEIKO EPSON CORP. Free format text: FORMER OWNER: RPX COMPANY Effective date: 20100608 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| COR | Change of bibliographic data |
Free format text: CORRECT: ADDRESS; FROM: CALIFORNIA, USA TO: TOKYO, JAPAN Free format text: CORRECT: ADDRESS; FROM: OREGON, USA TO: CALIFORNIA, USA |
|
| TR01 | Transfer of patent right |
Effective date of registration: 20100608 Address after: Tokyo, Japan Patentee after: Seiko Epson Corp. Address before: California, USA Patentee before: RPX Corp. Effective date of registration: 20100608 Address after: California, USA Patentee after: RPX Corp. Address before: oregon Patentee before: Infocus Corp. |
|
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20080430 Termination date: 20201215 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |