Summary of the invention
Main purpose of the present invention is to provide a kind of remote desktop image transfer method and system, is intended to obtain better effect of optimization.
The invention discloses a kind of remote desktop image transfer method, comprise the following steps:
The present frame of desktop picture is carried out piecemeal according to piecemeal rule by step S01, compression end, and set up hierarchical index, according to described hierarchical index, detect the modified-image block of the image block relative image block cache of described present frame and non-modified-image block, obtain view data by encapsulating together with the label of described non-modified-image block after described modified-image block compression, and described view data is sent to solution pressure side;
After step S02, described decompress(ion) termination receive described view data, after distinguishing the label of described modified-image block corresponding in described view data and non-modified-image block according to described piecemeal rule, be corresponding image block by described image data restoration; Merge described image block, show described desktop picture.
Preferably, described step S01 specifically comprises the following steps:
According to described piecemeal rule, be some described image blocks by the present frame cutting of described desktop picture;
According to described image block synthetic image fingerprint, and set up the hierarchical index of described image block;
Utilize described finger image and hierarchical index, successively search for by top layer index and judge whether described image block can find in described image block buffer memory;
If not, then described image block is modified-image block, is compressed by described modified-image block, obtains packed data;
If so, then described image block is non-modified-image block, the image tag of non-modified-image block described in record;
Encapsulate described packed data and described image tag, obtain the described view data after encapsulation and described view data is sent to described solution pressure side.
Preferably, described step S02 specifically comprises the following steps:
After receiving the described view data of described compression end transmission, distinguish described packed data and described image tag according to described piecemeal rule;
Non-modified-image block described in recovering according to described hierarchical index and described image tag, and packed data described in decompress(ion) recovers described modified-image block;
Merge described modified-image block and non-modified-image block, show described desktop picture, simultaneously piecemeal result and described hierarchical index described in buffer memory.
Preferably, described hierarchical index adopts the index entry of stratification to organize described image block, described index entry level represent the ascending of described image block respectively from low to high, and the described index entry of most top layer represents whole described image block.
Preferably, described piecemeal rule keeps synchronous in described compression end and described solution pressure side and can switch according to embody rule scene; Described piecemeal rule comprises employing Image Edge-Detection and edge compensation carries out random length image block.
Preferably, the step that described piecemeal rule carries out switching according to embody rule scene specifically comprises:
Utilize described piecemeal result and the hierarchical index of buffer memory, detect the modified-image block of described present frame relative image block cache and non-modified-image block, and testing result is transmitted;
Judge whether Remote desk process stops;
If not, then collect present image information and judge whether current scene meets piecemeal changing rules condition;
If meet, then according to described present image information, switch to new piecemeal rule;
Utilize described new piecemeal rule to carry out piecemeal to described present frame, and replace existing described piecemeal result and hierarchical index.
The present invention also discloses a kind of remote desktop image delivering system, comprising:
Compression end, for the present frame of desktop picture being carried out piecemeal according to described piecemeal rule, and set up hierarchical index, according to described hierarchical index, detect the modified-image block of the image block relative image block cache of described present frame and non-modified-image block, obtain view data by encapsulating together with the label of described non-modified-image block after described modified-image block compression, and described view data is sent to solution pressure side;
Described image data restoration, after receiving described view data, according to described piecemeal rule, after distinguishing the label of described modified-image block corresponding in described view data and non-modified-image block, is corresponding image block by described solution pressure side; Merge described image block, show described desktop picture.
Preferably, described compression end specifically comprises:
The present frame cutting of described desktop picture, for according to described piecemeal rule, is some image blocks by image block module;
Hierarchical index sets up module, for according to described image block synthetic image fingerprint, and sets up the hierarchical index of described image block;
Image change identification module, for utilizing described finger image and hierarchical index, successively searching for and judging whether described image block can find in described image block buffer memory by top layer index; If so, then described image block is non-modified-image block; If not, then described image block is modified-image block;
Image processing module, for when described image block is modified-image block, compresses described modified-image block, obtains packed data; When for there is not image block in described image block, the image tag of non-modified-image block described in record;
View data output module, for encapsulating described packed data and described image tag, obtaining the described view data after encapsulation and described view data is sent to described solution pressure side.
Preferably, described solution pressure side specifically comprises:
Image area sub-module, after receiving described view data that described compression end sends, distinguishes described packed data and described image tag according to described piecemeal rule;
Image-restoration module, for non-modified-image block according to described hierarchical index and the recovery of described image tag, and packed data described in decompress(ion) recovers described modified-image block;
Image display, for merging described modified-image block and non-modified-image block, shows described desktop picture, simultaneously piecemeal result and described hierarchical index described in buffer memory.
Preferably, described hierarchical index adopts the index entry of stratification to organize described image block, described index entry level represent the ascending of described image block respectively from low to high, and the described index entry of most top layer represents whole described image block.
Preferably, described compression end also comprises:
Piecemeal rule arranges module, for keeping described piecemeal regular and synchronized in described compression end and described solution pressure side and can switching described piecemeal rule according to embody rule scene; Described piecemeal rule comprises employing Image Edge-Detection and edge compensation carries out random length image block.
Preferably, described piecemeal rule arranges module and specifically comprises:
Image block detecting unit, for utilizing described piecemeal result and the hierarchical index of buffer memory, detecting the modified-image block of described present frame relative image block cache and non-modified-image block, and testing result being transmitted;
Image information collector unit, for judging whether Remote desk process stops, and when described Remote desk process does not stop, collecting present image information and judging whether current scene meets piecemeal changing rules condition;
Piecemeal changing rules unit, during for meeting piecemeal changing rules condition in current scene, according to described present image information, switches to new piecemeal rule;
Hierarchical index replacement unit, for utilizing described new piecemeal rule to carry out piecemeal to described present frame, and replaces existing described piecemeal result and hierarchical index.
The present invention, by being some image block according to piecemeal rule by desktop picture cutting in compression end, detecting the modified-image block of the image block relative image block cache of present frame and non-modified-image block, and sends to solution pressure side after carrying out respective handling to image block; Separate pressure side and distinguish modified-image block and non-modified-image block from the view data received, and be corresponding image block by image data restoration, this desktop picture will be shown and the method for whole process compresses end reconciliation pressure side maintenance piecemeal regular and synchronized after the image block merging of recovery, there is the beneficial effect reducing transmitted data amount, improve image compression ratio, improve systematic function, improve Consumer's Experience, obtain the effect of optimization of better remote desktop image transmitting.
Embodiment
Technical scheme of the present invention is further illustrated below in conjunction with Figure of description and specific embodiment.Should be appreciated that specific embodiment described herein only in order to explain the present invention, be not intended to limit the present invention.Remote desktop described in the embodiment of the present invention is a kind of patterned remote-access technology, the computer of user's remote control network other end can be made, the desktop picture of controlled terminal can be presented on the display device of user, makes user obtain the same experience of operation local device; Corresponding compression end is the controlled computer of telecommunication network in a preferred embodiment, separates pressure side and is the computer shown at user side.The implication of subsequent embodiment to remote desktop, compression end and solution pressure side repeats no more.
Reference Fig. 1, Fig. 1 are remote desktop image transfer method one embodiment schematic flow sheets of the present invention; As shown in Figure 1, remote desktop image transfer method of the present invention comprises the following steps:
The present frame of desktop picture is carried out piecemeal according to piecemeal rule by step S01, compression end, and set up hierarchical index, according to hierarchical index, detect the modified-image block of the image block relative image block cache of described present frame and non-modified-image block, obtain view data by encapsulating together with the label of described non-modified-image block after described modified-image block compression, and described view data is sent to solution pressure side;
Reference Fig. 2, Fig. 2 are compression end image procossing block schematic illustration described in remote desktop image transfer method of the present invention; As shown in Figure 2, first the cutting of current desktop image according to certain piecemeal rule, is some image blocks, and sets up hierarchical index by compression end; The modified-image block of the image block relative image block cache of present frame and non-modified-image block is detected according to described hierarchical index.Described image block buffer memory preserves the image block occurred in the recent period, can trace back to the image block that front N frame occurred, described image block is buffered in compression end and conciliates pressure side maintenance synchronously; Described N value can be arranged as the case may be, N be greater than 1 positive integer.The definition of described modified-image block and non-modified-image block is: according to described hierarchical index, and whether the image block detecting present frame can find in image block buffer memory; If so, the image block that then definition present frame finds in image block buffer memory is non-modified-image block; If not, then the image block that definition present frame does not find in image block buffer memory is modified-image block.Modified-image block described in the following embodiment of the present invention and non-modified-image block all adopt above-mentioned define method, follow-uply will repeat no more.
Image block for change will be compressed by compression module, and data after compression and unchanged image block label be encapsulated by certain format, form final output data and be transferred to solution pressure side.In a preferred embodiment, described piecemeal rule keeps synchronous in described compression end and described solution pressure side and can switch according to embody rule scene, and described piecemeal rule comprises employing Image Edge-Detection and edge compensation carries out random length image block; Described hierarchical index adopts the index entry of stratification to organize described image block, low level index represents less image block, multiple low level index will be integrated in high-level index, make the image block that high-level index can represent larger, and the described index entry of most top layer represents whole described image block.
After step S02, described decompress(ion) termination receive described view data, after distinguishing the label of described modified-image block corresponding in described view data and non-modified-image block according to described piecemeal rule, be corresponding image block by described image data restoration; Merge described image block, show described desktop picture.
Pressure side image procossing block schematic illustration is separated described in remote desktop image transfer method of the present invention with reference to Fig. 3, Fig. 3; As shown in Figure 3, after decompress(ion) termination receives the view data of compression end transmission, first distinguish the image tag of non-modified-image block and the packed data of modified-image block according to above-mentioned piecemeal rule; Then, utilize this image tag and hierarchical index to be recovered by unchanged image block, and utilize the relevant decompression module of separating pressure side that packed data corresponding for modified-image block is carried out decompress(ion); Finally, merge non-modified-image block and modified-image block, show final desktop picture.Meanwhile, this hierarchical index used and piecemeal result are carried out buffer memory.
The present embodiment is by being some image blocks in compression end by desktop picture cutting, and build hierarchical index, hierarchical index is utilized to detect the modified-image block of the image block relative image block cache of present frame and non-modified-image block, obtain view data by encapsulating together with the label of described non-modified-image block after described modified-image block compression, and described view data is sent to solution pressure side; Decompress(ion) petiolarea divides modified-image block and non-modified-image block, and be corresponding image block by image data restoration, desktop picture is shown and the method for whole process compresses end reconciliation pressure side maintenance piecemeal regular and synchronized after being merged by above-mentioned image block, there is the beneficial effect optimizing remote desktop image transmitting, reduce transmitted data amount, improve image compression ratio.
Reference Fig. 4, Fig. 4 are the schematic flow sheets of compression end image procossing one embodiment described in remote desktop image transfer method of the present invention; As shown in Figure 4, in remote desktop image transfer method of the present invention, compression end is carried out image procossing and is comprised the following steps:
Step S11, according to described piecemeal rule, be some described image blocks by the present frame cutting of described desktop picture;
According to piecemeal rule, be several image blocks by the present frame cutting of desktop picture.Wherein, partition can be stress and strain model scheme also can be random length division or both mixing segment partition schemes, and is not limited to specific piecemeal rule.Reference Fig. 5, Fig. 5 are image block one embodiment window schematic diagrames in remote desktop image transfer method of the present invention; As shown in Figure 5 a for adopting stress and strain model scheme the present frame of desktop picture to be carried out piecemeal, as shown in Figure 5 b for adopting random length image block that the present frame of desktop picture is carried out piecemeal.Piecemeal rule described in the embodiment of the present invention comprises the image block technology for Desktop Windows feature.The step adopting the image block technology of Desktop Windows feature to carry out fragmental image processing is: adopt technique of image edge detection, extract forms edge; Adopt straight-line detection technology, detect the roughly framework of forms; Utilize the compensation technique at edge, repair because small part edge lacks the discontinuous forms edge caused; Described Image Edge-Detection is a kind of Feature Extraction Technology of image, can detect in image and change obvious point, can according to the foundation of the obvious point of change in this image detected as Iamge Segmentation.Straight-line detection is a kind of technology of recognition image cathetus.Combining image rim detection and straight-line detection technology identify forms, and as the foundation of image block, mainly consider the most rectangular form of the forms of computer graphic user interface.Piecemeal rule based on the partition of forms feature can detect the current forms (if user is just at the input frame of input character) that changing more accurately and static not change forms, thus acquisition does not better change segment testing result.
Step S12, according to described image block synthetic image fingerprint, and set up the hierarchical index of described image block;
According to above-mentioned image block synthetic image fingerprint, finger image has multiple use, and the memory location of such as positioning image block and checking image block have unchanged etc.Set up hierarchical index to the image block be partitioned into, upper layer index will represent more image block than lower floor index, and the index of final top layer will represent monoblock image.The effect of hierarchical index please refer to Fig. 6, and Fig. 6 is the window of desktop picture hierarchical index one embodiment described in remote desktop image transfer method of the present invention schematic diagram; As shown in Figure 6, hierarchical index adopts the index entry of stratification to organize described image block, index entry level represent the ascending of described image block respectively from low to high; The index entry of low level represents less image block, and the index entry of multiple low level will be integrated in high-level index, make the image block that high-level index entry can represent larger, and the index entry of final top layer will represent whole image block.
Step S13, utilize described finger image and hierarchical index, successively search for by top layer index and judge whether described image block can find in described image block buffer memory; If not, then described image block is modified-image block, performs step S14; If so, then described image block is non-modified-image block, performs step S15;
Step S14, described modified-image block to be compressed, obtain packed data;
The image tag of non-modified-image block described in step S15, record;
Utilize image block and hierarchical index technology, successively search for by top layer index, judge whether the image block of the present frame searched for can find in described image block buffer memory.If the image block of search can not find in image block buffer memory, illustrate that this image block is modified-image block, then this modified-image block is compressed, obtain packed data; If the image block of search can find in image block buffer memory, illustrate that this image block is non-modified-image block, then the image tag of non-modified-image block described in record.Described image tag is the unique identification of described image block, each image block image tag that correspondence one is different respectively.
As shown in Figure 7 and Figure 8, Fig. 7 identifies the image block one example structure schematic diagram that changes in remote desktop image transfer method of the present invention to the example of change detected image block; Fig. 8 is the schematic diagram of hierarchical index one example structure described in remote desktop image transfer method of the present invention; As shown in Figure 7 and Figure 8, the image block changed black image block represents, white image block is the image block do not changed.Particularly, in a preferred embodiment, as shown in Figure 8, the modified-image block found is " 1-1 ", " 1-2 ", " 1-3 ", " 2-3 ", " 2-4 ", and unchanged image block is " 1-4 " and " 2-2 ", the data therefore transmitted are: image block " 1-4 " and " 2-2 " label in image block buffer memory and the modified-image block after overcompression.Along with data volume scale increases, this programme adopts the high compression rate advantage of hierarchical index will be more obvious.
Step S16, encapsulate described packed data and described image tag, obtain the described view data after encapsulation and also described view data is sent to described solution pressure side.
After the image block changed being compressed by compression algorithm, encapsulate together with unchanged image block label and obtain total view data, this view data is exported to solution pressure side.
The present embodiment is by utilizing piecemeal rule to be some image blocks by desktop picture cutting in compression end, the hierarchical index built is utilized to detect the modified-image block of present frame and non-modified-image block, and after respective handling is carried out to image block, send to the method for separating pressure side, there is the beneficial effect obviously reducing volume of transmitted data.
Pressure side image procossing one embodiment schematic flow sheet is separated in remote desktop image transfer method of the present invention with reference to Fig. 9, Fig. 9; As shown in Figure 9, separate pressure side image procossing in remote desktop image transfer method of the present invention to comprise the following steps:
Step S21, receive described view data that described compression end sends after, distinguish described packed data and described image tag according to described piecemeal rule;
The piecemeal rule adopted in solution pressure side is consistent with the piecemeal rule of compression end, and in remote desktop image transfer method of the present invention, piecemeal rule keeps synchronous in solution pressure side with compression end; In order to allow the piecemeal rule of separating the clearly current use of pressure side, the head that label corresponding for current piecemeal rule is placed on packed data is transferred to solution pressure side by compression end.Decompress(ion) termination utilizes the packed data in current piecemeal result and hierarchical index differentiate between images data and image tag after receiving the view data of compression end transmission.
Step S22, to recover according to described hierarchical index and described image tag described in non-modified-image block, and packed data described in decompress(ion) recovers described modified-image block;
According to hierarchical index and image tag, be non-modified-image block by the image data restoration of non-modified-image block corresponding in view data, the packed data in view data reverted to corresponding modified-image block.Because the piecemeal rule and hierarchical index of separating pressure side keep synchronous with compression end, therefore, identical piecemeal rule must be consistent to the piecemeal result of phase diagram picture frame, then the image block recovered according to identical piecemeal rule is also correct certainly.
Step S23, merge described modified-image block and non-modified-image block, show described desktop picture, simultaneously piecemeal result and described hierarchical index described in buffer memory.
Modified-image block and non-modified-image block are merged, and at user side, the desktop picture that remote desktop transmits is shown; Meanwhile, this piecemeal result and hierarchical index are carried out buffer memory, under same scene, carry out the reference of remote desktop image transmitting as next time.The piecemeal of next frame image will continue to use the piecemeal result and hierarchical index that have obtained, until piecemeal rule is replaced.
The present embodiment divides modified-image block and non-modified-image block by decompress(ion) petiolarea, and be corresponding image block by image data restoration, desktop picture is shown and the method for whole process compresses end reconciliation pressure side maintenance piecemeal regular and synchronized after being merged by above-mentioned image block, have correctly, intactly recover the beneficial effect of remote desktop image, improve systematic function, improve Consumer's Experience.
Being that piecemeal described in remote desktop image transfer method of the present invention is regular with reference to Figure 10, Figure 10 carries out switching one embodiment schematic flow sheet according to embody rule scene; As shown in Figure 10, in remote desktop image transfer method of the present invention, piecemeal rule is carried out switching according to embody rule scene and is comprised the following steps:
Step S31, the described piecemeal result utilizing buffer memory and hierarchical index, detect the modified-image block of described present frame relative image block cache and non-modified-image block, and testing result transmitted;
Different piecemeal rules adapts to no application scenarios, and remote desktop image transfer method of the present invention can carry out the switching of piecemeal rule according to different application scenarioss; Adopt compression end to conciliate pressure side both-end synchronization mechanism simultaneously, make both-end have consistent hierarchical index, ensureing to separate pressure side can correct Recovery image block.Utilize the piecemeal result of buffer memory and hierarchical index to detect the modified-image block of present frame relative image block cache and non-modified-image block, and testing result is transferred to the downstream of compression end.
Step S32, judge whether Remote desk process stops; If so, then whole transmitting procedure terminates; If not, then step S33 is performed;
Step S33, collect present image information judge whether current scene meets piecemeal changing rules condition; If so, then step S34 is performed; If not, then step S35 is performed;
Step S34, according to described present image information, switch to new piecemeal rule;
Step S35, keep current piecemeal rule;
Carry out piecemeal rule the need of the judgement carrying out switching before, need to guarantee that remote desktop still connects in maintenance.If Remote desk process stops, then whole transmitting procedure terminates; If Remote desk process is still continuing, then collecting present image information, and judging whether current scene meets the switching condition of piecemeal rule; Described switching condition can be the CPU(CentralProcessing Unit of compression effectiveness fall flat, equipment, central processing unit) occupation rate reaches the maximum of default occupation rate or network delay beyond pre-set delay maximum etc.If current scene meets piecemeal changing rules condition, then according to present image information, be suitable new piecemeal rule by current piecemeal changing rules.If current scene does not meet piecemeal rule, the piecemeal rule namely is still applicable to current application scene, then continue to use current piecemeal rule, preserves current piecemeal result and hierarchical index; Because it is considerable for carrying out the computing cost that point block operations brings continually.
Step S36, utilize described new piecemeal rule to carry out piecemeal to described present frame, and replace existing described piecemeal result and hierarchical index.
After current scene meets piecemeal changing rules condition and switches to new piecemeal rule, compression end utilizes new piecemeal rule to carry out piecemeal to present frame, and replaces existing piecemeal result and hierarchical index.After compression end switches piecemeal rule, need to allow solution pressure side keep synchronous with its piecemeal rule.The piecemeal rule clearly needing to switch to allow solution pressure side, in a preferred embodiment, the head that label corresponding for current new piecemeal rule is placed on packed data is transferred to solution pressure side by compression end.
The present embodiment is by meeting piecemeal changing rules condition in current scene and after switching to new piecemeal rule, utilizing new piecemeal rule to carry out piecemeal to present frame; When not meeting piecemeal changing rules condition, continuing the method adopting former piecemeal rule, there is the beneficial effect intelligently adopting different piecemeal rule according to different application scene, dramatically saves on computing cost simultaneously.
Reference Figure 11, Figure 11 are remote desktop image delivering system one example structure schematic diagrames of the present invention; As shown in figure 11, remote desktop image delivering system of the present invention comprises:
Compression end 01, for the present frame of desktop picture being carried out piecemeal according to described piecemeal rule, and set up hierarchical index, according to hierarchical index, detect the modified-image block of the image block relative image block cache of described present frame and non-modified-image block, obtain view data by encapsulating together with the label of described non-modified-image block after described modified-image block compression, and described view data is sent to solution pressure side;
With reference to compression end 01 image procossing block schematic illustration described in Fig. 2 remote desktop image transfer method of the present invention, first the cutting of current desktop image according to certain piecemeal rule, is some image blocks, and sets up hierarchical index by compression end 01; The modified-image block of present frame and non-modified-image block is detected according to described hierarchical index.Image block for change will be compressed by compression module, and data after compression and unchanged image block label be encapsulated by certain format, form final output data and be transferred to solution pressure side 02.In a preferred embodiment, described piecemeal rule keeps synchronous in described compression end 01 and described solution pressure side 02 and can switch according to embody rule scene, and described piecemeal rule comprises employing Image Edge-Detection and edge compensation carries out random length image block; Described hierarchical index adopts the index entry of stratification to organize described image block, low level index represents less image block, multiple low level index will be integrated in high-level index, make the image block that high-level index can represent larger, and the described index entry of most top layer represents whole described image block.
Separating pressure side 02, after receiving described view data, according to described piecemeal rule, after distinguishing the label of described modified-image block corresponding in described view data and non-modified-image block, is corresponding image block by described image data restoration; Merge described image block, show described desktop picture.
Pressure side 02 image procossing block schematic illustration is separated described in Fig. 3 remote desktop image transfer method of the present invention, after solution pressure side 02 receives the view data of compression end 01 transmission, first distinguish the image tag of non-modified-image block and the packed data of modified-image block according to above-mentioned piecemeal rule; Then, utilize this image tag and hierarchical index to be recovered by unchanged image block, and utilize the relevant decompression module of separating pressure side 02 that packed data corresponding for modified-image block is carried out decompress(ion); Finally, merge non-modified-image block and modified-image block, show final desktop picture.Meanwhile, this hierarchical index used and piecemeal result are carried out buffer memory.
Whether the present embodiment, by being some image blocks in compression end by desktop picture cutting, detecting present frame relative image block cache and changes and send to solution pressure side after carrying out respective handling to image block; Decompress(ion) petiolarea divides modified-image block and non-modified-image block, and be corresponding image block by image data restoration, desktop picture is shown and whole process compresses end reconciliation pressure side maintenance piecemeal regular and synchronized after being merged by above-mentioned image block, there is the beneficial effect optimizing remote desktop image transmitting, reduce transmitted data amount, improve image compression ratio.
Reference Figure 12, Figure 12 are the schematic diagrames of compression end first example structure described in remote desktop image delivering system of the present invention; The present embodiment is only further described compression end 01, and other modules involved by remote desktop image delivering system of the present invention please refer to the specific descriptions of above-described embodiment, do not repeat them here.
As shown in figure 12, in remote desktop image delivering system of the present invention, compression end 01 specifically comprises:
The present frame cutting of described desktop picture, for according to described piecemeal rule, is some image blocks by image block module 011;
According to piecemeal rule, the present frame cutting of desktop picture is several image blocks by image block module 011.Wherein, partition can be stress and strain model scheme also can be random length division or both mixing segment partition schemes, and is not limited to specific piecemeal rule.With reference to image block one embodiment window schematic diagram in Fig. 5 remote desktop image transfer method of the present invention; As shown in Figure 5 a, image block module 011 adopts stress and strain model scheme that the present frame of desktop picture is carried out piecemeal, and as shown in Figure 5 b, image block module 011 adopts random length image block that the present frame of desktop picture is carried out piecemeal.Piecemeal rule described in the embodiment of the present invention comprises the image block technology for Desktop Windows feature.Forms edge one embodiment window schematic diagram is obtained by Image Edge-Detection and straight-line detection with reference in Fig. 6 remote desktop image transfer method of the present invention, the step that image block module 011 adopts the image block technology of Desktop Windows feature to carry out fragmental image processing is: adopt technique of image edge detection, extract forms edge; Adopt straight-line detection technology, detect the roughly framework of forms; Utilize the compensation technique at edge, repair because small part edge lacks the discontinuous forms edge caused; Image block module 011 please refer in Fig. 7 remote desktop image transfer method of the present invention through the piecemeal effect that edge compensation obtains and obtains piecemeal effect one embodiment window schematic diagram by edge compensation.Image block module 011 combining image rim detection and straight-line detection technology identify forms, and as the foundation of image block, mainly consider the most rectangular form of the forms of computer graphic user interface.Piecemeal rule based on the partition of forms feature can detect the current forms (if user is just at the input frame of input character) that changing more accurately and static not change forms, thus acquisition does not better change segment testing result.
Hierarchical index sets up module 012, for according to described image block synthetic image fingerprint, and sets up the hierarchical index of described image block;
Hierarchical index sets up the image block synthetic image fingerprint that module 012 obtains according to image block module 011, and finger image has multiple use, and the memory location of such as positioning image block and checking image block have unchanged etc.Hierarchical index is set up module 012 and is set up hierarchical index to the image block be partitioned into, and upper layer index will represent more image block than lower floor index, and the index of final top layer will represent monoblock image.The effect of hierarchical index please refer to the schematic diagram of desktop picture hierarchical index one embodiment window described in Fig. 6 remote desktop image transfer method of the present invention, hierarchical index adopts the index entry of stratification to organize described image block, index entry level represent the ascending of described image block respectively from low to high; The index entry of low level represents less image block, and the index entry of multiple low level will be integrated in high-level index, make the image block that high-level index entry can represent larger, and the index entry of final top layer will represent whole image block.
Image change identification module 013, for utilizing described finger image and hierarchical index, successively searching for and judging whether described image block can find in described image block buffer memory by top layer index; If so, then described image block is non-modified-image block; If not, then described image block is modified-image block;
Image change identification module 013 utilizes image block and hierarchical index technology, successively searches for by top layer index, judges whether whether the image block searched can find in described image block buffer memory; If can find in image block buffer memory, then illustrate that this image block is non-modified-image block; If can not find in image block buffer memory, then illustrate that this image block is modified-image block.As shown in Figure 7 and Figure 8, the image block changed black image block represents the example of image change identification module 013 change detected image block, and white image block is the image block do not changed.Particularly, in a preferred embodiment, as shown in Figure 8, the modified-image block found is " 1-1 ", " 1-2 ", " 1-3 ", " 2-3 ", " 2-4 ", and unchanged image block is " 1-4 " and " 2-2 ", the data therefore transmitted are: the label that image block " 1-4 " and " 2-2 " are corresponding in image block buffer memory and the modified-image block after overcompression.Along with data volume scale increases, this programme adopts the high compression rate advantage of hierarchical index will be more obvious.
Image processing module 014, for when described image block is modified-image block, compresses described modified-image block, obtains packed data; When described image block is not for sending out modified-image block, the image tag of non-modified-image block described in record;
If image change identification module 013 detects modified-image block, then this modified-image block compresses by image processing module 014, obtains packed data; If image change identification module 013 detects non-modified-image block, then image processing module 014 record described in the image tag of non-modified-image block.Described image tag is the unique identification of described image block, each image block image tag that correspondence one is different respectively.
View data output module 015, for encapsulating described packed data and described image tag, obtaining the described view data after encapsulation and described view data is sent to described solution pressure side.
The packed data that image processing module 014 is obtained by lossless compression algorithm by view data output module 015 encapsulates together with the image tag of the non-modified-image block of record, obtains total view data, and this view data is exported to solution pressure side 02.
The present embodiment by utilizing piecemeal rule to be some image blocks by desktop picture cutting in compression end, and builds hierarchical index, utilizes hierarchical index to detect the image block of present frame in the situation of image block buffer memory, for non-modified-image block, need only transmit its label; For modified-image block, then send to solution pressure side after it being carried out compression by compression algorithm and after the Tag Packaging of non-modified-image block, there is the beneficial effect obviously reducing volume of transmitted data.
Pressure side one example structure schematic diagram is separated described in remote desktop image delivering system of the present invention with reference to Figure 13, Figure 13; The present embodiment is only further described solution pressure side 02, and other modules involved by remote desktop image delivering system of the present invention please refer to the specific descriptions of related embodiment, do not repeat them here.
As shown in figure 13, in remote desktop image delivering system of the present invention, separate pressure side 02 and specifically comprise:
Image area sub-module 021, after receiving described view data that described compression end sends, distinguishes described packed data and described image tag according to described piecemeal rule;
The piecemeal rule adopted in solution pressure side 02 is consistent with the piecemeal rule of compression end 01, and in remote desktop image transfer method of the present invention, piecemeal rule keeps synchronous in solution pressure side 02 with compression end 01; In order to allow the piecemeal rule of separating the clearly current use of pressure side 02, the head that label corresponding for current piecemeal rule is placed on packed data is transferred to solution pressure side by compression end 01.Separate after the image area sub-module 021 of pressure side 02 receives the view data that compression end 01 sends, utilize the packed data in current piecemeal result and hierarchical index differentiate between images data and image tag.
Image-restoration module 022, for non-modified-image block according to described hierarchical index and the recovery of described image tag, and packed data described in decompress(ion) recovers described modified-image block;
According to hierarchical index and image tag, the image data restoration of non-modified-image block corresponding in view data is non-modified-image block by image-restoration module 022, the packed data in view data is reverted to corresponding modified-image block.Because the piecemeal rule and hierarchical index of separating pressure side 02 keep synchronous with compression end 01, therefore, identical piecemeal rule must be consistent to the piecemeal result of phase diagram picture frame, then the image block that image-restoration module 022 is recovered according to identical piecemeal rule is also correct certainly.
Image display 023, for merging described modified-image block and non-modified-image block, shows described desktop picture, simultaneously piecemeal result and described hierarchical index described in buffer memory.
Modified-image block and non-modified-image block merge by image display 023, and are shown by the desktop picture that remote desktop transmits at user side; Meanwhile, this piecemeal result and hierarchical index are carried out buffer memory by image display 023, carry out the reference of remote desktop image transmitting as next time under same scene.The piecemeal of next frame image will continue to use the piecemeal result and hierarchical index that have obtained, until piecemeal rule is replaced.
The present embodiment divides modified-image block and non-modified-image block by decompress(ion) petiolarea, and be corresponding image block by image data restoration, desktop picture is shown and whole process compresses end reconciliation pressure side maintenance piecemeal regular and synchronized after being merged by above-mentioned image block, have correctly, intactly recover the beneficial effect of remote desktop image, improve systematic function, improve Consumer's Experience.
Reference Figure 14, Figure 14 are the schematic diagrames of compression end second example structure described in remote desktop image delivering system of the present invention; The difference of the present embodiment and described compression end first embodiment is, increase only piecemeal rule and arranges module 016; The present embodiment only arranges module 016 to piecemeal rule and is described specifically, and other modules involved by remote desktop image delivering system of the present invention please refer to the specific descriptions of related embodiment, do not repeat them here.
As shown in figure 14, in remote desktop image delivering system of the present invention, compression end 01 also comprises:
Piecemeal rule arranges module 016, for keeping described piecemeal regular and synchronized in described compression end and described solution pressure side and can switching described piecemeal rule according to embody rule scene; Described piecemeal rule comprises employing Image Edge-Detection and edge compensation carries out random length image block.
Different piecemeal rules adapts to no application scenarios, and piecemeal rule arranges module 016 carries out piecemeal rule switching according to different application scenarioss; Adopt compression end 01 to conciliate pressure side 02 both-end synchronization mechanism simultaneously, make both-end have consistent hierarchical index, ensureing to separate pressure side 02 can correct Recovery image block.
Arrange module one example structure schematic diagram with reference to piecemeal rule described in Figure 15 remote desktop image delivering system of the present invention, piecemeal rule of the present invention arranges module 016 and specifically comprises:
Image block detecting unit 161, for utilizing described piecemeal result and the hierarchical index of buffer memory, detecting the modified-image block of described present frame relative image block cache and non-modified-image block, and testing result is transferred to described compression end;
Image block detecting unit 161 utilizes the piecemeal result of buffer memory and hierarchical index to detect the modified-image block of present frame relative image block cache and non-modified-image block, and testing result is transferred to image information collector unit 162.
Image information collector unit 162, for judging whether Remote desk process stops, and when described Remote desk process does not stop, collecting present image information and judging whether current scene meets piecemeal changing rules condition;
Image information collector unit 162 carry out piecemeal rule the need of the judgement carrying out switching before, need to guarantee that remote desktop still connects in maintenance.If Remote desk process stops, then whole transmitting procedure terminates; If Remote desk process is still continuing, then image information collector unit 162 is collecting present image information, and judges whether current scene meets the switching condition of piecemeal rule; The maximum that described switching condition can be compression effectiveness fall flat, the CPU occupation rate of equipment reaches default occupation rate or network delay are beyond pre-set delay maximum etc.
Piecemeal changing rules unit 163, during for meeting piecemeal changing rules condition in current scene, according to described present image information, switches to new piecemeal rule;
If image information collector unit 162 judges that current scene meets piecemeal changing rules condition, then piecemeal changing rules unit 163 is according to present image information, is suitable new piecemeal rule by current piecemeal changing rules.If current scene does not meet piecemeal rule, namely the piecemeal rule before is still applicable to current application scene, then piecemeal changing rules unit 163 does not carry out the switching of piecemeal rule, and compression end 01 continues to use current piecemeal rule, and preserves current piecemeal result and hierarchical index; Because it is considerable for carrying out the computing cost that point block operations brings continually.
Hierarchical index replacement unit 164, for utilizing described new piecemeal rule to carry out piecemeal to described present frame, and replaces existing described piecemeal result and hierarchical index.
Judge that current scene meets piecemeal changing rules condition and after switching to new piecemeal rule, hierarchical index replacement unit 164 utilizes new piecemeal rule to carry out piecemeal to present frame, and replaces existing piecemeal result and hierarchical index at piecemeal changing rules unit 163.
Utilize piecemeal rule to arrange after module 016 switches piecemeal rule in compression end 01, need to allow and separate pressure side 02 and keep synchronous with its piecemeal is regular.The piecemeal rule clearly needing to switch to allow solution pressure side 02, in a preferred embodiment, the head that label corresponding for current new piecemeal rule is placed on packed data is transferred to solution pressure side 02 by compression end 01.
The present embodiment is by meeting piecemeal changing rules condition in current scene and after switching to new piecemeal rule, utilizing new piecemeal rule to carry out piecemeal to present frame; When not meeting piecemeal changing rules condition, continuing to adopt former piecemeal rule, there is the beneficial effect intelligently adopting different piecemeal rule according to different application scene, dramatically saves on computing cost simultaneously.
The foregoing is only the preferred embodiments of the present invention; not thereby its scope of the claims is limited; every utilize specification of the present invention and accompanying drawing content to do equivalent structure or equivalent flow process conversion; directly or indirectly be used in the technical field that other are relevant, be all in like manner included in scope of patent protection of the present invention.