CN108897596A - A kind of graphical interfaces transmission method and device - Google Patents
A kind of graphical interfaces transmission method and device Download PDFInfo
- Publication number
- CN108897596A CN108897596A CN201810729065.8A CN201810729065A CN108897596A CN 108897596 A CN108897596 A CN 108897596A CN 201810729065 A CN201810729065 A CN 201810729065A CN 108897596 A CN108897596 A CN 108897596A
- Authority
- CN
- China
- Prior art keywords
- update area
- application
- area data
- remote
- window
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/451—Execution arrangements for user interfaces
Abstract
This application provides a kind of graphical interfaces transmission method and devices, are applied to remote server, including：Remote application sends the render instruction for drawing the corresponding update area data of remote application to X graphics server module after the control instruction that response receives；Update area data include window metadata and windows content data；X graphics server module draws the corresponding update area data of remote application after receiving render instruction, according to render instruction；Application widget issuing service module obtains update area data, and update area data are sent to terminal device, and update area data are depicted as video in window so that terminal device is based on native window format.In this way, remotely answering the video in window of program consistent with local application, the seamless localized experience of user is realized.Meanwhile remote server without screenshotss and sends entire desktop screenshotss file, reduces the server resource of occupancy, ensure that the safety of remote application.
Description
Technical field
This application involves computer fields, in particular to the transmission method and device of a kind of graphical interfaces.
Background technique
Currently, user is the application checked or controlled by network on local client Yun Hang on the remote server
Program, in this way, user is facilitated remotely to carry out maintenance and management to server local.
Provide in the related technology it is a kind of based on Virtual Networking Computing technology (Virtual Network Computing,
VNC) the method for agreement transmission server graphical interfaces, it is related to the screen of remote server operate above it is all long-range
Application widget carries out screenshotss and is made into image, and is transmitted by remote framebuffer (Remote Frame Buffer, RFB) agreement
To local client, so that user controls remote application on local client.
But the transmission of VNC agreement be server remote desktop (hereinafter referred to as X desktop), it is generally the case that local visitor
The operating system of family end and server is all difference, this allows for user and operates on local client locally applied and remotely answer
Used time will switch between two entirely different operating systems back and forth, be unable to reach seamless localized experience.Meanwhile screenshotss
Need to expend biggish central processing unit (CPU, Central Processing Unit) resource.
Summary of the invention
In view of this, the embodiment of the present application is designed to provide a kind of graphical interfaces transmission method and device, it is long-range to take
The update area data of remote application are sent to terminal device by business device, are sent compared to screenshotss, and the CPU money of occupancy is reduced
Source, meanwhile, update area data can be depicted as the video in window of native window style by terminal device, and it is seamless to realize user
Localized experience.
In a first aspect, the embodiment of the present application provides a kind of graphical interfaces transmission method, it is applied to remote server, it is described
Remote server includes remote application, X graphics server module and application widget issuing service module, the method packet
It includes：
The remote application sends rendering to X graphics server module and refers to after the control instruction that response receives
It enables；Wherein, the render instruction is for drawing the corresponding update area data of the remote application；The update area number
According to including window metadata and windows content data；
The X graphics server module is drawn described remote after receiving the render instruction according to the render instruction
The corresponding update area data of journey application program；
The application widget issuing service module obtains the update area data, and the update area data are sent
To terminal device, the update area data are depicted as video in window so that the terminal device is based on native window format.
With reference to first aspect, the embodiment of the present application provides the first possible embodiment of first aspect, wherein institute
Stating remote server includes application widget frame buffer module；The X graphics server module draws institute according to the render instruction
After stating the corresponding update area data of remote application, further include：
The update area data are sent to the application widget frame buffer module by the X graphics server module；
The application widget frame buffer module receives and stores the update area data；
The application widget issuing service module obtains the update area data, and the update area data are sent
To terminal device, including：
The application widget issuing service module grabs the update area number from the application widget frame buffer module
According to being encoded to the windows content data in the update area data, and to windows content data after coding and described
Window metadata in update area data carries out compression processing, and obtained compressed data packets are sent to the terminal and are set
It is standby.
With reference to first aspect or the first possible embodiment of first aspect, the embodiment of the present application provide first
The possible embodiment of second of aspect, wherein the application widget issuing service module sends out the update area data
It send to terminal device, further includes：
If the time interval for detecting the feedback information that the terminal device returns is more than preset threshold, according to preset
Step-length reduces the acquisition frequency of the update area data and sends frequency.
With reference to first aspect, the embodiment of the present application provides the third possible embodiment of first aspect, wherein institute
It states after the update area data are sent to terminal device by application widget issuing service module, further includes：
If receiving the remote application that the terminal device is sent not in the state notification message of front stage operation,
Then stop obtaining and sending the update area data.
With reference to first aspect, the embodiment of the present application provides the 4th kind of possible embodiment of first aspect, wherein institute
It states after the update area data are sent to terminal device by application widget issuing service module, further includes：
, will be described after if the application widget issuing service module receives the control instruction that the terminal device is sent
Control instruction is forwarded to the X graphics server module；Wherein, it is carried in the control instruction and the remote application
Corresponding application identification information；
The X graphics server module is searched corresponding with the application identification information after receiving the control instruction
Remote application, and the control instruction is forwarded to the remote application found, is remotely answered so as to described
The control instruction is responded with program.
Second aspect, the embodiment of the present application also provides a kind of graphical interfaces transmission methods, are applied to terminal device, described
Method includes：
Receive the corresponding update area data of remote application that remote server is sent；Wherein, the update area
Data include window metadata and windows content data；
According to the window metadata and the windows content data and native window in the update area data
Format draws the corresponding video in window of the remote application；
Show the video in window.
In conjunction with second aspect, the embodiment of the present application provides the first possible embodiment of second aspect, wherein institute
After stating the display video in window, further include：
If detecting the duplicate instructions for the remote application, by the corresponding reproducting content of the duplicate instructions
It is stored on local pasting boards；
When receiving the stickup instruction for the remote application, extracted from the local pasting boards described multiple
Content processed, and the reproducting content is sent on the long-range pasting boards in the remote server, so as to the remote service
Remote application in device is pasted based on the reproducting content in the long-range pasting boards.
In conjunction with the first possible embodiment of second aspect, the embodiment of the present application provides second of second aspect
Possible embodiment, wherein it is described receive remote server send the corresponding update area data of remote application it
Afterwards, further include：
The corresponding update area data of the remote application are unziped it and decoding process, extracts the update
The window metadata and the windows content data in area data.
The third aspect, the embodiment of the present application also provides a kind of graphical interfaces transmitting devices, including：
Remote application, for sending rendering to X graphics server module and referring to after the control instruction that response receives
It enables；Wherein, the render instruction is for drawing the corresponding update area data of the remote application；The update area number
According to including window metadata and windows content data；
X graphics server module, for being drawn according to the render instruction described remote after receiving the render instruction
The corresponding update area data of journey application program；
Application widget issuing service module is sent out for obtaining the update area data, and by the update area data
It send to terminal device, the update area data is depicted as window figure so that the terminal device is based on native window format
Picture.
Fourth aspect, the embodiment of the present application also provides a kind of graphical interfaces transmitting devices, including：
Receiving module, for receiving the corresponding update area data of remote application of remote server transmission；Wherein,
The update area data include window metadata and windows content data；
Drafting module, for according to the window metadata and the windows content data in the update area data
And native window format, draw the corresponding video in window of the remote application；
Display module, for showing the video in window.
A kind of graphical interfaces transmission method and device provided by the embodiments of the present application, remote server includes remote application journey
Sequence, X graphics server module and application widget issuing service module are based on this, the control that remote application is received in response
After system instruction, render instruction is sent to X graphics server module in server internal, X graphics server module is based on rendering and refers to
It enables and draws the corresponding update area data of remote application, application widget issuing service module then sends update area data
To terminal device.The above process reduces the CPU money of occupancy compared to the method that server screenshotss send entire desktop picture
Source, memory source and Internet resources, meanwhile, server only transmits update area data to terminal device, also ensures terminal
The user of equipment side can only operate single application program, and menu and console without operating entire server improve
The safety of server end.On the other hand, update area data can be depicted as window based on native window format by terminal device
Mouth image, in this way, remote application and locally applied windowed format are unified on the terminal device, user is directed on the terminal device
Locally applied and remote application operation without switching between two entirely different operating systems back and forth, realize user without
The localized experience of seam.
To enable the above objects, features, and advantages of the application to be clearer and more comprehensible, preferred embodiment is cited below particularly, and cooperate
Appended attached drawing, is described in detail below.
Detailed description of the invention
Technical solution in ord to more clearly illustrate embodiments of the present application, below will be to needed in the embodiment attached
Figure is briefly described, it should be understood that the following drawings illustrates only some embodiments of the application, therefore is not construed as pair
The restriction of range for those of ordinary skill in the art without creative efforts, can also be according to this
A little attached drawings obtain other relevant attached drawings.
Fig. 1 shows a kind of structural schematic diagram of graphical interfaces Transmission system provided by the embodiment of the present application.
Fig. 2 shows the application scenarios schematic diagrams of another kind graphical interfaces Transmission system provided by the embodiment of the present application.
Fig. 3 shows a kind of flow chart of graphical interfaces transmission method provided by the embodiment of the present application.
Fig. 4 shows the flow chart of another kind graphical interfaces transmission method provided by the embodiment of the present application.
Fig. 5 shows the flow chart of another kind graphical interfaces transmission method provided by the embodiment of the present application.
Fig. 6 shows the flow chart of another kind graphical interfaces transmission method provided by the embodiment of the present application.
Fig. 7 shows the flow chart of another kind graphical interfaces transmission method provided by the embodiment of the present application.
Fig. 8 shows a kind of structural schematic diagram of graphical interfaces transmitting device provided by the embodiment of the present application.
Fig. 9 shows the structural schematic diagram of another kind graphical interfaces transmitting device provided by the embodiment of the present application.
Figure 10 shows the structural schematic diagram of computer equipment 60 provided by one embodiment of the application.
Figure 11 shows the structural schematic diagram of computer equipment 70 provided by another embodiment of the application.
Specific embodiment
To keep the purposes, technical schemes and advantages of the embodiment of the present application clearer, below in conjunction with the embodiment of the present application
Middle attached drawing, the technical scheme in the embodiment of the application is clearly and completely described, it is clear that described embodiment is only
It is some embodiments of the present application, instead of all the embodiments.The application being usually described and illustrated herein in the accompanying drawings is real
The component for applying example can be arranged and be designed with a variety of different configurations.Therefore, below to the application's provided in the accompanying drawings
The detailed description of embodiment is not intended to limit claimed scope of the present application, but is merely representative of the selected reality of the application
Apply example.Based on embodiments herein, those skilled in the art institute obtained without making creative work
There are other embodiments, shall fall in the protection scope of this application.
Currently, VNC agreement and its similar agreement can only in the method based on VNC agreement transmission server graphical interfaces
Issue X desktop, after the client connection provided by these agreements, shown on user's local terminal one it is long-range
Desktop.This mode, is primarily present following problems：
A. user experience problem：User will two entirely different styles two operating systems (native operating sys-tern and
Remote desktop) between switch, operate remote application when, need to be switched to X desktop, be unable to reach seamless localized experience.
B. machine resources consumption is big：In VNC agreement, server is by screenshotss mode, by what is changed in remote desktop
Client is sent to after compressing after the screenshotss of part using certain compression algorithm to show.1) screenshotss need biggish server CPU money
Source expense；2) screenshotss are carried out to the part of entire desktop variation, rather than screenshotss is carried out to the application that user currently pays close attention to, caused
Each update area is bigger than the region that application actually updates, and when compressing and transmitting, needs bigger CPU, memory source and net
Network resource overhead.
C. safety, manageability problem：Publisher server is applied for what more people shared, user passes through long-range X desktop,
It may have access to the menu and console other than application.Destructive procedure, which is carried out, for user provides chance.
Based on this, the embodiment of the present application provides a kind of graphical interfaces Transmission system, as depicted in figs. 1 and 2, including terminal
Equipment 1 and remote server 2；Wherein, operation has JHApp client modules 10 in terminal device 1, and running in remote server has
JHApp server module 20 and remote application 21 (namely remote application module) include X in JHApp server module 20
Graphics server module 201, application widget frame buffer module 203 and application widget issuing service module 202.JHApp client
Module 10 operates on the terminal device 1 of user local, operating system can for Windows, Linux, Unix, Android or
All kinds of client operating systems such as iOS；JHApp server module 20 operates on remote server 2, and operating system can be with
All kinds of server operating systems of Linux, Unix, Android or iOS.Under normal conditions, 10 He of JHApp client modules
The operating system of JHApp server module 20 is different.
Drawn in the embodiment of the present application, in remote server the corresponding update area data of remote application and by this more
New district numeric field data is sent to terminal device after coding compression, and terminal device uses this after receiving update area data
Ground window style carries out application widget reconstruction (namely the video in window including content for repainting update area data), this
The windowed format of sample, locally applied and remote application on the terminal device is unified, and user is directed to local answer on the terminal device
With and remote application operation without back and forth switching between two entirely different operating systems, give the completely seamless sheet of user
Groundization experience.In addition, user is only capable of using being distributed to his application, and the menu on remote server and control are not available it
Platform improves manageability and safety.It is described below by embodiment.
As shown in figure 3, the application first embodiment provides a kind of graphical interfaces transmission method, it is applied to remote service
Device, the remote server include remote application, X graphics server module and application widget issuing service module, described
Method includes：
S101, the remote application send wash with watercolours after the control instruction that response receives, to X graphics server module
Have a finger in every pie order；Wherein, the corresponding update area data of the remote application are carried in the render instruction；The more new district
Numeric field data includes the content-data in window data and the corresponding window of the window data.
In the embodiment of the present application, before remote application starting, start JHApp server module, X graphics server
A current unappropriated display number (i.e. DISPLAY) can be automatically opened, and application widget issuing service module can be beaten automatically
Open the port of a receivable JHApp client modules connection.Application widget issuing service module is infused to X graphics server module
Volume Hook Function (namely call back function), for obtaining the drawing result data of X graphics server module.User or automatic script
Set DISPLAY=on the remote server<Display number>, and start remote application (i.e. application program), it grasps
Making system will be automatically the X graphics server in all graphical display guiding JHApp server modules of the remote application.
Wherein, the control instruction of remote application response can be administrator in server external trigger (as remotely
The enabled instruction of application program, also as controlling the instruction that mouse slides into the lower right corner from the upper left corner of desktop), it can also be clothes
(the timing screen protection of such as desktop) for device in house software automatic trigger of being engaged in.Wherein, the control instruction of administrator's triggering can be management
What member triggered on the remote server, it is also possible to what administrator was sent by terminal device.
Graphical interfaces after a remote application control instruction of every response can mutually change than before, therefore, far
After the control instruction that the every response of journey application program receives, render instruction will be sent to X graphics server module, so as to X figure
Shape server module draws the corresponding update area data of the remote application according to render instruction.
S102, the X graphics server module draw institute after receiving the render instruction, according to the render instruction
State the corresponding update area data of remote application.
In the embodiment of the present application, X graphics server is after receiving the render instruction for drawing update area data, root
The corresponding update area data of the remote application are drawn according to the render instruction.Wherein, the update area data tool
Body includes in the metadata (specific hierarchical relationship such as between window size, window name, the window's position, window) and window of window
Hold data (specific such as window modification region)
In the embodiment of the present application, the update area data after the instruction of remote application response control are taken in remote application
Rendered on business device, therefore, the connection of user's local client and remote server be it is stateless, therefore, in user local
Client, can disconnect or reconnection at any time, without the operating status of application.In this way, in the big or unstable network environment that is delayed,
It is not affected using the speed of service and reliability.
S103, the application widget issuing service module obtain the update area data, and by the update area number
According to being sent to terminal device, so as to the terminal device be based on native window format and be depicted as the update area data include
The video in window of content.
Here, application widget issuing service module is in advance to X graphics server Module registers Hook Function (i.e. readjustment letter
Number), call back function can monitor the update area data of X graphics server module generation in real time, once monitor update area number
According to will grab the update area Data Concurrent monitored and send to application widget issuing service module.
Application widget issuing service module sends out the update area data after obtaining above-mentioned update area data
It send to terminal device, and the terminal device is after receiving update area data, it is described more based on native window format general
New district numeric field data is depicted as including that the video in window of content draws the video in window of the remote application, in this way, in terminal
Remote application and locally applied windowed format are unified in equipment, and user is directed to locally applied and remote application on the terminal device
Operation without switching between two entirely different operating systems back and forth, realize the seamless localized experience of user.
Graphical interfaces transmission method in the embodiment of the present application sends entire desktop picture compared to remote server screenshotss
Method, reduce the cpu resource, memory source and Internet resources of occupancy, meanwhile, server only to terminal device transmission more
New district numeric field data also ensures that the user of terminal equipment side can only operate single application program, without operating entire service
The menu and console of device, improve the safety of server end.
Further, as shown in figure 4, in graphical interfaces transmission method provided by the embodiments of the present application, the remote service
Device includes application widget frame buffer module；Step 102, the X graphics server module is according to render instruction drafting
After the corresponding update area data of remote application, further include：
The update area data are sent to the application widget frame buffer mould by S201, the X graphics server module
Block.
S202, the application widget frame buffer module receive and store the update area data.
In conjunction with step 201~step 202, in the embodiment of the present application, X graphics server module is drawing update area number
According to later, update area data can't be sent to display, but be sent to application widget frame buffer module, by application widget
Frame buffer module receives and stores update area data.
Step 103, the application widget issuing service module obtains the update area data, and by the update area
Data are sent to terminal device, including：
The application widget issuing service module grabs the update area number from the application widget frame buffer module
According to being encoded to the windows content data in the update area data, and to windows content data after coding and described
Window metadata in update area data carries out compression processing, and obtained compressed data packets are sent to the terminal and are set
It is standby.
In the embodiment of the present application, when the JHApp client modules in terminal device pass through transmission control protocol
Interconnected between (Transmission Control Protocol, TCP) TCP/ network agreement (Internet Protocol,
IP after) agreement is connected to the application widget issuing service module of remote server, application widget issuing service module (is specially answered
With window issuing service module to the call back function of X graphics server Module registers) with (such as every 20 milliseconds of crawls of preset rate
The rate of one frame data packet) update area data are grabbed from application widget frame buffer module, and to the window of each crawl
Content-data is encoded, then by the windows content data after coding and together with window metadata (such as window name, size, position
Set, the hierarchical relationship between window) together compress after, put to downstream data message to column；Application widget issuing service module is pressed
Downstream data message is added to the chronological order in column, successively by downstream data message to the pressure in column according to compressed data packets
Contracting data packet is sent to the JHApp client modules, and JHApp client modules are after receiving compressed data packets, to described
Compressed data packets are unziped it and are decoded, and are drawn the video in window of remote application and showed user.
The X graphics server module that the application starts a light weight on the remote server is connect by memory switching technology
The window and its content render instruction of remote application are received, and window and figure rendering framing are stored in memory.It uses
Hook Technique, it is automatic to obtain update area data, only update area data will be needed to take out from memory, be sent by coding compression
Redrawing for video in window is carried out to client, to reach the interactive experience of low latency, low CPU overhead.
Further, in graphical interfaces transmission method provided by the embodiments of the present application, step 103, the application widget hair
After the update area data are sent to terminal device by cloth service module, further include：
If the time interval for detecting the feedback information that the terminal device returns is more than preset threshold, according to preset
Step-length reduces the acquisition frequency of the update area data and sends frequency.
In the embodiment of the present application, the JHApp client modules in terminal device are from JHApp service in remote server
After any compressed data packets that device module is sent, feedback information can be returned to JHApp server module module, if
JHApp server module module judges from the time interval for sending compressed data packets to the feedback information for receiving the compressed data packets
Greater than preset threshold, correspondingly, application widget issuing service module grabs update area data and sends update area data
Frequency is constant, then, it will cause the compressed data packets serious accumulation blocking in down queue, therefore, JHApp server module mould
Block adjusts the rate of application widget issuing service module crawl update area data and sends the speed of the update area data
Rate, in this way, solving the problems, such as to overstock compressed data packets to column according to downstream data message.
Further, in graphical interfaces transmission method provided by the embodiments of the present application, step 103, the application widget hair
After the update area data are sent to terminal device by cloth service module, further include：
If receiving the remote application that the terminal device is sent not in the state notification message of front stage operation,
Then stop obtaining and sending the update area data.
In the embodiment of the present application, terminal device (such as remote application after detecting foreground not running remote application
After program minimizes on the terminal device or opens connection with terminal), by not transported on foreground for remote application
Capable state notification message is sent to the application widget issuing service module in remote server, application widget issuing service module
After receiving state notification message, stops crawl or send modification data, in such manner, it is possible to by central processing unit (Central
Processing Unit, CPU) network overhead minimize.
Further, as shown in figure 5, in graphical interfaces transmission method provided by the embodiments of the present application, the application widget
After the update area data are sent to terminal device by issuing service module, further include：
After if S301, the application widget issuing service module receive the control instruction that the terminal device is sent,
The control instruction is forwarded to the X graphics server module；Wherein, it carries in the control instruction and is remotely answered with described
With the corresponding application identification information of program.
S302, the X graphics server module are searched and the application identification information after receiving the control instruction
Corresponding remote application, and the control instruction is forwarded to the remote application found, so as to described remote
Control instruction described in journey application response.
In the embodiment of the present application, in conjunction with step 301~step 302, JHApp client is responsible for from downstream data message to column
Compressed data packets are received, and compressed data packets are decompressed and decoded, being then based on native window format will the more new district
Numeric field data is depicted as video in window.
Meanwhile JHApp client is responsible for obtaining user's control instruction (such as mouse, keyboard, pasting boards operation), and will control
Instruction is sent in the upstream data Message queue of remote server.Application widget issuing service module is responsible for disappearing from upstream data
Breath is forwarded to X graphics server to the control instruction that obtains control instruction in column, and will acquire.X graphics server passes through system
Included X protocol is interacted with application program.
In the embodiment of the present application, the local current active window of terminal device real time monitoring, if current active window is
When remote application, the control information (such as keyboard, mouse message) for the remote application that user is sent is transmitted
To remote application server, remote application is controlled.If when the not remote application program of current active window, not to remote
Journey server sends the control information for the not remote application program that user sends.
A kind of graphical interfaces transmission method provided by the embodiments of the present application, remote server include remote application, X figure
Shape server module and application widget issuing service module are based on this, the control instruction that remote application is received in response
Afterwards, render instruction is sent to X graphics server module in server internal, X graphics server module is drawn based on render instruction
Update area data are then sent to terminal by the corresponding update area data of remote application, application widget issuing service module
Equipment.The above process reduces cpu resource, the memory of occupancy compared to the method that server screenshotss send entire desktop picture
Resource and Internet resources, meanwhile, server only transmits update area data to terminal device, also ensures terminal equipment side
User can only operate single application program, and menu and console without operating entire server improve server end
Safety.On the other hand, update area data can be depicted as video in window based on native window format by terminal device, this
Sample, remote application and locally applied windowed format are unified on the terminal device, and user is directed to locally applied on the terminal device
And the operation of remote application realizes the seamless local of user without switching between two entirely different operating systems back and forth
Change experience.
As shown in fig. 6, the application second embodiment additionally provides a kind of graphical interfaces transmission method, set applied to terminal
It is standby, the method includes：
S401, the corresponding update area data of remote application that remote server is sent are received；Wherein, the update
Area data includes window metadata and windows content data.
S402, according in the update area data the window metadata and the windows content data and local
Windowed format draws the corresponding video in window of the remote application.
JHApp Client Model in terminal device after receiving update area data, to the update area data into
Row decompression and decoding process extract the window metadata and windows content data in the update area data, and according to this
Ground windowed format, will be according to the window metadata and windows content data drafting in the update area data
The corresponding video in window of remote application.
S403, the display video in window.
Here, the corresponding video in window of remote application that the JHApp Client Model in terminal device obtains drafting
The display module being sent in terminal device, so that display module is shown, so that user checks.
Further, as shown in fig. 7, in graphical interfaces transmission method provided by the embodiments of the present application, described in the display
After video in window, further include：
It is if S501, detecting duplicate instructions for the remote application, the duplicate instructions are corresponding multiple
Content processed is stored on local pasting boards.
S502, when receiving the stickup instruction for the remote application, extracted from the local pasting boards
The reproducting content, and the reproducting content is sent on the long-range pasting boards in the remote server, so as to described remote
Remote application in journey server is pasted based on the reproducting content in the long-range pasting boards.
The application using delay pasting boards simultaneous techniques, i.e., only monitor duplicate instructions be directed to it is local long-range
At this moment the corresponding reproducting content of duplicate instructions is stored on local pasting boards, only exists by the corresponding active window of application program
After the stickup instruction for monitoring the corresponding active window of the remote application, it is synchronous with local pasting boards just to execute long-range stickup
(reproducting content of the storage in local pasting boards being sent to long-range pasting boards, stored by long-range pasting boards), so as to
Remote application carries out paste operation based on the reproducting content stored in long-range pasting boards, can reduce net to greatest extent in this way
Network expense.
A kind of graphical interfaces transmission method provided by the embodiments of the present application, remote server include remote application, X figure
Shape server module and application widget issuing service module are based on this, the control instruction that remote application is received in response
Afterwards, render instruction is sent to X graphics server module in server internal, X graphics server module is drawn based on render instruction
Update area data are then sent to terminal by the corresponding update area data of remote application, application widget issuing service module
Equipment.The above process reduces cpu resource, the memory of occupancy compared to the method that server screenshotss send entire desktop picture
Resource and Internet resources, meanwhile, server only transmits update area data to terminal device, also ensures terminal equipment side
User can only operate single application program, and menu and console without operating entire server improve server end
Safety.On the other hand, update area data can be depicted as video in window based on native window format by terminal device, this
Sample, remote application and locally applied windowed format are unified on the terminal device, and user is directed to locally applied on the terminal device
And the operation of remote application realizes the seamless local of user without switching between two entirely different operating systems back and forth
Change experience.
As shown in figure 8, a kind of graphical interfaces transmitting device that the application 3rd embodiment provides, for executing above-mentioned first
Graphic interface transmission method in embodiment, described device include：
Remote application 21, for sending and rendering to X graphics server module after the control instruction that response receives
Instruction；Wherein, the render instruction is for drawing the corresponding update area data of the remote application；The update area
Data include window metadata and windows content data；
X graphics server module 201, for drawing institute according to the render instruction after receiving the render instruction
State the corresponding update area data of remote application；
Application widget issuing service module 202, for obtaining the update area data, and by the update area data
It is sent to terminal device, the update area data are depicted as window figure so that the terminal device is based on native window format
Picture.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, X graphics server module 201 is also used
In the update area data are sent to the application widget frame buffer module；
Application widget frame buffer module 203, for receiving and storing the update area data；
It is described more to be specifically used for the crawl from the application widget frame buffer module for application widget issuing service module 202
New district numeric field data encodes the windows content data in the update area data, and to the windows content number after coding
Compression processing is carried out according to the window metadata in the update area data, and obtained compressed data packets are sent to described
Terminal device.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, application widget issuing service module
202, it is also used to：
When detecting that the time interval of feedback information of the terminal device return is more than preset threshold, according to preset
Step-length reduces the acquisition frequency of the update area data and sends frequency.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, application widget issuing service module
202, it is also used to：
The remote application of the terminal device transmission is being received not in the state notification message of front stage operation
When, stop obtaining and sending the update area data.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, further include：
Application widget issuing service module 202 is also used to after receiving the control instruction that the terminal device is sent, will
The control instruction is forwarded to the X graphics server module；Wherein, it is carried in the control instruction and the remote application
The corresponding application identification information of program；
X graphics server module 201, is also used to after receiving the control instruction, searches and believes with the application identities
Corresponding remote application is ceased, and the control instruction is forwarded to the remote application found, so as to described
Remote application responds the control instruction.
A kind of graphical interfaces transmitting device provided by the embodiments of the present application, the control that remote application is received in response
After instruction, render instruction is sent to X graphics server module in server internal, X graphics server module is based on render instruction
The corresponding update area data of remote application are drawn, update area data are then sent to by application widget issuing service module
Terminal device.The above process compared to the method that server screenshotss send entire desktop picture, reduce occupancy cpu resource,
Memory source and Internet resources, meanwhile, server only transmits update area data to terminal device, also ensures terminal device
The user of side can only operate single application program, and menu and console without operating entire server improve service
The safety at device end.On the other hand, update area data can be depicted as window figure based on native window format by terminal device
Picture, in this way, remote application and locally applied windowed format are unified on the terminal device, user is directed to local on the terminal device
Using and remote application operation without back and forth switching between two entirely different operating systems, it is seamless to realize user
Localized experience.
As shown in figure 9, a kind of graphical interfaces transmitting device that the application fourth embodiment provides, for executing above-mentioned second
Graphic interface transmission method in embodiment, described device include：
Receiving module 30, for receiving the corresponding update area data of remote application of remote server transmission；Its
In, the update area data include window metadata and windows content data；
Drafting module 40, for according in the update area data the window metadata and the windows content number
Accordingly and native window format, the corresponding video in window of the remote application is drawn；
Display module 50, for showing the video in window.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, further include：
Memory module, for when detecting the duplicate instructions for the remote application, by the duplicate instructions
Corresponding reproducting content is stored on local pasting boards；
Sending module, for locally being pasted from described when receiving the stickup instruction for the remote application
The reproducting content is extracted in plate, and the reproducting content is sent on the long-range pasting boards in the remote server, with
Toilet is stated the remote application in remote server and is pasted based on the reproducting content in the long-range pasting boards.
Further, in graphical interfaces transmitting device provided by the embodiments of the present application, further include：
Extraction module, for the corresponding update area data of the remote application unzip it and decoding at
Reason, extracts the window metadata and the windows content data in the update area data.
A kind of graphical interfaces transmitting device provided by the embodiments of the present application, the control that remote application is received in response
After instruction, render instruction is sent to X graphics server module in server internal, X graphics server module is based on render instruction
The corresponding update area data of remote application are drawn, update area data are then sent to by application widget issuing service module
Terminal device.The above process compared to the method that server screenshotss send entire desktop picture, reduce occupancy cpu resource,
Memory source and Internet resources, meanwhile, server only transmits update area data to terminal device, also ensures terminal device
The user of side can only operate single application program, and menu and console without operating entire server improve service
The safety at device end.On the other hand, update area data can be depicted as window figure based on native window format by terminal device
Picture, in this way, remote application and locally applied windowed format are unified on the terminal device, user is directed to local on the terminal device
Using and remote application operation without back and forth switching between two entirely different operating systems, it is seamless to realize user
Localized experience.
Figure 10 is the structural schematic diagram for the computer equipment 60 that one embodiment of the application provides, as shown in Figure 10, for holding
Graphical interfaces transmission method in row Fig. 3, the equipment include memory 601, processor 602 and are stored on the memory 601
And the computer program that can be run on the processor 602, wherein above-mentioned processor 602 executes real when above-mentioned computer program
The step of existing above-mentioned graphical interfaces transmission method.
Specifically, above-mentioned memory 601 and processor 602 can be general memory and processor, do not do have here
Body limits, and when the computer program of 602 run memory 601 of processor storage, is able to carry out above-mentioned graphical interfaces transmission side
Method.
Corresponding to the graphical interfaces transmission method in Fig. 3, the embodiment of the present application also provides a kind of computer-readable storages
Medium is stored with computer program on the computer readable storage medium, executes when which is run by processor
The step of stating graphical interfaces transmission method.
Specifically, which can be general storage medium, such as mobile disk, hard disk, on the storage medium
Computer program when being run, be able to carry out above-mentioned graphical interfaces transmission method.
Figure 11 is the structural schematic diagram for the computer equipment 70 that one embodiment of the application provides, as shown in figure 11, for holding
Graphical interfaces transmission method in row Fig. 6, the equipment include memory 701, processor 702 and are stored on the memory 701
And the computer program that can be run on the processor 702, wherein above-mentioned processor 702 executes real when above-mentioned computer program
The step of existing above-mentioned graphical interfaces transmission method.
Specifically, above-mentioned memory 701 and processor 702 can be general memory and processor, do not do have here
Body limits, and when the computer program of 702 run memory 701 of processor storage, is able to carry out above-mentioned graphical interfaces transmission side
Method.
Corresponding to the graphical interfaces transmission method in Fig. 6, the embodiment of the present application also provides a kind of computer-readable storages
Medium is stored with computer program on the computer readable storage medium, executes when which is run by processor
The step of stating graphical interfaces transmission method.
Specifically, which can be general storage medium, such as mobile disk, hard disk, on the storage medium
Computer program when being run, be able to carry out above-mentioned graphical interfaces transmission method.
Graphical interfaces transmitting device provided by the embodiment of the present application for the specific hardware in equipment or can be installed on
Software or firmware in equipment etc..The technical effect of device provided by the embodiment of the present application, realization principle and generation is with before
It is identical to state embodiment of the method, to briefly describe, Installation practice part does not refer to place, can refer to phase in preceding method embodiment
Answer content.It is apparent to those skilled in the art that for convenience and simplicity of description, the system of foregoing description,
The specific work process of device and unit, the corresponding process during reference can be made to the above method embodiment, details are not described herein.
In embodiment provided herein, it should be understood that disclosed device and method, it can be by others side
Formula is realized.The apparatus embodiments described above are merely exemplary, for example, the division of the unit, only one kind are patrolled
Function division is collected, there may be another division manner in actual implementation, in another example, multiple units or components can combine or can
To be integrated into another system, or some features can be ignored or not executed.Another point, shown or discussed is mutual
Coupling, direct-coupling or communication connection can be INDIRECT COUPLING or communication link by some communication interfaces, device or unit
It connects, can be electrical property, mechanical or other forms.
The unit as illustrated by the separation member may or may not be physically separated, aobvious as unit
The component shown may or may not be physical unit, it can and it is in one place, or may be distributed over multiple
In network unit.It can select some or all of unit therein according to the actual needs to realize the mesh of this embodiment scheme
's.
In addition, each functional unit in embodiment provided by the present application can integrate in one processing unit, it can also
To be that each unit physically exists alone, can also be integrated in one unit with two or more units.
It, can be with if the function is realized in the form of SFU software functional unit and when sold or used as an independent product
It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the application is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) execute each embodiment the method for the application all or part of the steps.
And storage medium above-mentioned includes：USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
It should be noted that：Similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing, in addition, term " the
One ", " second ", " third " etc. are only used for distinguishing description, are not understood to indicate or imply relative importance.
Finally it should be noted that：Embodiment described above, the only specific embodiment of the application, to illustrate the application
Technical solution, rather than its limitations, the protection scope of the application is not limited thereto, although with reference to the foregoing embodiments to this Shen
It please be described in detail, those skilled in the art should understand that：Anyone skilled in the art
Within the technical scope of the present application, it can still modify to technical solution documented by previous embodiment or can be light
It is readily conceivable that variation or equivalent replacement of some of the technical features；And these modifications, variation or replacement, do not make
The essence of corresponding technical solution is detached from the spirit and scope of the embodiment of the present application technical solution.The protection in the application should all be covered
Within the scope of.Therefore, the protection scope of the application should be based on the protection scope of the described claims.
Claims (10)
1. a kind of graphical interfaces transmission method, which is characterized in that be applied to remote server, the remote server includes long-range
Application program, X graphics server module and application widget issuing service module, the method includes：
The remote application sends render instruction after the control instruction that response receives, to X graphics server module；Its
In, the render instruction is for drawing the corresponding update area data of the remote application；The update area data packet
Include window metadata and windows content data；
The X graphics server module is remotely answered after receiving the render instruction according to render instruction drafting is described
With the corresponding update area data of program；
The application widget issuing service module obtains the update area data, and the update area data are sent to end
The update area data are depicted as video in window so that the terminal device is based on native window format by end equipment.
2. graphical interfaces transmission method according to claim 1, which is characterized in that the remote server includes application window
Mouth frame buffer module；The X graphics server module is corresponding more according to the render instruction drafting remote application
After the numeric field data of new district, further include：
The update area data are sent to the application widget frame buffer module by the X graphics server module；
The application widget frame buffer module receives and stores the update area data；
The application widget issuing service module obtains the update area data, and the update area data are sent to end
End equipment, including：
The application widget issuing service module grabs the update area data from the application widget frame buffer module, right
Windows content data in the update area data are encoded, and to after coding windows content data and the more new district
Window metadata in numeric field data carries out compression processing, and obtained compressed data packets are sent to the terminal device.
3. graphical interfaces transmission method according to claim 1 or 2, which is characterized in that the application widget issuing service
After the update area data are sent to terminal device by module, further include：
If the time interval for detecting the feedback information that the terminal device returns is more than preset threshold, according to preset step-length
Reduce the acquisition frequency of the update area data and sends frequency.
4. graphical interfaces transmission method according to claim 1, which is characterized in that the application widget issuing service module
After the update area data are sent to terminal device, further include：
If receiving the remote application of the terminal device transmission not in the state notification message of front stage operation, stop
Only obtain and send the update area data.
5. graphical interfaces transmission method according to claim 1, which is characterized in that the application widget issuing service module
After the update area data are sent to terminal device, further include：
If the application widget issuing service module receives the control instruction that the terminal device is sent, the control is referred to
Order is forwarded to the X graphics server module；Wherein, it is carried in the control instruction corresponding with the remote application
Application identification information；
The X graphics server module is searched corresponding with the application identification information remote after receiving the control instruction
Journey application program, and the control instruction is forwarded to the remote application found, so as to the remote application journey
Sequence responds the control instruction.
6. a kind of graphical interfaces transmission method, which is characterized in that it is applied to terminal device, the method includes：
Receive the corresponding update area data of remote application that remote server is sent；Wherein, the update area data
Including window metadata and windows content data；
According to the window metadata and the windows content data and native window format in the update area data,
Draw the corresponding video in window of the remote application；
Show the video in window.
7. graphical interfaces transmission method according to claim 6, which is characterized in that the display video in window it
Afterwards, further include：
If detecting the duplicate instructions for the remote application, the corresponding reproducting content of the duplicate instructions is stored
On local pasting boards；
When receiving the stickup instruction for the remote application, extracted in the duplication from the local pasting boards
Hold, and the reproducting content is sent on the long-range pasting boards in the remote server, so as in the remote server
Remote application pasted based on the reproducting content in the long-range pasting boards.
8. graphical interfaces transmission method according to claim 7, which is characterized in that described to receive what remote server was sent
After the corresponding update area data of remote application, further include：
The corresponding update area data of the remote application are unziped it and decoding process, extracts the update area
The window metadata and the windows content data in data.
9. a kind of graphical interfaces transmitting device, which is characterized in that including：
Remote application, for sending render instruction to X graphics server module after the control instruction that response receives；
Wherein, the render instruction is for drawing the corresponding update area data of the remote application；The update area data
Including window metadata and windows content data；
X graphics server module, for remotely being answered according to render instruction drafting is described after receiving the render instruction
With the corresponding update area data of program；
Application widget issuing service module is sent to for obtaining the update area data, and by the update area data
The update area data are depicted as video in window so that the terminal device is based on native window format by terminal device.
10. a kind of graphical interfaces transmitting device, which is characterized in that including：
Receiving module, for receiving the corresponding update area data of remote application of remote server transmission；Wherein, described
Update area data include window metadata and windows content data；
Drafting module, for according in the update area data the window metadata and the windows content data and
Native window format draws the corresponding video in window of the remote application；
Display module, for showing the video in window.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810729065.8A CN108897596A (en) | 2018-07-05 | 2018-07-05 | A kind of graphical interfaces transmission method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810729065.8A CN108897596A (en) | 2018-07-05 | 2018-07-05 | A kind of graphical interfaces transmission method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108897596A true CN108897596A (en) | 2018-11-27 |
Family
ID=64348833
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810729065.8A Pending CN108897596A (en) | 2018-07-05 | 2018-07-05 | A kind of graphical interfaces transmission method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108897596A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111263004A (en) * | 2020-01-21 | 2020-06-09 | 维达力实业(深圳)有限公司 | Terminal control method and device, computer equipment and storage medium |
| CN111818150A (en) * | 2020-07-03 | 2020-10-23 | 重庆中科云仿科技有限公司 | CAE simulation system based on cloud platform |
| CN111836092A (en) * | 2019-04-15 | 2020-10-27 | 深信服科技股份有限公司 | Data processing method and device of virtual desktop and related components |
| CN112181340A (en) * | 2020-09-29 | 2021-01-05 | 联想(北京)有限公司 | AR image sharing method and electronic device |
| CN114020396A (en) * | 2021-11-18 | 2022-02-08 | 湖南麒麟信安科技股份有限公司 | Display method of application program and data generation method of application program |
| CN114327712A (en) * | 2021-12-24 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method and device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102033726A (en) * | 2009-09-30 | 2011-04-27 | 联想(北京)有限公司 | Display method and server |
| CN102893658A (en) * | 2010-05-10 | 2013-01-23 | 日本电气株式会社 | Remote mobile communication system, server device and remote mobile communication system control method |
| CN103403695A (en) * | 2010-12-10 | 2013-11-20 | 韦斯技术有限公司 | Methods and systems for facilitating a remote desktop session utilizing a remote desktop client common interface |
| CN104765636A (en) * | 2014-01-02 | 2015-07-08 | 华为技术有限公司 | Remote desktop image synthesis method and device |
| CN107959687A (en) * | 2017-12-13 | 2018-04-24 | 上海曼恒数字技术股份有限公司 | A kind of data processing method, system, equipment and storage medium |
-
2018
- 2018-07-05 CN CN201810729065.8A patent/CN108897596A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102033726A (en) * | 2009-09-30 | 2011-04-27 | 联想(北京)有限公司 | Display method and server |
| CN102893658A (en) * | 2010-05-10 | 2013-01-23 | 日本电气株式会社 | Remote mobile communication system, server device and remote mobile communication system control method |
| CN103403695A (en) * | 2010-12-10 | 2013-11-20 | 韦斯技术有限公司 | Methods and systems for facilitating a remote desktop session utilizing a remote desktop client common interface |
| CN104765636A (en) * | 2014-01-02 | 2015-07-08 | 华为技术有限公司 | Remote desktop image synthesis method and device |
| CN107959687A (en) * | 2017-12-13 | 2018-04-24 | 上海曼恒数字技术股份有限公司 | A kind of data processing method, system, equipment and storage medium |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111836092A (en) * | 2019-04-15 | 2020-10-27 | 深信服科技股份有限公司 | Data processing method and device of virtual desktop and related components |
| CN111263004A (en) * | 2020-01-21 | 2020-06-09 | 维达力实业(深圳)有限公司 | Terminal control method and device, computer equipment and storage medium |
| CN111263004B (en) * | 2020-01-21 | 2021-06-18 | 维达力实业(深圳)有限公司 | Terminal control method and device, computer equipment and storage medium |
| CN111818150A (en) * | 2020-07-03 | 2020-10-23 | 重庆中科云仿科技有限公司 | CAE simulation system based on cloud platform |
| CN111818150B (en) * | 2020-07-03 | 2021-03-05 | 重庆中科云仿科技有限公司 | CAE simulation system based on cloud platform |
| CN112181340A (en) * | 2020-09-29 | 2021-01-05 | 联想(北京)有限公司 | AR image sharing method and electronic device |
| CN114020396A (en) * | 2021-11-18 | 2022-02-08 | 湖南麒麟信安科技股份有限公司 | Display method of application program and data generation method of application program |
| CN114327712A (en) * | 2021-12-24 | 2022-04-12 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method and device |
| WO2023116184A1 (en) * | 2021-12-24 | 2023-06-29 | 阿里巴巴（中国）有限公司 | Cloud application system, window management method, and device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108897596A (en) | A kind of graphical interfaces transmission method and device | |
| CN109582425B (en) | GPU service redirection system and method based on cloud and terminal GPU fusion | |
| CN111433743B (en) | APP remote control method and related equipment | |
| JP5455930B2 (en) | Graphic remote architecture | |
| CN106330986B (en) | Message transmission method and device | |
| CN105228080B (en) | Multi-screen interaction method, system and browser | |
| CN103369288A (en) | Instant communication method based on network video and system thereof | |
| WO2017092547A1 (en) | Data transmission method and device | |
| WO2013024342A1 (en) | Method for flow control and for reliable communication in a collaborative environment | |
| CN112114928B (en) | Processing method and device for display page | |
| CN105335176B (en) | A kind of method and device for calling the application being installed on electronic equipment | |
| CN102646031A (en) | Method for controlling large screen to display contents by mobile terminal | |
| CN102523109A (en) | Resource state updating method, management client ends, and server | |
| EP2849080A1 (en) | Image display method and device | |
| CN113157362A (en) | Android multi-window display method applied to Linux | |
| CN111208960A (en) | Remote display delay reducing method based on frame extraction control and time synchronization algorithm | |
| CN103501295B (en) | A kind of remote access method based on virtual machine (vm) migration and equipment | |
| CN113778583A (en) | Method, device, equipment and medium for publishing local application of cloud desktop | |
| WO2024061308A1 (en) | Notification processing method, and terminal device, server side and computer storage medium | |
| CN110661884A (en) | Switching system and method of cloud desktop and local desktop | |
| CN103986759A (en) | Multi-person real-time synchronous graphic edition and display method based on network transmission | |
| CN113778593A (en) | Cloud desktop control method and device, electronic equipment, storage medium and program product | |
| CN103049262A (en) | Window restoring method based on remote frame buffering | |
| CN111007982A (en) | Electronic whiteboard implementation method, device and related system | |
| CN114443192A (en) | Multi-window virtual application method and device based on cloud desktop |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |