CN102855133A - Interactive system of computer processing unit - Google Patents

Abstract

The invention relates to an interactive system of a computer processing unit. The processing unit is between an application program and a system API (application program interface) or a network and comprises a sound exporting module, an interface exporting module, another information exporting module and an input importing module. The interface exporting module is used for converting a game picture into video stream to output, and the sound exporting module converts background music and effect sound into video stream to output. The other information exporting module exports position and style of cursor in the application program and outputs the position and style from an independent information stream channel. Cost of hardware resources can be reduced, and multiparty interaction can be achieved.

Description

A kind of computer processing unit interactive system

Technical field

The present invention relates to a kind of computer processing unit interactive system, relate to a plurality of technical fields such as computing machine, network, Streaming Media.

Background technology

Cloud computing refers to payment and the use pattern of IT infrastructure, refer to by network with as required, the mode of easily expansion obtains resource requirement; The broad sense cloud computing refers to payment and the use pattern of serving, refer to by network with as required, the mode of easily expansion obtains required service.It is relevant with software, internet that this service can be IT, also other services.Cloud computing (Cloud Computing) is that the product that the traditional computer such as (Utility Computing), the network storage (Network Storage Technologies), virtual (Virtualization), load balancing (Load Balance) and network technical development merge is calculated in grid computing (Grid Computing), Distributed Calculation (DistributedComputing), parallel computation (Parallel Computing), effectiveness.

A problem carrying out interactive program in cloud computing system is that interactive program may be designed to only be undertaken alternately by local interaction devices on the computing machine that it moved by the user.That is user's mutual action depends on the interactive device (such as keyboard, mouse, display etc.) on the local computer.This means that in the specific moment computing node can only provide service for unique user at most in the cloud system.

For large-scale cloud computing system, the cost of its total construction cost and single computing node is closely related.Normally, as shown in Figure 1, the P/C of hardware device increases along with the raising of performance than within the specific limits, and simultaneously, higher performance also helps computing node can bear the stricter performance requirement of program.Yet high-performance simultaneously also means the raising of cost, and has two problems: 1. for the computing node of the application program of interactive behavior, program is uneven to the use of the computing power of node in time usually.For example in the application of a three-dimensional modeling, the process of user input data (low computational resource takies) may account for the major part of service time and render process (high computational resource takies) only accounts for the short time, so computational resource is left unused in the most time; 2. the computational resource of interactive application uses the actual upper limit that does not reach hardware performance, and this is especially obvious in interactive PC game.Because PC game often is designed to only need consume the computational resource of certain limit when reaching certain user's experience requirements, and often limits the consumption of resource in order to adapt to more different model hardware device.

Therefore in existing cloud computing service system, the user of interactive PC game experiences and is limited on certain level, the therefore also just consumption of the hardware resource of restriction, and final result is, purchases the hardware device cost that more computational resources pay and is wasted.

In addition, single computing node also has extra fixed cost, bus apparatus (mainboard) for example, and memory device, power supply and shell, and for space cost of placing computing node etc.Can see that when the cost of each user's of cloud computing system cost of serving and computing node was relevant, user's cost of serving that the method for the hardware cost by reducing single computing node can reduce was very limited.

Apparently, effectively the descend scheme of alone family cost is to make single computing node can be simultaneously a plurality of users' services.But as previously mentioned, the interactive device that need to monopolize on the computing node of interactive application moves.Virtual (Virtualization) technology provides a possible implementation.Yet virtualized problem is to use this technology will bring the performance of the expense of extra hardware resource, particularly graphic process unit (GPU) to tend to suffer significant loss, and have the defective on the hardware supported.For to provide the windows application program for the main cloud computing service system, Intel Virtualization Technology will increase windows operating system quantity and then the raising mandate expense of moving on the computing node; The combination of Intel Virtualization Technology itself and windows operating system is very unripe yet.

Summary of the invention

In order to solve the problems of the technologies described above, the invention provides a kind of computer processing unit, between application program and the API of system or network, it is characterized in that: it comprises that sound is derived module, the interface is derived module, other information derivation modules and input and imported module; It is video flowing output for the picture conversion that will play that module is derived at the interface, and described sound is derived module background music and effect sound are converted to audio stream output; Other information derive modules, with the position of cursor in the application program with pattern is derived and from an independently channel of information flow output.

Other information derive modules and directly the information of cursor are not merged in the program interface and transmit as video stream; But directly draw out by the information of supplemental stream transmission mouse pointer and in client.

When user's rolling mouse pointer, client terminal local will at first respond the event that mouse moves and repaint mouse pointer, and other information derive modules are sent to this event long-range execution simultaneously by control information application program.

Further comprise storage and an application context interface, described interface comprises file access API, registry access API, environmental variance and kernel objects etc.; Described processing unit is redirected the collision problem that can avoid resource to use when interface is carried out a plurality of identical application programs under same operating system; Described processing unit further comprises a record cell, is used for record and keeps these contextual informations, thereby prepare from the current context information and executing for application program next time.

Described processing unit is connected with mixed-media network modules mixed-media with the video/audio compression module, with the interface of application program with play up data-switching and become video/audio stream, be sent to other terminal on the internet through mixed-media network modules mixed-media.

It further comprises a redirection module, described redirection module is by revising the parameter of API Calls, and then application program is handed to a specific hardware of deriving the module connection to the access of certain hardware device carry out, thereby the information that the program that is applied is exported from hardware device.

Described processing unit further comprises a plurality of processes, and described process is connected with client, carries out specific request by client-requested, and described request is the data of application program or transmission or reception; Described processing unit is written into internal memory with above-mentioned request, before the actual execution of above-mentioned request, redirection module is loaded the process space into application program, and revise the loaded run time version of application program in the memory headroom of program process.

Described function address redirection module is a hardware, and it changes the funcall entry position of the operating system application programming interfaces API of function; After application program was carried out, during routine call operating system API, redirection module jumped to the function address that redirection module is stored with the function address that is modified

It comprises terminal device based on the long-distance interactive system of cloud computing to the present invention further provides one, and web front-end, security gateway is characterized in that:

Described security gateway comprises one or more service nodes;

Described service node further comprises a kind of computer processing unit, and between application program and the API of system or network, described processing unit comprises that sound is derived module, the interface is derived module, other information derivation modules and input and imported module; It is video flowing output for the picture conversion that will play that module is derived at the interface, and described sound is derived module background music and effect sound are converted to audio stream output; Other information derive modules, with the position of cursor in the application program with pattern is derived and from an independently channel of information flow output;

Described terminal device is at first accessed described front end, by obtaining the authority that connects a certain service node in the security gateway after the checking; Terminal device uses this authority to be connected specifically with cloud computing system subsequently, and service node connects; In described security gateway, the interface of all application programs and audio frequency output are converted into Audio and Video stream, and a relative additional information stream, and pass to described terminal device by the internet.

Description of drawings

Fig. 1 is the P/C ratio synoptic diagram of the hardware device of extensive cloud computing system;

Fig. 2 is based on the framework of the long-distance interactive system of cloud computing node;

Fig. 3 is based on the module map of the long-distance interactive system of cloud computing node;

Fig. 4 is based on the hierarchy chart of the long-distance interactive system of cloud computing node;

Fig. 5 is the synoptic diagram of a kind of embodiment of microenvironment;

Fig. 6 is redirected synoptic diagram under the microenvironment;

Fig. 7 is the mutual synoptic diagram of mouse message among the present invention;

Fig. 8 is the synoptic diagram that among the present invention picture is carried out pipeline processes;

Fig. 9 is the synoptic diagram of the long-distance interactive system access network file of cloud computing node;

Figure 10 is the synoptic diagram that interactive information is distributed between different user;

Figure 11 is that the present invention is about the synoptic diagram of coding and decoding video;

Figure 12 is the synoptic diagram of the present invention's " displaying wall ";

Figure 13 is the present invention to the synoptic diagram of the processing mode of the mouse action that receives in the limited rectangular extent;

Figure 14 is the performed workflow diagram of user that uses cloud computing remote interaction procedure service;

Figure 15 is the deployment diagram of distributed calculation services node array.

Embodiment

Fig. 2 and Fig. 3 have listed a framework based on the long-distance interactive system of cloud computing node.It comprises terminal device, web front-end, security gateway.Described terminal device can be PC, and any computing powers such as portable terminal or TV set-top box enough are competent at the H.264 equipment of video flowing of decoding SD or high definition.Described security gateway comprises one or more service nodes, and terminal device is at first accessed the front end of the system of cloud computing, by obtaining the authority that connects a certain service node in the security gateway after the checking; Equipment uses this authority to be connected specifically with cloud computing system subsequently, and service node connects.

By one specifically by hardware, software, or it realizes above-mentioned functions in conjunction with platform or the environment (following abbreviation " microenvironment ") realized, take windows operating system as example, can be virtual its use to user interactive system of other windows application program, thereby make the computing node of single operation windows operating system can serve simultaneously a plurality of users.With respect to other schemes, microenvironment has following advantage:

Microenvironment is so that a plurality of user utilizes the resource (software resource that comprises single operating itself) of computing node to become possibility simultaneously.

The application program that is performed is with almost primary mode access hardware resource (particularly graphic process unit resource).

Microenvironment does not change the binary code of performed application program.

Among the present invention, in service node, a microenvironment starts the application service of appointment for the user, described service can comprise all services of application program, the interface of application program and audio frequency output are converted to Audio and Video stream by microenvironment, and the very little additional information stream of (optionally) data volume passes to terminal by the internet.The user is after using terminal device to receive above-mentioned Audio and Video stream and (optionally) additional information stream, interface hardware (keyboard by equipment, mouse, telepilot, game console etc.) send the control data to the microenvironment of calculation services node, microenvironment receives these control data as the application program of moving user interactive data.Thereby finished a mutual process.

Microenvironment be implemented in the operating system level, between application program and the API of system or network, those skilled in the art is to be understood that, although used " microenvironment " this term here, but should be with the functional structure body of its extend geographic solution for being realized by hardware, software, perhaps a processing unit.Consider an arbitrarily application program, for computing machine on the hardware interactive device carry out alternately the interface accessing hardware driving that application program must provide by operating system, and then control hardware.(such as Fig. 4 left) microenvironment comprises a function address monitoring modular, function address variable for detection of application program in the internal memory, and function address redirection module, be used for to change the function address variable that the internal memory of application program is stored, and an address preservation module, be used for preserving the function address that again is directed; And comprise a plurality of interfaces, be used for realization and computer hardware and windows application programming interfaces between communicate by letter, program interface for example, picture is played up, audio frequency is exported and the input of control information (keyboard, the information such as mouse).

By the function address variable of storing in the internal memory that changes application program, make application program be redirected to interface that microenvironment provides to the access of operating system interface and no longer contact (right-hand such as Fig. 4) with hardware layer or the hardware driving of the PC system of reality.For application program, microenvironment provides interface and the respondent behavior identical with operating system.Therefore in microenvironment, carry out and to affect its normal workflow.Microenvironment has then obtained application program and user's interaction data, comprises program interface, and picture is played up, the input of audio frequency output and control information (keyboard, the information such as mouse).Microenvironment can realize in cloud computing system, and be connected with mixed-media network modules mixed-media with the video/audio compression module, thereby with the interface of application program with play up data-switching and become video/audio stream, be sent to other terminal (user of cloud computing service) on the internet through mixed-media network modules mixed-media.Equally, after the customer interaction information of terminal arrives microenvironment by the internet, the customer interaction information that has replaced the interactive device on the former cause computing node to send conveys to application program, the final purpose that realizes carrying out at remote computer (cloud computing service node) by the internet interactive application.

Fig. 5 has shown the another kind of general embodiment of microenvironment.Take windows operating system as example, the realization mechanism of microenvironment is in conjunction with windows system distinctive dynamic link library (DLL) mode of operation and independently the process space is machine-processed.In operating system, microenvironment can further comprise a plurality of processes, and described process is connected with client, carries out specific request by client-requested, and described request can be the data of application program or transmission or reception.Microenvironment is written into internal memory with above-mentioned request subsequently, before the actual execution of above-mentioned request, microenvironment utilizes the working method of dynamic link library that redirection module (with the form of DLL) is loaded the process space of application program, and revises the loaded run time version of application program in the memory headroom of program process.Perhaps in order to save the resource of PC hardware, microenvironment directly utilizes the redirected hardware module of self to revise the function address variable that comprises in the above-mentioned request, with the funcall entry position of the operating system application programming interfaces API that changes function.After application program was carried out, during routine call operating system API, redirection module jumped to the function address that redirection module is stored with the function address that is modified.The microenvironment main body of moving in redirection module and other process that is connected connects by specific Communication tunnel, and the interactive content when program is carried out outputs in the microenvironment process, finally sends to client by network after treatment.

Fig. 6 has shown the typical workflow when being called after revising of the API of system in the memory headroom of program process: by revising internal storage data, the machine instruction that the API entrance function starts most is modified to a jump instruction, jumps to the interface function entrance that redirection module that microenvironment loads provides.Thereby application program at first is redirected in the microenvironment redirection module to the use of the API of system.All parameters when the interface function of microenvironment has received the API of application call system, after this can use one of following manner to obtain the interactive information of application program:

The parameter of importing into during the API of 1) application call system has just comprised required interactive information, for example to the location of mouse pointer, and the path parameter of file access etc.

2) microenvironment is used the parameter that obtains, and replaces the required system API that calls of its script of application call.As shown in Figure 5, can at first carry out the instruction of the internal memory that is modified part among the former API in the microenvironment, internal memory begins to carry out until api function finishes after being modified part from former API again, namely finish the invoked procedure of an API, and obtain the interactive information of application program the result data that after API Calls, returns, such as coloured program interface picture etc.These information microenvironment interface functions finish, give back application program before, can at first be output to the derivation module of microenvironment.

3) application program is often relevant with hardware device to the use of the API of system, and specific hardware (or hardware driving) can be realized and directly and be connected the module connection by microenvironment.By revising the parameter of API Calls, the specific hardware (or driving) that microenvironment can be handed to this to the access of certain hardware device with application program and be connected the module connection carries out, thus the information that the program that is applied is exported from hardware device.The for example derivation of audio-frequency information.Microenvironment also has more effect with application program to the function that hardware access is redirected, and will set forth in the back.

The importing module workflow of microenvironment and derive module class seemingly: the microenvironment interface function obtains the information that needs from import module, the parameter of importing into during by application call API or rreturn value etc. pass to application program.

Further referring to accompanying drawing 3, in microenvironment, mutual all information that comprise are processed separately, and finally by different information flows, be sent to client-side program, in this embodiment, microenvironment includes but not limited to lower module: sound is derived module, the interface is derived module, other information derivation modules and input and imported module.Take the PC game as example, it is video flowing output that the picture of game is derived module converts by the interface, and it is audio stream output that background music and effect sound sound are derived module converts.In order to realize the function of user interactions, the position of some extra information such as cursor in the application program and pattern etc. are also derived module from an independently channel of information flow output by information in addition.The interactive information that receives at subscription client then is sent to input and imports module, and is converted to the contents such as the mouse of the actual acquisition of application program and Keyboard Message.

Further, the microenvironment redirection module comprises the interface of sequence of operations system call, includes but not limited to Types Below:

Graphical display interface comprises Windows GDI and DirectX.In recent years, the PC of overwhelming majority game utilizes the DirectX interface to play up pictures whole in its implementation.

Audio frequency play interface, for example DirectSound or be windows media device interface (Multimedia Device API).

Input/output interface comprises Direct Input and keyboard/mouse message interface.

Storage and application context interface comprise file access API, registry access API, environmental variance and kernel objects etc.The collision problem that can avoid resource to use when these interfaces that microenvironment is redirected are carried out a plurality of identical application programs under same operating system.The all right record cell of microenvironment is used for record and keeps these contextual informations, thereby prepares from the current context information and executing for application program next time.

Because microenvironment can replace application program to carry out calling of the API of system.Therefore when application program need to be used hardware resource, microenvironment did not need to simulate all Hardware Response Delay.In the system of reality, respectively derive the request of module application programs in the microenvironment after carrying out necessary processing, usually all application program can be come the directed time original position of the access of hardware resource (by the system function interface of reality).In fact application program is compared with original working method in microenvironment, only has more required extra operation in two step jump instructions and the microenvironment, and this compares the expense with much less with Intel Virtualization Technology, can think to be in close proximity to primary working method.

Although Windows operating system can support many group hardware resources to work simultaneously, for example the CPU of multi-core and many GPU.But operating system still is confined on the CPU the Automatic dispatching of resource, and can't Automatic dispatching and the computing power of distributing other hardware (particularly GPU).Even a plurality of GPU of computing node deploy for example, general application program often only uses in the operating system equipment that is designated as " main GPU " to carry out playing up of 3D picture.Even application program can have access to the information of whole GPU equipment and can pass through certain GPU equipment of Instruction Selection of user, when moving a plurality of application program on the computing node, for each user, it also is impossible selecting suitable GPU equipment.

Utilize the microenvironment interface to derive module to being redirected of DirectX interface, microenvironment can be the suitable GPU resource of certain specific application assigned at a plurality of GPU equipment rooms of a plurality of computing nodes (or service node).Namely select current GPU equipment with enough idle computing powers, and give different resource occupation to these GPU equipment and finish a specific calculation.For this reason, further comprise a logging modle in the microenvironment, be used for recording current all application programs of carrying out therein to the operating position of GPU resource, further, it also comprises one and monitors module, be used for monitoring the duty of GPU, in order to be the GPU equipment that the next application assigned of carrying out is fit to, that is to say, the GPU of each computing node (or service node) may be assigned with and participate in the calculating irrelevant with this computing node, and an embodiment of described supervision module is based on the system interface of the standard of reading and the real-time parameter of driving interface monitors its duty.

The redirected output that sound is derived module application programs audio frequency is implemented in the hardware driving aspect.Microenvironment realizes that with the form of hardware driving a plurality of (virtual) audio frequency plays up and playback equipment before intervention.Application program realizes that to calling by virtual unit of audio frequency play interface rear boil down to audio stream outputs to long-distance user's computing machine by the internet.Under normal circumstances, when a plurality of application programs were moved simultaneously, the audio frequency of each program output was with mixed and be output in simultaneously the audio frequency apparatus of system.On the other hand, microenvironment can be one of them of these virtual audio device of application assigned of wherein each execution, makes the audio frequency output of different application be become audio stream independently and the voice-grade channel by different microenvironments transmits.

Outside picture and audio content, application program for the user correctly mutual required out of Memory also to derive module by out of Memory collected and be sent to client by accessory channel.For example, the physical location of mouse pointer need to be sent to client in the current application program, in order to show exactly its position and out of Memory on client screen.

Mouse pointer is the most common element in the interactively windows program.As seen from Figure 7, when mutual with long-range application program, any interactive operation to mouse pointer all needs to obtain by Internet Transmission, could show reuse family client after the response.This means the user when client rolling mouse pointer, will observe one and the suitable delay of network delay (TTL).We observe, and the user is highstrung to the delay of rolling mouse pointer; During element-specific, user's the tolerance to postponing is then much bigger on clicking screen but relatively.Therefore, directly the information of cursor is not merged in the program interface in the microenvironment and transmit as video stream.But directly draw out by the information of supplemental stream transmission mouse pointer and in client.When user's rolling mouse pointer, client terminal local will at first respond the event that mouse moves and repaint mouse pointer, and simultaneously this event will be sent to the application program of long-range execution by control information.Noticeable delay when this time strategy has been eliminated the rolling mouse pointer.The movement that the delay that network delay causes occurs over just mouse pointer causes program interface to occur in the process of specific response.And as aforementioned, this delay can be tolerated by most users.

Specify the screen resolution that specific remote application may have relatively high expectations and client screen possibly can't satisfy this requirement, this is particularly common on the equipment such as portable terminal.In the case, and the interface Zoom module of deriving the module coupling output interface of application program can be narrowed down to the acceptable scope of client screen and be output as video flowing.After the interface was reduced, original Partial Elements, particularly word content etc. may lose its readability owing to dwindling on the interface.At this moment, an extra application program will start in microenvironment, the interface of this routine access interface exporter, the picture (keeping its original size) of ad-hoc location on the program interface (usually to be a little square region centered by the mouse pointer position pointed) is marked separately, utilize video encoder to be converted into a less video flowing, be transferred to client (such as Fig. 7) by supplemental stream, namely partly do not encode at normal video flowing; Normal video flowing can be used for encoding whole image.This video flowing after client decoding, reappear module with its with remote application on identical resolution show, and be superimposed on shown (reduced) the program interface picture of client screen.The user can wait operation by moving of mouse pointer, from then on reads the picture element that dwindles rear None-identified on the amplification picture of stack.Client is reappeared the part that module will mark separately and is shown with former resolution, and the described picture that presents is zoomed to the resolution that adapts with the employed display device of this client and shows that the described part that marks separately shows as for the superiors.

The Audio and Video of microenvironment derives module and the low coding module that postpones is coupled.The required computing of coding module can utilize CPU or GPU on the calculation services node to finish, or uses special coded hardware to process.The graphical information of Application Program Interface consists of each frame of outputting video streams by certain time interval.At a time, the Application Program Interface figure is converted to that the scramble time of corresponding frame is not more than 10 milliseconds in the video flowing; On user's terminal device, this frame of video also is not more than 10 milliseconds by video decoder decodes and time of being presented on terminal screen.Under normal circumstances, the Internet Transmission of our expectations from the service system to the user terminal delayed time (TTL) about the 40-50 millisecond.On the other hand, the per second frame per second of video flowing is limited between the 25-30, thus the user with the viewed delay of the interbehavior of remote application at most about 100 milliseconds.And even mutual PC game all is an acceptable scope for general application program for this.

Fig. 8 has shown video encoding module, the streamline strategy (nethermost strategy among the figure) that decoder module and transport module are taked in order to reduce coding-decoding delay.The picture of program is actual to be split into a plurality of parts, each part is encoded into respectively an independent video streaming to client, by after the decoder module decoding, by reappearing the picture amalgamation that module will cut apart and showing, the picture of wherein said independent division is with higher bandwidth for transmission again.GPU carries out compressed encoding with changing program interface and is converted to video flowing, and in the process of Terminal Server Client decoding.Coding-decode procedure is quasi real time, its postpone and data to transmit in network that the delay that causes compares be a small amount of.In order to reduce the delay of Code And Decode, Application Program Interface is actual in cataloged procedure to be divided into a plurality of parts, and every part encodes independently-transmits-decodes, the treatment step of each several part is with the concurrent execution of pipeline system.

Be that single video flowing is compared (Fig. 8 top strategy) with whole picture coding, although lose in compressibility, can effectively utilize the parallel ability of computing node, reduce coding delay (strategy in the middle of Fig. 8).Yet therefore the decoder module of client can not significantly reduce decoding delay because the restriction of client computing power does not often have the parallel decoding ability.Simultaneously can see that also in these two kinds of strategies, the computing power of equipment is left unused in front in transmission course.And in the streamline strategy, the process of Code And Decode and the process of transmission are carried out between the different pictures piece alternately, during a therein picture block transmission, be encoded module and decoder module of the computing power of equipment is used for other picture block is processed, in fact three steps of will encoding-transmit-decode partly walk abreast, thereby have reduced total mutual delay.

H.264 video decode module on the client device of the video encoding module of microenvironment and correspondence is used or similar standard.In Video coding, must comprise two types image frame.One of them (I frame) do not rely on any other image frame when decoding, the information of some frame that is received before the decoding of another kind of (P frame) then depends on.Single I frame need to be larger than P frame byte number, the distribution of I frame in video flowing is uncertain, the time interval between the appearance of per two I frames is at 1-10 between second.Video flowing between two I frames is comprised of the less P frame of byte number.

What provide the miscellaneous application program of execution for the node of cloud computing system in constantly may be difficult specific.One of distinct issues are that the scale-of-two run time version of preserving each application program may need a large amount of stored bits units.Particularly concerning the PC game, the scale-of-two run time version of the PC game in the present age often surpasses 4G byte (1 byte number that standard DVD can carry).If the binary code of program is stored in respectively on the local storage (hard disk) of each computing node, then need to provide for each computing node the storage capacity of several TB bytes.The cost of the storage system that this gives and maintenance all bring considerable difficulty.Scheme of saving the storage space of computing node is that the binary code with program is kept on the single memory device, i.e. as shown in Figure 3 procedure stores array, and each node is accessed code data on this equipment by network connection at a high speed when executive routine.

Can this Concurrency Access can only be on file system Network Based (for example SAMBA) aspect and can not realize at network disk system (for example iSCSI).Yet to a lot of only application programs (for example nearly all PC game) for design at a high speed local storage operation, program execution to user interactions before need from storage system, to read in advance a large amount of data in internal memory.In theory, in the time transmitting data with the ideal velocity of gigabit (Gigabit) network between computing node and the memory node, the reading rate of data (~ 100MB/s) can be complementary with the local storage (hard disk, about 70MB/s) of routine.If stand-by period that the reading of data causes and on local storage quite when (even slightly long), this wait is familiar with by the user and is accepted, and will can not reduce user's experience.Yet, these data are usually deposited with the form of small documents, and under the windows commonly used based on the SAMBA agreement network file system(NFS) and be not suitable for the concurrent task that reads of large amount of small documents, the result causes actual valid data reading speed often less than 10MB/s, before program is carried out and executory data read waiting time be several times and even ten several times on local storage, seriously reduced user's experience.

By microenvironment, we have optimized the data reading performance using redundancy of network file system(NFS) when application access under the SAMBA agreement, and do not need to develop in addition more difficult and stable other network file system(NFS) that leaves a question open.In the system of Fig. 3, all service nodes connect with the storage array that is connected in the cloud computing system.As shown in Figure 9, under normal circumstances, application program is directly by the network file position (dotted arrow) on the SAMBA protocol access storage array.And in microenvironment, the actual buffer memory that is arranged in local memory device that is redirected to of the access of application program.Cache contents is managed by microenvironment.When computing node ran application first, microenvironment read the network file on the storage array by another agreement based on the rSync scheme (we are called rSync_i).When reading, a plurality of concurrent file access requests are merged into single TCP request and are conducted interviews according to directory location, the extreme value that the reading speed of file reality is allowed near network environment.

A plurality of computing nodes will be accessed the application data on the same memory node jointly, and the excessive data that each user is produced when the executive utility stores on the single memory node.Importing and exporting and transmitting by microenvironment of data realizes.When microenvironment has used specific strategy to come accelerating application to carry out from the memory node the speed of reading out data.System consists of a node array with a plurality of computing nodes, and a plurality of node arrays are shared the data that a memory node (array) reads performed application data and recording user generation.The user can be by in a plurality of computing node arrays of the data selections such as network delay and bandwidth.A plurality of memory nodes consisted of can the phase mutually synchronization distributed memory system.

Subscription client is accepted discernible any user interactive and is sent in the microenvironment by control information.Yet the input filtering module of microenvironment will be differentiated all user's operations and only allow specific operation can send in the middle of the application program that is performed.This judgement is based on the situation at current application program interface and dynamically changes.When the window of the file of the local store path of computing node for example can be accessed and carry out to current cursor () at the interface that is judged as " danger ", the input filtering module can stop the long-distance user to carry out may be to computing node and the adventurous operation of whole service system.

The data that derive the video/audio stream of module output and supplemental stream from interface and sound are stored in as shown in Figure 4 the microenvironment output buffer module.Therefore, the user's that the information such as interface of performed application program not only can requested execution terminal sees, also can output on one or more users' that participate in onlooker's identity the terminal by output buffer module and transport module.Connect the microenvironment of same computing node as onlooker's user, flow in the corresponding microenvironment but can't send control information.Onlooker user can also send specific interactive information to cloud computing system by other passage except watching the action with the user of program interaction.As shown in figure 10, a specific program can start in this microenvironment, receives and gathers the exchange of information that other watches the user.The interface is derived module a video flowing is merged at the interface of these exchange of information and application program, thereby with these interactive information again be distributed to current all connect user's (comprising interactive user and onlooker user) of this microenvironment, and may be displayed on all users' the terminal.For example, on the microenvironment at a user place of carrying out PC game, the onlooker can input comment in real time with written form, and these comments can send to all other onlooker and game user.

The video flowing that beholder's user and the mutual user of execution access is synchronous (except the impact that causes owing to different network delays between the user).Yet the video decode module of user terminal generally can not be since frame (B frame) decoding and a correct display of video streams that reference is arranged.In output buffer module, can preserve all videos frame data within a period of time of interactive user output, wherein must comprise at least one without the I frame of reference.As shown in figure 11, when a beholder user initial when connecting microenvironment, output buffer module will preserve from sending to beholder user when the nearest I frame of output frame of forward direction interactive user, until when having next I frame to occur, this frame will see that beholder user and interactive user export simultaneously.Because the time interval that two I frames occur may reach 10 seconds, the terminal that this strategy has significantly reduced beholder user is from the delay of successful connection when video occurring.

Particular application that is called as " displaying wall " may operate in one independently in the microenvironment, as shown in figure 12.The intrasystem service managerZ-HU of displaying wall access computation is obtained the application program of moving in other microenvironment, and is connected in these microenvironments as beholder user.Show all video flowings that the decoding of wall program gets access to, and the size of these video flowings with preview graph reorganized in the single interface.This interface is encoded as new video flowing and sends to user terminal screen.The user can select to show certain preview video stream within the walls, and exchanges with other users in preview video flows corresponding microenvironment as a beholder user.

The Audio and Video of microenvironment stream can be merged into both the multimedia content flows (as shown in Figure 4) of specific format by the agreement module of reforming before sending to network.Such multimedia content flows can further be arrived at the client terminal program with special realization by the internet.For example, the multimedia content flows that generates from the reformation module can be by with RTMP(Real Time Messaging Protocol) mode of the agreement Adobe Shockwave Flash platform that carries and be widely used presents.Thereby the user allows to use cloud service system on the basis of not moving special client-side program.For example, this namely can be used as beholder user's access so that any user only need to have the terminal (all PC, net book, WebOS system and most portable terminal etc.) that can move modern Web browser.

Use Adobe Shockwave Flash platform can receive the supplemental stream that microenvironment is sent, and the operation of user on the flash platform sent to input as control information stream import module, thereby realize a complete user terminal and be not only the beholder.Yet based on the safe limit of Flash platform, some required interbehavior can not directly be realized when carrying out remote application.Portion keyboard function for example.A typical situation is, for example in the PC game of first person, allows mouse to do unlimited movement (corresponding to the continuous rotation at visual angle) to a specific direction in some application program.But the mouse action that can only be received in a limited rectangular extent by the client terminal that Flash realizes, when mouse moves when causing pointer movement to the edge of rectangle to a direction, follow-up movement can't be detected again.

Figure 13 has shown a solution to this problem: the rectangular area that receives the mouse pointer mobile message is split into two parts, and in the normal region, the information that mouse moves is normally sent to long-range application program.When mouse pointer moves in the buffering edge zone of rectangular edges scope, before receiving next time mouse move, last mobile information can be sent to remote application at certain intervals repeatedly, continues mobile information thereby allow the user to use mouse to send to specific direction.When the buffering edge zone, specific pattern will be presented near the mouse pointer focal position, occur with special mobile behavior of prompting user.

In service system shown in Figure 3, service managerZ-HU is responsible for controlling the situation of the application program of carrying out in whole microenvironments on all service nodes in the cloud system, and itself and customer data base are associated.Use a performed typical workflow of the user of cloud computing remote interaction procedure service as shown in figure 14.User's the terminal program at first front end of access system (usually with Web form) obtains by security gateway and the authority that is connected a certain microenvironment (1); Front end and service managerZ-HU be alternately opening an outlet at security gateway as user terminal, and according to the information of customer data base corresponding microenvironment is set.When user terminal connects this microenvironment, the tabulation of the application program that the user can carry out in microenvironment by service managerZ-HU by database information specified (2); Carry out in the process of specific application program the user, relevant information (contents of program of for example carrying out, execution time, the data that the user generates etc.) is sent back customer data base (3) by service managerZ-HU; Finally, complete when user's executive utility, the user leaves microenvironment and informs the information that service managerZ-HU withdraws from. before service managerZ-HU will be collected in the microenvironment required information synchronization to customer data base, for the user accesses prepare (4) next time.

In the remote interaction procedure service was used, network environment was experienced the user and has been caused the most direct restriction.Experience in order to obtain enough users, the user needs the network of a low delay and high bandwidth to the binding of computing node.Usually, the transmission delay of network should be lower than 50 milliseconds.In addition, test shows that under a rational level of resolution (usually being assumed to be 1024x768), the video flowing of transmission reaches 25-30FPS and keeps enough readability to require the bandwidth of minimum 2Mbps.This means when a cloud service system is served a large number of users at the same time that outlet will cause huge pressure to the flow of network to grid.

Such cloud service system requires a distributed deployment scheme so that the network bandwidth that effectively utilizes differentiation is to improve transfer efficiency and Cost reduction.As shown in figure 15, distributed calculation services node array is deployed in a plurality of positions in all parts of the country, the network egress of its served user terminals single line network that normally campus network is lower.The array of a plurality of deployment is connected with the distributed storage array of access on the backbone network by special line.All these distributed systems are all with high-quality but limited bandwidth is connected to central database.User's terminal at first from connecting a plurality of calculation services nodes and selecting connective best one, connects the microenvironment of certain node wherein with the executive utility service again behind the access web front-end.The subscriber data file that in user and application program reciprocal process, the produces direct-connected storage array of service node that to write direct.After user interactions finishes, service managerZ-HU will send request, (usually be less than 12 hours) in a specific time domain, and the data file that user interactions produces will be synchronized in whole storage arrays.A small amount of core data is then write customer data base in real time.

This distributed schemes is based on following some consideration: 1. the network egress at single calculation services node array place can't bear bandwidth load that numerous users bring and provide the JA(junction ambient) that meets the demands (bandwidth and the delay of minimum 2Mbps are lower than 50ms) for all users; 2. the user may produce quite a large amount of user data when carrying out an interactive program.For example, when the user carries out the game of a physical culture racing tip, can record a video and generate the data file that reaches hundreds of MB some scene in the game.Single storage array can't be in real time or the data of quasi real time finishing magnanimity with whole distributed array of services synchronous, still, and limited array service node array to make real-time data be possible synchronously; 3. because the restriction of geographical conditions, user's possible translational speed in network will be lower than and finish synchronous time threshold between the storage array.For example, suppose to have the storage array that is distributed in two areas in Beijing and Guangzhou, the user store into user data in the storage array of In Guangzhou Area can be in the longest 12 hours and the array of Beijing area to finish data synchronous.And in most cases, a user who finishes to serve at In Guangzhou Area (we suppose its corresponding nature person's individuality) can be in the new service of Beijing area application in 12 hours in existing communications and transportation level.This real time threshold makes above-mentioned distributed schemes become possibility.

Claims (9)

1. computer processing unit between application program and the API of system or network, is characterized in that: it comprises that sound derives that module, interface derive module, other information derive modules and input imports module; It is video flowing output for the picture conversion that will play that module is derived at the interface, and described sound is derived module background music and effect sound are converted to audio stream output; Other information derive modules, with the position of cursor in the application program with pattern is derived and from an independently channel of information flow output.

2. processing unit as claimed in claim 1 is characterized in that: other information derive modules and directly the information of cursor are not merged in the program interface and transmit as video stream; But directly draw out by the information of supplemental stream transmission mouse pointer and in client.

3. processing unit as claimed in claim 2, it is characterized in that: when user's rolling mouse pointer, client terminal local will at first respond the event that mouse moves and repaint mouse pointer, and other information derive modules are sent to this event long-range execution simultaneously by control information application program.

4. processing unit as claimed in claim 1 is characterized in that: further comprise storage and an application context interface, described interface comprises file access API, registry access API, environmental variance and kernel objects etc.; Described processing unit is redirected the collision problem that can avoid resource to use when interface is carried out a plurality of identical application programs under same operating system; Described processing unit further comprises a record cell, is used for record and keeps these contextual informations, thereby prepare from the current context information and executing for application program next time.

5. processing unit as claimed in claim 1, it is characterized in that: described processing unit is connected with mixed-media network modules mixed-media with the video/audio compression module, with the interface of application program with play up data-switching and become video/audio stream, be sent to other terminal on the internet through mixed-media network modules mixed-media.

6. processing unit as claimed in claim 1, it is characterized in that: it further comprises a redirection module, described redirection module is by revising the parameter of API Calls, and then application program is handed to a specific hardware of deriving the module connection to the access of certain hardware device carry out, thereby the information that the program that is applied is exported from hardware device.

7. processing unit as claimed in claim 1, it is characterized in that: described processing unit further comprises a plurality of processes, described process is connected with client, carries out specific request by client-requested, and described request is the data of application program or transmission or reception; Described processing unit is written into internal memory with above-mentioned request, before the actual execution of above-mentioned request, redirection module is loaded the process space into application program, and revise the loaded run time version of application program in the memory headroom of program process.

8. processing unit as claimed in claim 1, it is characterized in that: described function address redirection module is a hardware, it changes the funcall entry position of the operating system application programming interfaces API of function; After application program was carried out, during routine call operating system API, redirection module jumped to the function address that redirection module is stored with the function address that is modified.

9. long-distance interactive system based on cloud computing, it comprises terminal device, web front-end, security gateway is characterized in that:

Described security gateway comprises one or more service nodes;

Described service node further comprises a kind of computer processing unit, and between application program and the API of system or network, described processing unit comprises that sound is derived module, the interface is derived module, other information derivation modules and input and imported module; It is video flowing output for the picture conversion that will play that module is derived at the interface, and described sound is derived module background music and effect sound are converted to audio stream output; Other information derive modules, with the position of cursor in the application program with pattern is derived and from an independently channel of information flow output;

Described terminal device is at first accessed described front end, by obtaining the authority that connects a certain service node in the security gateway after the checking; Terminal device uses this authority to be connected specifically with cloud computing system subsequently, and service node connects; In described security gateway, the interface of all application programs and audio frequency output are converted into Audio and Video stream, and a relative additional information stream, and pass to described terminal device by the internet.

Images